h|[dZddlZddlZddlZddlZddlZddlZddlmZmZm Z  ddl m Z  ddlmZdZdZej(j+ej(j-edZej2dZej2d Zej2d Zej2d Zd Zej2d Zej2dZ ej2dZ!ej2dZ"ej2dZ#dQdZ$dQdZ%Gdde&Z'Gdde'Z(Gdde'Z)dZ*e(dddddd !e(d"d#d$d%d&d'd()e(d*d+d$d%d,d-!e(d.d/d$d%d0d1d2)e(d3d4d$d%d5d6d78e(d9d:d%d$d;dd?d$d%d@dA!e(dBdCd$d%dDdE!e(dFdGd$d%dHdIdJ)e(dKdLdMdNdOd!dP Z+y#e $r ddl m Z YwxYw#e $rddlZYwxYw)Ra Directories that provide `relay descriptor information <../tutorials/mirror_mirror_on_the_wall.html>`_. At a very high level tor works as follows... 1. Volunteer starts a new tor relay, during which it sends a `server descriptor `_ to each of the directory authorities. 2. Each hour the directory authorities make a `vote `_ that says who they think the active relays are in the network and some attributes about them. 3. The directory authorities send each other their votes, and compile that into the `consensus `_. This document is very similar to the votes, the only difference being that the majority of the authorities agree upon and sign this document. The idividual relay entries in the vote or consensus is called `router status entries `_. 4. Tor clients (people using the service) download the consensus from an authority, fallback, or other mirror to determine who the active relays in the network are. They then use this to construct circuits and use the network. :: Directory - Relay we can retrieve descriptor information from | |- from_cache - Provides cached information bundled with Stem. | +- from_remote - Downloads the latest directory information from tor. | |- Authority - Tor directory authority +- Fallback - Mirrors that can be used instead of the authorities .. versionadded:: 1.7.0 N) connection str_tools tor_tools) OrderedDictzHhttps://gitweb.torproject.org/tor.git/plain/src/app/config/auth_dirs.inczLhttps://gitweb.torproject.org/tor.git/plain/src/app/config/fallback_dirs.inczcached_fallbacks.cfgz"(\S+) orport=(\d+) .*"z"v3ident=([\dA-F]{40}) "z"ipv6=\[([\da-f:]+)\]:(\d+) "z!"([\d\.]+):(\d+) ([\dA-F ]{49})",z /* ===== */z/\*\s+(\S+)=(\S*)\s+\*/z0"([\d\.]+):(\d+) orport=(\d+) id=([\dA-F]{40}).*z/\* nickname=(\S+) \*/z/\* extrainfo=([0-1]) \*/z" ipv6=\[([\da-f:]+)\]:(\d+)"ci}|D]X}|D]Q}|jtj|}|s*|j}t |dkDr|n|d||<SZ|r(|D]#}||vst ddj |z|S)a Scans the given content against a series of regex matchers, providing back a mapping of regexes to their capture groups. This maping is with the value if the regex has just a single capture group, and a tuple otherwise. :param list lines: text to parse :param list regexes: regexes to match against :param list required: matches that must be in the content :returns: **dict** mapping matchers against their capture groups :raises: **ValueError** if a required match is not present rz(Failed to parse mandatory data from: %s )searchr _to_unicodegroupslen ValueErrorjoin) linesregexesrequiredmatcheslinematcherm match_groupsrequired_matchers X/var/www/betterdocs.net/sherlock_api/venv/lib/python3.12/site-packages/stem/directory.py _match_withrQs 'VdV ....t4 5a xxz +.|+`_. Note that while convenient, this reliance on GitWeb means you should alway call with a fallback, such as... :: try: authorities = stem.directory.Authority.from_remote() except IOError: authorities = stem.directory.Authority.from_cache() .. versionadded:: 1.5.0 .. versionchanged:: 1.7.0 Support added to the :class:`~stem.directory.Authority` class. :param int timeout: seconds to wait before timing out the request :returns: **dict** of **str** identifiers to their :class:`~stem.directory.Directory` :raises: **IOError** if unable to retrieve the fallback directories r:r;timeouts r from_remotezDirectory.from_remotes6 k llrc Ltjj|ddddddS)Nr/r1r2r3r4r5)stemutil _hash_attrr6s r__hash__zDirectory.__hash__s% 99  iJ Wacn oorcTt|trt|t|k(SdSNF)r+r"hashr6others r__eq__zDirectory.__eq__#(25)(D4:e $O%Orc||k( Srr=rLs r__ne__zDirectory.__ne__u} rN<) __name__ __module__ __qualname____doc__r8 staticmethodr>rBrHrNrQr=rrr"r"zsJ.N:mm"mm8pPrr"cheZdZdZd fd ZedZed dZedZdZ dZ dZ xZ S) Authoritya Tor directory authority, a special type of relay `hardcoded into tor `_ to enumerate the relays in the network. .. versionchanged:: 1.3.0 Added the is_bandwidth_authority attribute. .. versionchanged:: 1.7.0 Added the orport_v6 attribute. .. deprecated:: 1.7.0 The is_bandwidth_authority attribute is deprecated and will be removed in the future. :var str v3ident: identity key fingerprint used to sign votes and consensus c tt| |||||||r1tj|s|r|d|dn|} t | d|||_||_y)Nr$r%z has an invalid v3ident: )superr[r8rr)rv3identis_bandwidth_authority) r6r/r1r2r3r4r5r^r_r7 __class__s rr8zAuthority.__init__sZ )T#GWh XW`ay55g>:B X6 j *gN OODL"8Drc ttSr)dictDIRECTORY_AUTHORITIESr=rrr>zAuthority.from_caches % &&rc  tjtjt|j j }|s td i}t|tjtt t"t$ftt$fD]~}|j't\}}|j't$\} } } t| || | j)dd ||j't"|j't  ||< |S#tjdd\}}dtd|}tjt|||xYw#t*$r}tt-|d}~wwxYw) Nr@ no contentrz4Unable to download tor's directory authorities from : r )r/r1r2r3r4r5r^)rr urlliburlopenGITWEB_AUTHORITY_URLread splitlinesIOErrorsysexc_inforDDownloadFailedr r[ _pop_sectionAUTHORITY_NAMEAUTHORITY_V3IDENTAUTHORITY_IPV6AUTHORITY_ADDRgetreplacerr-) rArexc stacktracemessageresultsrr4r1r/r2r3s rrBzAuthority.from_remote srP##FNN3GSZ$[$`$`$bcnnpe l##g'y/E/EXiky|JHKYgiwXxy  '#KK7')0^)D&;%#++C4kk.1KK 12   N=P q+oc:Pdfijg    4c:w OO0  CH s&ADB>E#AE # F,FFcg}|rm|j|jd|rK|djdr7|j|jd|r|djdr7|S)z, Provides the next authority entry. rri)appendpop startswith)r section_liness rrtzAuthority._pop_section0sj M 599Q<( eAh))#.UYYq\* eAh))#. rcRtjj|ddtdS)Nr^r_TparentcacherDrErFr"rGs rrHzAuthority.__hash__@s% 99  i1IT]gk  llrcTt|trt|t|k(SdSrJ)r+r[rKrLs rrNzAuthority.__eq__CrOrc||k( Srr=rLs rrQzAuthority.__ne__FrRr)NNNNNNNFrS) rUrVrWrXr8rYr>rBrtrHrNrQ __classcell__r`s@rr[r[s[$9''$$L  mPrr[c~eZdZdZd fd Zed dZed dZedZee fdZ dZ dZ d Z xZS) Fallbacka Particularly stable relays tor can instead of authorities when bootstrapping. These relays are `hardcoded in tor `_. For example, the following checks the performance of tor's fallback directories... :: import time from stem.descriptor.remote import get_consensus from stem.directory import Fallback for fallback in Fallback.from_cache().values(): start = time.time() get_consensus(endpoints = [(fallback.address, fallback.dir_port)]).run() print('Downloading the consensus took %0.2f from %s' % (time.time() - start, fallback.fingerprint)) :: % python example.py Downloading the consensus took 5.07 from 0AD3FA884D18F89EEA2D89C019379E0E7FD94417 Downloading the consensus took 3.59 from C871C91489886D5E2E94C13EA1A5FDC4B6DC5204 Downloading the consensus took 4.16 from 74A910646BCEEFBCD2E874FC1DC997430F968145 ... .. versionadded:: 1.5.0 .. versionchanged:: 1.7.0 Added the has_extrainfo and header attributes which are part of the `second version of the fallback directories `_. :var bool has_extrainfo: **True** if the relay should be able to provide extrainfo descriptors, **False** otherwise. :var collections.OrderedDict header: metadata about the fallback directory file this originated from c tt| ||||||||_|rt ||_yt |_yr)r]rr8 has_extrainforheader) r6r/r1r2r3r4rr5rr`s rr8zFallback.__init__qs; (D"7GX{HV_`&D)/+f%DK[]DKrc |t}tjjj }|j |t |jDcgc]:}|jds|jddd|j|f<c}}i}t|jDcgc]}|jddc}D]}|dvri}dD]<}|d|}|j|||<||r$|dvs)td|d t|d r|d r|d t|d f} nd} t|d t|d t|d||d|ddk(| |||<|Scc}wcc}w)Nzheader..rr) tor_commit stem_commitr)r/r1r2r4rorport6_address orport6_port)r4rrr'z' is missing from rrr/r1r2r4rtruer/r1r2r3r4rr5r)FALLBACK_CACHE_PATHrDrEconfConfigloadrkeysrsplitrysetrpr0r) pathrkheadersr~keyr3attr attr_namer5s rr>zFallback.from_cachevs |" 99>> "DIIdOdiikmUVUaUabkUlAGGCOA. <mnGGE#CIIcN1-EF ? ? dyP)$i0((3-YI94t#tS:MNO O P  T.%9+,c$~2F.GH  %y/d9o&tJ'(! #_-7 gk%: NCnFsF0(F8Fc  tjtjt|j j }|s td |ddk7rtd tzi}tj|D]J}tj|}|r$|j!d ||j!d<>td |ztj| i}t#|tjt$t&t(t*ft$f D]t} | t$\} } } } t| t-| t-| | | j/t&| j/t(d k(| j/t*||| <v |S#tjdd\}}dtd|}tjt|||xYw#t0$r}tt3|d}~wwxYw)Nr@rerrfz3Unable to download tor's fallback directories from rgrz/* type=fallback */zJ%s does not have a type field indicating it is fallback directory metadatar&z,Malformed fallback directory header line: %srh1r)rr rkrlGITWEB_FALLBACK_URLrnrorprqrrrDrsrrtFALLBACK_MAPPINGmatchgroupr FALLBACK_ADDRFALLBACK_NICKNAMEFALLBACK_EXTRAINFO FALLBACK_IPV6r0ryrr-)rArr{r|r}rrmappingr~rr/r2r1r3s rrBzFallback.from_remotesO##FNN3FRY$Z$_$_$abmmoe l## Qx(( `cvv ww F%%e,M &&t,g #*==#3w}}Q DtKLL M % g'x/D/D}Vgi{~KGLZgYij  '29-2H/7K'L]#[[!23!++&89S@kk-0    NaO q+oc:Obdghg    3S*g NNT  CH s&AF .B/G' AG$' H 0HH cg}|r]|jd}|rJ|tk7rA|jdk7r|j||jd}|r |tk7rA|S)z Provides lines up through the next divider. This excludes lines with just a comma since they're an artifact of these being C strings. r,)r FALLBACK_DIVstripr)rrrs rrtzFallback._pop_sections`M YYq\d dl* ::<3    t $yy| dl* rctjjj}|j d||j d||j D]\}}|j d|z|t |jdD]#}|j} |j d| z|j|j d| zt|j|j d| zt|j|j d | z|j|j d | z|jrd nd |js|j d | zt|jd|j d| zt|jd&|j!|y)a Persists fallback directories to a location in a way that can be read by from_cache(). :param dict fallbacks: mapping of fingerprints to their fallback directory :param str tor_commit: tor commit the fallbacks came from :param str stem_commit: stem commit the fallbacks came from :param dict headers: metadata about the file these came from :param str path: location fallbacks will be persisted to rrz header.%sc|jSr)r3)xs rz!Fallback._write..s  r)rz %s.addressz %s.or_portz %s.dir_portz %s.nicknamez%s.has_extrainforfalsez%s.orport6_addressrz%s.orport6_portrN)rDrErrritemssortedvaluesr3r/r-r1r2r4rr5save) fallbacksrrrrrrv directoryr3s r_writezFallback._writes{ 99>> "DHH\:&HH]K( #1 hh{Q"#I,,.6MN O ))k hh|k)9+<+<= hh|k)3y/@/@+AB hh}{*C 0B0B,CD hh}{*I,>,>? hh!K/9;R;RX_`    % 3S9L9LQ9O5PQ "[0#i6I6I!6L2MN O IIdOrcRtjj|ddtdS)NrrTrrrGs rrHzFallback.__hash__s# 99  ox)]a  bbrcTt|trt|t|k(SdSrJ)r+rrKrLs rrNzFallback.__eq__s#(25((C4:e $NNrc||k( Srr=rLs rrQzFallback.__ne__rRr)NNNNNFNNrrS)rUrVrWrXr8rYr>rBrtrrrHrNrQrrs@rrrJsv$LC &&P66p&ATBcOrrc g}t|jj|j}t|jj|j}|D]}||}|jrd|jznd}|d|jzd|j zd|j zd|jzd|jzd|jzd |zd gz }|D]}|jd |z|D]]}||vs||vr ||}||} || k7sd D]=} t|| } t| | } | | k7s!|jd | d|d| d| ?_dj|S)z@ Provides a description of how fallback directories differ. z%s:%sz[none]z'* Added %s as a new fallback directory:z address: %sz or_port: %sz dir_port: %sz nickname: %sz has_extrainfo: %sz orport_v6: %srjz$* Removed %s as a fallback directory)r/r1r2r3r5z* Changed the z of z from z to r ) rr differencer5r3r/r1r2r4rrgetattrr) previous_directoriesnew_directoriesradded_fp removed_fpfprr5previous_directory new_directoryrold_attrnew_attrs r_fallback_directory_differencesrs % %%' ( 3 34H4M4M4O P(',,./::?;O;O;QR*  b#I1:1D1D)---(I /)2G2GG ))) )))++++++i555)#  E  >b LL7"<=>  `b X~z)-b1#B'M]*P`$-t4=$/ x  ,,r8U]^ _ ` ` 5 rmoria1z 128.31.0.39i#i#(9695DFC35FFEB861329B9F1AB04C46397020CE31(D586D18309DED4CD6D57C18FDB97EFA96D330566)r4r/r1r2r3r^tor26z 86.59.21.38P(847B1F850344D7876491A54892F904934E4EB85D)z2001:858:2:2:aabb:0:563b:1526r(14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4)r4r/r1r2r3r5r^dizumz 45.66.33.45(7EA6EAD6FD83083C538F44038BBFA077587DD755(E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58gabelmooz131.188.40.189(F2044413DAC2E02E3D6BCF4735A19BCA1DE97281)z2001:638:a000:4140::ffff:189r(ED03BB616EB2F60BEC80151114BB25CEF515B226 dannenbergz193.23.244.244)z2001:678:558:1000::244r(7BE683E65D48141321C5ED92F075C55364AC7123(0232AF901C31A04EE9848595AF9BB7620D4C5B2E)r4r/r1r2r5r3r^maatuskaz 171.25.193.9(BD6A829255CB08E66FBE7D3748363586E46B3810)z2001:67c:289c::9r(49015F787433103580E3B66A1707A00E60F2D15B Faravaharz154.35.175.225(CF6D0AAFB385BE71B8E111FC5CFF4B47923733BC(EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97longclawz 199.58.81.140(74A910646BCEEFBCD2E874FC1DC997430F968145(23D15D965BC35114467363C165C4F724B64B4F66bastetz204.13.164.118(24E2F139121D4394C54B5BCC368B3B411857C413)z2620:13:4000:6000::1000:118r(27102BC123E7AF1D4741AE047E160C91ADC76B21Sergez 66.111.2.131i)#iF#(BA44A889E64B93FAA2B114E02C2A279A8555C533) rrrrrrrrrrr),rXosrerqrD stem.utilstem.util.confrrr collectionsr ImportErrorstem.util.ordereddicturllib.requestrequestrkurllib2rmrrrdirname__file__rcompilerurvrwrxrrrrrrrr objectr"r[rrrcr=rrrs#J 660%!bdggll277??8#<>TU78BJJ:;?@CD 2::=> RS BJJ:;RZZ => >? B+kk\_ _DKyK\+^ <8   <68  <8  <58/<8<(8<8 <8  <48  <  ]Va0//0 s"GG GG G&%G&