hSo TdZddlZddlZddlZddlZddlZddlZddlZddlZddl Zddl Zddl Z ddl m Z ej j#rddlmZnddlmZ ddlmZej2j4j7ddddd d d d d ZdZej<j?ej<jAe!dZ"da#ej2jHjK xrLej2jHjM xr%ej2jHjO Z(dZ)dde)zdddddfZ*e dejVfdejXfdejZfdej\fdej^fdej`fdejbfdejdffZ3Gd d!e4Z5d"Z6Gd#d$e7Z8ed0d%Z9d&Z:d'Z;dded(fd)Z<Gd*d+e7Z=d,Z>d-Z?d.Z@d/ZAy#e$r ddlm Z YwxYw#e$rddlZYwxYw)1a Information available about Tor from `its manual `_. This provides three methods of getting this information... * :func:`~stem.manual.Manual.from_cache` provides manual content bundled with Stem. This is the fastest and most reliable method but only as up-to-date as Stem's release. * :func:`~stem.manual.Manual.from_man` reads Tor's local man page for information about it. * :func:`~stem.manual.Manual.from_remote` fetches the latest manual information remotely. This is the slowest and least reliable method but provides the most recent information about Tor. Manual information includes arguments, signals, and probably most usefully the torrc configuration options. For example, say we want a little script that told us what our torrc options do... .. literalinclude:: /_static/example/manual_config_options.py :language: python | .. image:: /_static/manual_output.png | **Module Overview:** :: query - performs a query on our cached sqlite manual information is_important - Indicates if a configuration option is of particularly common importance. download_man_page - Downloads tor's latest man page. Manual - Information about Tor available from its manual. | |- from_cache - Provides manual information cached with Stem. | |- from_man - Retrieves manual information from its man page. | +- from_remote - Retrieves manual information remotely from tor's latest manual. | +- save - writes the manual contents to a given location .. versionadded:: 1.5.0 N) OrderedDict) lru_cacheGENERALCLIENTRELAY DIRECTORY AUTHORITYHIDDEN_SERVICEDENIAL_OF_SERVICETESTINGUNKNOWNz9https://gitweb.torproject.org/tor.git/plain/doc/tor.1.txtzcached_manual.sqlitez$CREATE TABLE schema(version INTEGER)z'INSERT INTO schema(version) VALUES (%i)zdCREATE TABLE metadata(name TEXT, synopsis TEXT, description TEXT, man_commit TEXT, stem_commit TEXT)zACREATE TABLE commandline(name TEXT PRIMARY KEY, description TEXT)z=CREATE TABLE signals(name TEXT PRIMARY KEY, description TEXT)z;CREATE TABLE files(name TEXT PRIMARY KEY, description TEXT)zCREATE TABLE torrc(key TEXT PRIMARY KEY, name TEXT, category TEXT, usage TEXT, summary TEXT, description TEXT, position INTEGER)zGENERAL OPTIONSzCLIENT OPTIONSzSERVER OPTIONSzDIRECTORY SERVER OPTIONSz"DIRECTORY AUTHORITY SERVER OPTIONSzHIDDEN SERVICE OPTIONSz$DENIAL OF SERVICE MITIGATION OPTIONSzTESTING NETWORK OPTIONSc"eZdZdZfdZxZS)SchemaMismatchz Database schema doesn't match what Stem supports. .. versionadded:: 1.6.0 :var int database_schema: schema of the database :var tuple supported_schemas: schemas library supports cHtt| |||_||_yN)superr__init__database_schemalibrary_schema)selfmessagerr __class__s U/var/www/betterdocs.net/sherlock_api/venv/lib/python3.12/site-packages/stem/manual.pyrzSchemaMismatch.__init__ys" .$(1*D(D)__name__ __module__ __qualname____doc__r __classcell__)rs@rrros))rrctjjs tdddl}t |j tat j||S)a, Performs the given query on our sqlite manual cache. This database should be treated as being read-only. File permissions generally enforce this, and in the future will be enforced by this function as well. :: >>> import stem.manual >>> print(stem.manual.query('SELECT description FROM torrc WHERE key=?', 'CONTROLSOCKET').fetchone()[0]) Like ControlPort, but listens on a Unix domain socket, rather than a TCP socket. 0 disables ControlSocket. (Unix and Unix-like systems only.) (Default: 0) .. versionadded:: 1.6.0 :param str query: query to run on the cache :param list param: query parameters :returns: :class:`sqlite3.Cursor` with the query results :raises: * **ImportError** if the sqlite3 module is unavailable * **sqlite3.OperationalError** if query fails z$Querying requires the sqlite3 modulerN) stemprereqis_sqlite_available ImportErrorsqlite3DATABASEconnect CACHE_PATHexecute)queryparamr&s rr+r+sL0  ( ( * < ==z*H   % ''rcFeZdZdZej dddfdZdZdZdZ y) ConfigOptiona Tor configuration attribute found in its torrc. :var str name: name of the configuration option :var stem.manual.Category category: category the config option was listed under, this is Category.UNKNOWN if we didn't recognize the category :var str usage: arguments accepted by the option :var str summary: brief description of what the option does :var str description: longer manual description with details cJ||_||_||_||_||_yrnamecategoryusagesummary description)rr2r3r4r5r6s rrzConfigOption.__init__s&DIDMDJDL"Drc Ntjj|ddddddS)Nr2r3r4r5r6Tcacher"util _hash_attrrs r__hash__zConfigOption.__hash__s' 99  fj'9mei  jjrcTt|trt|t|k(SdSNF) isinstancer.hashrothers r__eq__zConfigOption.__eq__s#(25,(G4:e $RURrc||k( SrrCs r__ne__zConfigOption.__ne__u} rN) rrrrCategoryr rr>rErHrGrrr.r.s/ '/&6&6b`b#kSrr.c tjjj}tj j tj jtd} |j|t|jDcgc]9}|jds|r|jn||j|f;c}}|jdgdDcgc]}|r|jn|c}|d<|Scc}wcc}w#t$r;}tjj j#d|d|icYd}~Sd}~wwxYw) aM Provides a dictionary for our settings.cfg. This has a couple categories... * manual.important (list) - configuration options considered to be important * manual.summary.* (str) - summary descriptions of config options :param bool lowercase: uses lowercase keys if **True** to allow for case insensitive lookups z settings.cfgzmanual.summary.manual.importantT)multiplez?BUG: stem failed to load its internal manual information from '': N)r"r;confConfigospathjoindirname__file__loaddictkeys startswithlower get_value Exceptionlogwarn) lowercaseconfig config_pathkey config_dictr2excs r_configres0 99>> "& RWW__X6G+ KK ^d^i^i^kRWZoro}o}PpQ sF %sr1z modified z (z) => z removing config option => %sz adding %s => %sz removing %s => %s )getattrappenditemsgetsetrX differencerS) previous_manual new_manuallinesattr previous_attrnew_attr config_name config_attrprevious added_items removed_items added_item removed_items r_manual_differencesrs %n!dOT2Mz4(H  ll6=> 4 4 .>? -89 # #(0(8 k $K"&&{3(   LL;kI Jh&O kd+wx/FF ;gVacgNhij k  k}1134?? P GK ,,7+E F G(..*+66}7J7J7LM M//12==hnn>NO % 9J ,,*Z7 8 9* =L ,,,|; < = ll4C!F 5 rc<|jtdvS)z Indicates if a configuration option of particularly common importance or not. :param str option: tor configuration option to check :returns: **bool** that's **True** if this is an important option and **False** otherwise rL)rZre)options r is_importantrs 79%78 88rc|s |s tdtjjj ds t dt j}tjj|d}tjj|d} t|d5}tj||}tj||ddd tjjj'd|ztjj)|s t+d |rj tjj3|} tjj)| stj4| tj6|||r;t|d5} tj| ||j9dddtj:|y#1swY!xYw#t!j"d d \} } d |d |d | } tj$|| | | xYw#tjjj,$r*} t d| j.d| j0d} ~ wwxYw#t*$r} t | d} ~ wwxYw#1swYxYw#tj:|wxYw)a# Downloads tor's latest man page from `gitweb.torproject.org `_. This method is both slow and unreliable - please see the warnings on :func:`~stem.manual.Manual.from_remote`. :param str path: path to save tor's man page to :param file file_handle: file handler to save tor's man page to :param str url: url to download tor's asciidoc manual from :param int timeout: seconds to wait before timing out the request :raises: **IOError** if unable to retrieve the manual z?Either the path or file_handle we're saving to must be provideda2xz2We require a2x from asciidoc to provide a man pagez tor.1.txtztor.1wb)timeoutNrz%Unable to download tor's manual from z to : za2x -f manpage %szno man page was generatedUnable to run 'rNrb) ValueErrorr"r;system is_availableIOErrortempfilemkdtemprQrRrSopenurlliburlopenshutil copyfileobjsysexc_infoDownloadFailedcallexistsOSError CallErrorcommandstderrrTmakedirscopyfileflushrmtree)rR file_handleurlrdirpath asciidoc_path manual_path asciidoc_filerequestrd stacktracerpath_dir manual_files rdownload_man_pagers; k V WW 99   ( ( / F GG    ''',,w 4- Wg.+"?  t $3 ..87M23J ii/-?@ WW^^K (122)  77??4(ww~~h' ++h  T*  T "k; 4 MM'A33? q+oc:GJM[^_g   S*g >> 99   % %J  SZZH IIJcl MM's H&.G3HAI3J>7A)J J>.'J2J>3G=8H?H??J>'J)%JJJ> J/ J**J//J>2J;7J>>KceZdZdZdZeddZedZedZeddZ eddZ d Z d Z d Z d Zd ZdZy)Manuala Parsed tor man page. Tor makes no guarantees about its man page format so this may not always be compatible. If not you can use the cached manual information stored with Stem. This does not include every bit of information from the tor manual. For instance, I've excluded the 'THE CONFIGURATION FILE FORMAT' section. If there's a part you'd find useful then `file an issue `_ and we can add it. :var str name: brief description of the tor command :var str synopsis: brief tor command usage :var str description: general description of what tor does :var collections.OrderedDict commandline_options: mapping of commandline arguments to their descripton :var collections.OrderedDict signals: mapping of signals tor accepts to their description :var collections.OrderedDict files: mapping of file paths to their description :var collections.OrderedDict config_options: :class:`~stem.manual.ConfigOption` tuples for tor configuration options :var str man_commit: latest tor commit editing the man page when this information was cached :var str stem_commit: stem commit to cache this manual information c||_||_||_t||_t||_t||_t||_d|_d|_ d|_ yr) r2rgr6rrhrirjrk man_commit stem_commitschema)rr2rgr6rhrirjrks rrzManual.__init__usaDIDM"D*+>?Dw'DLU#DJ%n5DDODDKrNc|t}|&|jdrtj|Stj |S)ax Provides manual information cached with Stem. Unlike :func:`~stem.manual.Manual.from_man` and :func:`~stem.manual.Manual.from_remote` this doesn't have any system requirements, and is faster too. Only drawback is that this manual content is only as up to date as the Stem release we're using. .. versionchanged:: 1.6.0 Added support for sqlite cache. Support for :class:`~stem.util.conf.Config` caches will be dropped in Stem 2.x. :param str path: cached manual content to read, if not provided this uses the bundled manual information :returns: :class:`~stem.manual.Manual` with our bundled manual information :raises: * **ImportError** if cache is sqlite and the sqlite3 module is unavailable * **IOError** if a **path** was provided and we were unable to read it or the schema is out of date .sqlite)r)endswithr_from_sqlite_cache_from_config_cache)rRs r from_cachezManual.from_cachesA6 | d DMM)4  & &t ,,  & &t ,,rc tjjs tdddl}t j j|std|z|j|5} |jdjd}|tk7rtdtd|d||tf|jdj\}}}}}t|jd j!} t|jd j!} t|jd j!} t#} |jdj!D]}|\}}}}}t%|||||| |<t'|||| | | | }||_||_||_|cdddS#|j$r} td |d | d} ~ wwxYw#1swYyxYw)Nz2Reading a sqlite cache requires the sqlite3 modulerz%s doesn't existzSELECT version FROM schemaz(Stem's current manual schema version is z, but z was version zISELECT name, synopsis, description, man_commit, stem_commit FROM metadataz&Failed to read database metadata from rz)SELECT name, description FROM commandlinez%SELECT name, description FROM signalsz#SELECT name, description FROM fileszOSELECT name, category, usage, summary, description FROM torrc ORDER BY position)r"r#r$r%r&rQrRrrr(r*fetchoneSCHEMA_VERSIONrOperationalErrorrWfetchallrr.rrrr)rRr&connrr2rgr6rrrd commandlinerirjrkentryrr3r4r5option_descriptionmanuals rrzManual._from_sqlite_caches  ;; * * , L MM 77>>$  &- ..  $T:;DDFqI ^ #ftvz}C DFLO]N_` `?C||MX@Y@b@b@d<h Z&QR[[]^kT\\"IJSSUVg4<< EFOOQRe"}n<< qr{{}d%?D<%*> "DIIdI' ]Nyy{   ) *iinQ n $ , HH-3R 8 HH1C7 < HH.4b 9 HH036 ; HH4s:B ? ! .    hhvr hhz2 hh}b! hh$km4 hhy+-( hhw &Ft4F-6F Mrc Pdtrdndd|} tjjj |ddi}t|t}}tjD]"\}}t|||j|g$|D]M}|jd s|tvs|d vs#t|tj|j|gOt!t#|jd gt#|jdgt#|jdgt%|jdgt%|jdgt%|jdg|S#t $r}t d|d |d }~wwxYw)a Reads and parses a given man page. On OSX the man command doesn't have an '--encoding' argument so its results may not quite match other platforms. For instance, it normalizes long dashes into '--'. :param str man_path: path argument for 'man', for example you might want '/path/to/tor/doc/tor.1' to read from tor's git repository :returns: :class:`~stem.manual.Manual` for the system's man page :raises: **IOError** if unable to retrieve the manual zman z--encoding=asciir/z -P cat MANWIDTH10000000)envrrNNz OPTIONS)COMMAND-LINE OPTIONSzNON-PERSISTENT OPTIONSNAMESYNOPSIS DESCRIPTIONrSIGNALSFILES)HAS_ENCODING_ARGr"r;rrrr_get_categoriesrCATEGORY_SECTIONSro_add_config_optionsrprrJr r _join_lines_get_indented_descriptions) man_pathman_cmd man_outputrd categoriesrkcategory_header category_enumr3s rfrom_manzManual.from_mans";K$6PR$RT\]G?99##((Z8P(Qj"1!Z[\ *..,-*..R01*..34 0F!KL  2!>? !<=  ? = >>?s-F F%F  F%ctj5}t||tj |j cdddS#1swYyxYw)a Reads and parses the latest tor man page `from gitweb.torproject.org `_. Note that while convenient, this reliance on GitWeb means you should alway call with a fallback, such as... :: try: manual = stem.manual.from_remote() except IOError: manual = stem.manual.from_cache() In addition to our GitWeb dependency this requires 'a2x' which is part of `asciidoc `_ and... isn't quick. Personally this takes ~7.41s, breaking down for me as follows... * 1.67s to download tor.1.txt * 5.57s to convert the asciidoc to a man page * 0.17s for stem to read and parse the manual :param int timeout: seconds to wait before timing out the request :returns: latest :class:`~stem.manual.Manual` available for tor :raises: **IOError** if unable to retrieve the manual )rrN)rNamedTemporaryFilerrrr2)rtmps r from_remotezManual.from_remotesA<  $ $ &'#cW= __SXX &'''s ,A  Ach|jdr|j|S|j|S)a Persists the manual content to a given location. .. versionchanged:: 1.6.0 Added support for sqlite cache. Support for :class:`~stem.util.conf.Config` caches will be dropped in Stem 2.x. :param str path: path to save our manual content to :raises: * **ImportError** if saving as sqlite and the sqlite3 module is unavailable * **IOError** if unsuccessful r)r_save_as_sqlite_save_as_config)rrRs rsavez Manual.save:s2$ }}Y  ! !$ ''  ! !$ ''rc tjjs tdddl}|dz}t j j|rt j||j|5}tD]}|j||jd|j|j|j|j|j f|j"j%D]\}}|jd||f|j&j%D]\}}|jd||f|j(j%D]\}}|jd||ft+|j,j/D]h\}}|jd|jj1|j|j2|j4|j6|j|fj dddt j j|rt j|t j8||y#1swYTxYw) Nz1Saving a sqlite cache requires the sqlite3 modulerz.newz]INSERT INTO metadata(name, synopsis, description, man_commit, stem_commit) VALUES (?,?,?,?,?)z7INSERT INTO commandline(name, description) VALUES (?,?)z3INSERT INTO signals(name, description) VALUES (?,?)z1INSERT INTO files(name, description) VALUES (?,?)zdINSERT INTO torrc(key, name, category, usage, summary, description, position) VALUES (?,?,?,?,?,?,?))r"r#r$r%r&rQrRrremover(SCHEMAr*r2rgr6rrrhrorirj enumeraterkvaluesupperr3r4r5rename) rrRr&tmp_pathrcmdkvis rrzManual._save_as_sqliteQs ;; * * , K LLf}H ww~~hii  "Id# S llruyu~u~AEANANPTP`P`bfbqbqswsCsCuDE**002X$!Q NQRTUPVWX,,$$&T$!Q JQPQFST**""$R$!Q H1a&QRD//6689I$!Q {~EEKKMOPOUOUWXWaWacdcjcjlmluluwxwDwDFG~H III$ ww~~diioIIh+IIs 8FI""I+ctjjj}|j d|j |j d|j |j d|j|jr|j d|j|jr|j d|j|jjD]\}}|j d|d|d  |jjD]\}}|j d |d|d  |jjD]\}}|j d |d|d  |jjD]\}}|j d |z|j|j d |z|j |j d|z|j |j d|z|j"|j d|z|j|j%|y)Nr2rgr6rrrhz => F) overwriterirjrrrrr)r"r;rOrPrqr2rgr6rrrhrorirjrkr3r4r5r)rrRrOrrs rrzManual._save_as_configrs 99>> "DHHVTYYHHZ'HH]D,,-  hh|T__-  hh}d../((..0N1 hh$Aq&9uhMN ""$B1 hhy1-5hAB   "@1 hhwa+h?@##))+C1 hh+a/< hh'!+QVV4 hh(1,agg6 hh*Q. : hh.2AMMB C IIdOrc Rtjj|dddddddd S) Nr2rgr6rhrirjrkTr8r:r=s rr>zManual.__hash__s= 99  fj-I^`ikruEOS  TTrcTt|trt|t|k(SdSr@)rArrBrCs rrEz Manual.__eq__s#(25&(A4:e $LuLrc||k( SrrGrCs rrHz Manual.__ne__rIrr)tor)<)rrrrr staticmethodrrrrrrrrr>rErHrGrrrrZs4  - -D##J  D((T''B(.B<TMrrc2|r d|dvr|dd}|r|djdr|dd}t}dg}}|D]}tjj rt nt }|j|ddj|d d j|d d }|r:|jd s)|r|r |ddk(r|dd}|||<|jg}}|jdr|dd}|j||r|||<|S)z The man page is headers followed by an indented section. First pass gets the mapping of category titles to their lines. zTOR(1)rrNTori 'i -* r/z ) rYrr"r# is_python_3chrunichrreplacestriprn)contentrr3rulinechar_fors rrrs) WQZ'abkG ''.crlG}*"E(dkk--/sVH <<(# . 6 6x7G M U UV^_cVdfi jD DOOC(  U2Y"_*%$ 8 bh  #ABx ll4-0 Jx rc Ttd}}|D]R}|dk(rd} |r|jds g|c||<}'|s*|jds<||j|ddTt|j Dcgc]\}}|s |dj |fc}}Scc}}w)a Parses the commandline argument and signal sections. These are options followed by an indented description. For example... :: -f FILE Specify a new configuration file to contain further Tor configuration options OR pass - to make Tor read its configuration from standard input. (Default: /usr/local/etc/tor/torrc, or $HOME/.torrc if that file is not found) There can be additional paragraphs not related to any particular argument but ignoring those. Nz Noter  )rrYrnrWrorS)ruoptionslast_argrarg desc_liness rrrs""mT8')d zh dooc* "DgdmX doof- htABx( ) '--/`sJU_SXXj)*` aa`s < B$ B$ czfdt|Dcgc]\}}d|vsd|vs|}}}|r"|t|d}||jdd}dg}}|D]L}|r!|jds|r |||g}}&|jdr|dd}|j |N|r ||yycc}}w) ai Parses a section of tor configuration options. These have usage information, followed by an indented description. For instance... :: ConnLimit NUM The minimum number of file descriptors that must be available to the Tor process before it will start. Tor will ask the OS for as many file descriptors as the OS will allow (you can find this by "ulimit -H -n"). If this number is less than ConnLimit, then Tor will refuse to start. You probably don't need to adjust this. It has no effect on Windows since that platform lacks getrlimit(). (Default: 1000) c :d|vryd|vr |jdD] }|| yd|vr|jddn|df\}}tjd|jzd}t |||t |j |<y)NzPER INSTANCE OPTIONSz, r rr/zmanual.summary.%s)rrerprZr.rr) titler6subtitler2r4r5 add_optionr3rks rrz'_add_config_options..add_options&  u} kk$'*(8[)*,/%r9s[-^  0%;;&&(!+! 99>>  y(G[+Wgi|HJS TO WW\\"''//(35K L yy''..00z9I9I9P9P9R5Rz[_[d[d[k[k[x[x[zWz(+n<hEA?E  h&&'X__%X^^$x112'););<X445)8+E+EFh../ !  )W ) +(\6:  0*Z 9"=NZ\:z}V}@ ,^b<;(|!E0//0s#HH HH H'&H'