ff  dZdZddlmZddlmZmZddlZddl Z ddl m Z ddl Z ddlZddlZddlTddlmZejd ejZejd ejZddlZejgd zZ[Gd d ejd dZdZdZdZdZdZeedezzeezZdZdZ dZ!dZ"dZ#ee e!e"e#Z$dZ%eddee%zZ&de%zZ'ee&e'Z(ede(dzZ)ee)e(e$Z*d Z+e j,d!Z-ee+Z.d"Z/d#Z0d$Z1d%Z2ee.d&ze.d'zZ3ee.d(ze.d)zZ4ee5ej6e7ed*+Z8ed,e8Z9ee*e9e4eZ:ee:zZ;ee.d-zed.dze.d/zed0dzZ<ed1ee3Z=eee=e*e9eiZ?e+D]"Z@e/e?e@d.z<e0e?e@d0z<e1e?e@d&z<e2e?e@d'z<#[@eAZBeAZCe+D]FZDeDd0zeDd.zfD]ZEeBFeEeDd'zeDd&zfD]ZEeCFeEG[D[Ed2ZGGd3d4eHZIGd5d6eHZJGd7d8ZKd9ZLd:ZMd;ZNd<Zd=ZOd>ZPd?ZQd@ZRdAZSeTdBkr eRdSdS)CzKa-Ping Yee zpGvR, ESR, Tim Peters, Thomas Wouters, Fred Drake, Skip Montanaro, Raymond Hettinger, Trent Nelson, Michael Foord)open)lookupBOM_UTF8N) TextIOWrapper)*)EXACT_TOKEN_TYPESz&^[ \t\f]*#.*?coding[:=][ \t]*([-\w.]+)s^[ \t\f]*(?:[#\r\n]|$))tokenizegenerate_tokensdetect_encoding untokenize TokenInfoc*eZdZdZedZdS)r cld|jt|jfz}d||zS)Nz%d (%s)z8TokenInfo(type=%s, string=%r, start=%r, end=%r, line=%r))type)rtok_name_replace)selfannotated_types //opt/alt/python311/lib64/python3.11/tokenize.py__repr__zTokenInfo.__repr__/s8"di$)1D%EEJ > 223 4cp|jtkr |jtvrt|jS|jSN)rOPstringrrs r exact_typezTokenInfo.exact_type4s/ 9??t{.???$T[1 19 rN)__name__ __module__ __qualname__rpropertyrrrr r .s>444 Xrr ztype string start end linec8dd|zdzS)N(|))joinchoicess rgroupr*;sC#((7"3"33c99rct|dzS)Nrr*r(s ranyr-<s%/C//rct|dzS)N?r,r(s rmayber0=sE7Oc11rz[ \f\t]*z #[^\r\n]*z\\\r?\nz\w+z0[xX](?:_?[0-9a-fA-F])+z0[bB](?:_?[01])+z0[oO](?:_?[0-7])+z(?:0(?:_?0)*|[1-9](?:_?[0-9])*)z[eE][-+]?[0-9](?:_?[0-9])*z)[0-9](?:_?[0-9])*\.(?:[0-9](?:_?[0-9])*)?z\.[0-9](?:_?[0-9])*z[0-9](?:_?[0-9])*z[0-9](?:_?[0-9])*[jJ]z[jJ]cgd}dh}|D]\}tj|D]E}tjd|DD]*}|d|+F]|S)N)brufbrfrc:g|]}||fSr")upper).0cs r z(_all_string_prefixes.._s$)D)D)DQ1aggii.)D)D)Dr) _itertools permutationsproductaddr')_valid_string_prefixesresultprefixtr4s r_all_string_prefixesrFTs>==TF(''(00 ' 'A ')D)D!)D)D)DE ' ' 2771::&&&& ' ' Mrc@tj|tjSr)recompileUNICODE)exprs r_compilerLcs :dBJ ' ''rz[^'\\]*(?:\\.[^'\\]*)*'z[^"\\]*(?:\\.[^"\\]*)*"z%[^'\\]*(?:(?:\\.|'(?!''))[^'\\]*)*'''z%[^"\\]*(?:(?:\\.|"(?!""))[^"\\]*)*"""z'''z"""z'[^\n'\\]*(?:\\.[^\n'\\]*)*'z"[^\n"\\]*(?:\\.[^\n"\\]*)*"T)reversez\r?\nz'[^\n'\\]*(?:\\.[^\n'\\]*)*'z"[^\n"\\]*(?:\\.[^\n"\\]*)*"z \\\r?\n|\ZceZdZdS) TokenErrorNrrr r"rrrRrRrrRceZdZdS)StopTokenizingNrSr"rrrVrVrTrrVc&eZdZdZdZdZdZdS) Untokenizerc>g|_d|_d|_d|_dS)Nr)tokensprev_rowprev_colencodingrs r__init__zUntokenizer.__init__s"    rcf|\}}||jks||jkr:||jkr/td|||j|j||jz }|r$|jd|zd|_||jz }|r|jd|zdSdS)Nz+start ({},{}) precedes previous end ({},{})\ r )r\r] ValueErrorformatr[append)rstartrowcol row_offset col_offsets radd_whitespacezUntokenizer.add_whitespacesS   #"6"63;N;NJ$fS#t}dmLLNN N4=(   K  v 2 3 3 3DM4=(  1 K  sZ/ 0 0 0 0 0 1 1rcNt|}g}d}|D]u}t|dkr|||nH|\}}}} } |tkr||_J|t krn|t kr||y|tkr$| | \|_ |_ |ttfvrd}nU|rS|rQ|d} |dt| kr.|j| t| |_ d}|||j|| \|_ |_ |ttfvr|xj dz c_ d|_ wd|jS)NFTrZrr8)iterlencompatENCODINGr^ ENDMARKERINDENTreDEDENTpopr\r]NEWLINENLr[rkr') riterableitindents startlinerEtok_typetokenrfendlineindents rr zUntokenizer.untokenizes (^^  " "A1vv{{ Ar"""01 -HeUC8## % 9$$6!!u%%%V## /2, t}gr]**  "w " 8s6{{**K&&v...$'KKDM!    & & & K  u % % %+. (DM4=GR=(( " ! wwt{###rcg}|jj}|dttfv}d}t j|g|D]}|dd\}} |t kr| |_"|ttfvr| dz } |tkr |rd| z} d}nd}|tkr|| o|tkr| |ttfvrd}n|r|r||dd}|| dS)NrFrmrbTrn)r[rerwrxr>chainrrr^NAMENUMBERSTRINGrtrurv) rr~ryr{ toks_appendr| prevstringtoktoknumtokvals rrqzUntokenizer.compatsDk( !H" -  #UGX66  C !WNFF!! & $''# * 6\F! " v&&&6!! GR=((  "w " GBK(((! K    9  rN)rrr r_rkr rqr"rrrXrXsQ 1 1 1#$#$#$J" " " " " rrXc t}||}|j||j}|Sr)rXr r^encode)ryutouts rr r sC& B -- ! !C {jj%% Jrc |dddd}|dks|drdS|dvs|drdS|S) N _-utf-8zutf-8-)zlatin-1 iso-8859-1z iso-latin-1)zlatin-1-z iso-8859-1-z iso-latin-1-r)lowerreplace startswith)orig_encencs r_get_normal_namer sw2 3B3-     ' 'S 1 1C g~~11~w 666 ~~ABB7| Orc jjn#t$rdYnwxYwdd}d}fd}fd}|}|trd|dd}d}|s|gfS||}|r||gfSt |s||gfS|}|s||gfS||}|r|||gfS|||gfS)NFrc< S#t$rYdSwxYwNr) StopIterationreadlinesr read_or_stopz%detect_encoding..read_or_stopCs6 8::    33 s c( |d}n7#t$r*d}d|}t|wxYwt|}|sdSt |d} t|}n;#t$r.d|z}nd|}t|wxYwr4|dkr)d}nd}t||d z }|S) Nrz'invalid or missing encoding declarationz {} for {!r}rZzunknown encoding: zunknown encoding for {!r}: {}zencoding problem: utf-8z encoding problem for {!r}: utf-8z-sig) decodeUnicodeDecodeErrorrd SyntaxError cookie_rematchrr*r LookupError)r line_stringmsgrr^codec bom_foundfilenames r find_cookiez$detect_encoding..find_cookieIsX #++g..KK! # # #;C##**399c"" "  #  ,, 4#EKKNN33 #8$$EE # # #*X55<>%   \^^F  {6""H)%(( UFO ##s  ""c t|d} t|j\}}|dt ||d}d|_|S#|xYw)NrbrT)line_bufferingr3) _builtin_openr rseekrmodeclose)rbufferr^linestexts rrrsx8T * *F)&/::% AVXdCCC    s AAA1c t|\}}tjd}tj|t |d|}t |j|Sr)r r>repeatrro _tokenize__next__)rr^consumedemptyrl_gens rr r s[$)22Hh  c " "E  hXs(;(;U C CF V_h / //rc # Kdx}x}}d}d\}}d}dg} |"|dkrd}tt|dddVd} d} | } |} n#t$rd} YnwxYw|| |} |d z }dt | } } |r| st d ||| }|rJ|dx} }tt|| d|z|||f|| zVd\}}d}n'|rN| d dd kr@| dddkr2tt|| z||t | f|Vd}d}|| z}|| z}|dkr|s| snd}| | krO| | dkr|d z }n2| | dkr|tzd ztz}n| | dkrd}nn | d z } | | kO| | krnR| | dvr| | dkr]| | d d}tt||| f|| t |zf| V| t |z } tt| | d|| f|t | f| V4|| dkr;| |tt| d| |df|| f| V|| dkrL|| vrt!dd|| | f| dd} tt"d|| f|| f| V|| dkLn| st d|dfd}| | krt%t&| | }|r|d \}}||f||f|} }}||kr[| ||| |}}||vs|dkr(|dkr"|dkrtt*|||| VnL|dvr>|dkrtt|||| Vn&tt,|||| Vn |dkrtt|||| Vn|t.vrt%t0|}|| | }|r=|d} | || }tt|||| f| Vnp||f}| |d}| }ne|t2vs"|ddt2vs|ddt2vr|ddkrz||f}t%t0|p?t0|d pt0|d}| |dd }}| }ntt|||| Vn|rtt8|||| Vnc|dkrd }nZ|d vr|d z }n |d!vr|d z}tt:|||| Vn,tt| | || f|| d zf| V| d z } | | k| rr| ddvrh| dsAtt,d|d z t | f|d z t | d zfdV| d dD] }tt"d|df|dfdV!tt@d|df|dfdVdS)"Nr 0123456789)r8rrr)rrr8rTrZzEOF in multi-line stringraz\ rb  z# #z rnz3unindent does not match any outer indentation levelz zEOF in multi-line statement.z...rmr \z([{z)]})!r rrrrrprRrrr ERRORTOKENtabsizerstripCOMMENTrxrertIndentationErrorrurL PseudoTokenspanrrw triple_quotedendpats single_quotedget isidentifierrrstriprrs)rr^lnumparenlev continuednumcharscontstrneedcontcontliner{ last_linerposmaxstrstartendprogendmatchrcolumn comment_token pseudomatchrfsposeposr~initialrs rrrs"##D#8iHGXHcG { " "H(FFB?????I D_  I8::DD   DDD   ;;x((D  c$iiS ?  G !;XFFF}}T**H $LLOO+c$tt*(<$ho?????$)! d233i611d233i86K6K GdN#dCII%6BBBBB!D.#d? ]]9] F))9##aKFF#Y$&&$go17:FF#Y$&&FFq))czzCyG##9##$(J$5$5f$=$=M#G] #;sS5G5G/G(H$PPPPP3}---CDJ #;s4yy(94AAAAA ##v&&&TcT T1Ic{DQQQQQ72;&&((*M%tS$7999"#2#,T3K$dKKKKK72;&& K !>q JJJICii";//55dC@@KL (--a00 s#'-$cCdC<<!%eCi$u+wx''^^ %#FE4tDDDDDD&&!||'E4tDDDDDD'dDIIIIII^^#GUD$EEEEEEm++&wu~66G&}}T377H&ll1oo $U3Y'tdC[$OOOOOO$(%="&uvv,#'..BQBi=00BQBi=00RyD(($(%=#+7;;w+?+?,A+2;;uQx+@+@,A+2;;uQx+@+@#B#B-1L!#''tT4HHHHHH))++ A#D%tTBBBBBB__ !II%'' A  E)) A #BtT4@@@@@@ DI #;s1u t=====q]Ciic_DeYr]&009J9J9U9UVY9Z9Z0dQhI%?$(CPYNN]^L^A_acddddd!""+>>T1Iay"====== IrD!9tQi < <<<<<.perrorts4 !!! rc|r|f|z|fz}d|zn!|r|d|nd|ztjddS)Nz%s:%d:%d: error: %sz : error: z error: %srZ)rexit)rrlocationargsrs rerrorzmain..errorxs  *;)WJ6D F(4/ 0 0 0 0  * Fhhh8 9 9 9 9 F;( ) ) )  rzpython -m tokenize)progrr/z filename.pyz'the file to tokenize; defaults to stdin)destnargsmetavarhelpz-ez--exactexact store_truez(display token names using the exact type)ractionrrzz %d,%d-%d,%d:2015rZrz interrupted zunexpected error: %s)NN)argparseArgumentParser add_argument parse_argsrrlistr rrrstdinrrrrfrprintrrrrrRrOSErrorKeyboardInterrupt Exception)r rparserrrr5r[r~ token_type token_rangeerrrrrs @rmainrpslOOO $ $*> $ ? ?F Zs -FHHH iglGIII     D  = 9}Hx.. 4!hqz2233 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4!Hsy1488F E EEJz ."- (EK%),CDK ;; 4 4 4elllD E E E E  E E 555x{1Q3' f chqk8dF^444444444 555x{ f chqk8dF^444444444  c8  c  o %+,,, ss-D< "B9- D<9B==D<B=A9D<< H42E>> H4 *F;; H4 G H4' G88H4 H4 H//H4c#K ddl}||D]%}|\}}}}}}} t||||f||f| V&dS)Nr)r TokenizerIterr ) source c_tokenizerinforrlineno end_linenocol_off end_col_offrs r!_generate_tokens_from_c_tokenizerr#s~a####))&11WWDHAT6:w TcFG#4z;6OQUVVVVVVWWr__main__)U __author__ __credits__builtinsrrcodecsrr collections functoolsior itertoolsr>rHrr~rrIASCIIrr__all__ namedtupler r*r-r0 WhitespaceCommentIgnoreName Hexnumber Binnumber Octnumber Decnumber IntnumberExponent PointfloatExpfloat Floatnumber ImagnumberNumberrF lru_cacherL StringPrefixSingleDoubleSingle3Double3TripleStringmapescapesortedSpecialFunny PlainTokenTokenContStr PseudoExtrasrr_prefixsetrrrEr4rArrrRrVrXr rr r rr rr#rr"rrrRs,*  +*****######## ###### BJ@"( K K 2:0"( ; ; -666 6      & &{4PQQ   :99///111  cc*z122 2UU7^^ C  &  . E)Y 9 = = ( U?)++-2U8__= ( *eJ)) U+[7-B C C z; 2 2    ((( u**,,-  $ # 2 2 |e#\E%9 : : |==== ? ? %RY'8$ G G GHH I h   U65&$ / /  % ==c:&&'==c:&&' ( (u]GV44 55vugtLLL ##%%''G#GGcM#GGcM&GGeO&GGeO      A#gq3w !%iU #!q !!!!!!!!%%%%%Y%%%\ \ \ \ \ \ \ \ ~6   Z$Z$Z$z    0002t=t=t=n%%%;;;zWWW zDFFFFFr