Nfj3dZddlZddlZddlZddlmZddlmZmZm Z ddl m Z ddl m Z ddlmZdd lmZdd lmZGd d ZdS) a Module containing the UniversalDetector detector class, which is the primary class a user of ``chardet`` should use. :author: Mark Pilgrim (initial port to Python) :author: Shy Shalom (original C code) :author: Dan Blanchard (major refactoring for 3.0) :author: Ian Cordasco N)CharSetGroupProber) InputStateLanguageFilter ProbingState)EscCharSetProber) Latin1Prober)MBCSGroupProber)SBCSGroupProber) UTF1632Proberc eZdZdZdZejdZejdZejdZ dddd d d d d dZ e j fdZ edZedZedZdZdZdZdS)UniversalDetectoraq The ``UniversalDetector`` class underlies the ``chardet.detect`` function and coordinates all of the different charset probers. To get a ``dict`` containing an encoding and its confidence, you can simply run: .. code:: u = UniversalDetector() u.feed(some_bytes) u.close() detected = u.result g?s[-]s(|~{)s[-]z Windows-1252z Windows-1250z Windows-1251z Windows-1256z Windows-1253z Windows-1255z Windows-1254z Windows-1257)z iso-8859-1z iso-8859-2z iso-8859-5z iso-8859-6z iso-8859-7z iso-8859-8z iso-8859-9z iso-8859-13cd|_d|_g|_d|_d|_d|_d|_d|_||_tj t|_ d|_ |dSN)_esc_charset_prober_utf1632_prober_charset_probersresultdone _got_data _input_state _last_char lang_filterlogging getLogger__name__logger_has_win_bytesreset)selfrs yC:\Users\Terasoftware\OneDrive\Desktop\faahhh\fyndo\fyndo\venv\Lib\site-packages\pip/_vendor/chardet/universaldetector.py__init__zUniversalDetector.__init__Tsq#' # "   &'11 " c|jSr)rr s r! input_statezUniversalDetector.input_statebs   r#c|jSr)rr%s r! has_win_byteszUniversalDetector.has_win_bytesfs ""r#c|jSr)rr%s r!charset_probersz!UniversalDetector.charset_probersjs $$r#c2dddd|_d|_d|_d|_tj|_d|_|jr|j |j r|j |j D]}| dS)z Reset the UniversalDetector and all of its probers back to their initial states. This is called by ``__init__``, so you only need to call this directly in between analyses of different documents. Nencoding confidencelanguageFr#) rrrrr PURE_ASCIIrrrrrr)r probers r!rzUniversalDetector.resetns $(sMM  #&1  # -  $ * * , , ,   )  & & ( ( (+  F LLNNNN  r#c$|jrdS|sdSt|tst|}|js|t jr dddd|_n|t jt j fr dddd|_nx|dr dddd|_nW|d r d ddd|_n6|t j t j fr d ddd|_d |_|jd  d |_dS|j tjkrt|j|rtj|_ nH|j tjkr3|j|j|zrtj|_ |dd|_|jst-|_|jjt0jkr]|j|t0jkr5|jj|jdd|_d |_dS|j tjkr|jst?|j |_|j|t0jkr?|jj|j|jj!d|_d |_dSdS|j tjkr|j"sztG|j g|_"|j tHj%zr&|j"&tO|j"&tQ|j"D]U}||t0jkr0|j||j!d|_d |_nV|j)|r d |_*dSdSdS)a Takes a chunk of a document and feeds it through all of the relevant charset probers. After calling ``feed``, you can check the value of the ``done`` attribute to see if you need to continue feeding the ``UniversalDetector`` more data, or if it has made a prediction (in the ``result`` attribute). .. note:: You should always call ``close`` when you're done feeding in your document if ``done`` is not already ``True``. Nz UTF-8-SIG?r-zUTF-32szX-ISO-10646-UCS-4-3412szX-ISO-10646-UCS-4-2143zUTF-16Tr.)+r isinstance bytearrayr startswithcodecsBOM_UTF8r BOM_UTF32_LE BOM_UTF32_BEBOM_LEBOM_BErrr1HIGH_BYTE_DETECTORsearch HIGH_BYTE ESC_DETECTORr ESC_ASCIIrr stater DETECTINGfeedFOUND_IT charset_nameget_confidencerrrr0rr rNON_CJKappendr r WIN_BYTE_DETECTORr)r byte_strr2s r!rGzUniversalDetector.feedsT 9  F  F(I.. + **H~# ""6?33 X!,"% " $$f&96;N%OPP X,43TVWW $$%899 X!9"% " $$%899 X!9"% " $$fmV]%CDD X,43TVWW !DN{:&      5 5 9&--h77 9$.$8!!!Z%:: 9%,,T_x-GHH 9%/$8!"233-# 3#0??D   %)? ? #((22l6KK  $ 4 A"&"6"E"E"G"G " !    4 4! ++ N+;D22$$$$$$$%  K$2#5    !!X!##X#%%X%&~+~+~+@GGGGGr#r)rar:rrbcharsetgroupproberrenumsrrr escproberr latin1proberr mbcsgroupproberr sbcsgroupproberr utf1632proberr rrfr#r!rns8  222222;;;;;;;;;;''''''&&&&&&,,,,,,,,,,,,((((((TTTTTTTTTTr#