File: //proc/2/cwd/usr/lib/python2.4/urllib.pyo
m
=5Dc @ sM d Z d k Z d k Z d k Z d k Z d k Z d k l Z d d d d d d d d
d d d
d d d d d d d d d d d d d d d d d d g Z d Z
d! Z e i d" j o d# k
l Z l Z na e i d$ j o d# k l Z l Z n: e i d% j o d# k l Z l Z n d& Z d' Z e a e e d( Z e e e d) Z d* Z d+ e f d, YZ h Z d f d- YZ d e f d. YZ e a d/ Z e a d0 Z e a d1 Z! e a" d2 Z# d3 f d4 YZ$ d5 f d6 YZ% d7 e% f d8 YZ& d9 e% f d: YZ' d; e% f d< YZ( y e) Wn e* j
o d= Z+ n Xd> Z+ d? Z, d@ Z- e a. dA Z/ e a0 dB Z1 e a2 dC Z3 e a4 dD Z5 e a6 dE Z7 e a8 dF dG Z9 e a: dH Z; e a< dI Z= dJ Z> e a? dK Z@ dL ZA dM ZB eC dN eB D ZD dO ZE dP ZF dQ ZG h ZH dR dS ZI dT dU ZJ dV dW ZK dX ZL dY ZM e iN dZ j o d[ ZO d\ ZP d] ZQ nE e i d$ j o( d^ ZR d_ ZQ d` ZS da ZP n
eL ZQ eM ZP db ZT dc ZU g dd ZV de ZW eX df j o eW n d S(g s Open an arbitrary URL.
See the following document for more info on URLs:
"Names and Addresses, URIs, URLs, URNs, URCs", at
http://www.w3.org/pub/WWW/Addressing/Overview.html
See also the HTTP spec (from which the error codes are derived):
"HTTP - Hypertext Transfer Protocol", at
http://www.w3.org/pub/WWW/Protocols/
Related standards and specs:
- RFC1808: the "relative URL" spec. (authoritative status)
- RFC1738 - the "URL standard". (authoritative status)
- RFC1630 - the "URI spec". (informational status)
The object returned by URLopener().open(file) will differ per
protocol. All you know is that is has methods read(), readline(),
readlines(), fileno(), close() and info(). The read*(), fileno()
and close() methods work like those of open files.
The info() method returns a mimetools.Message object which can be
used to query various info about the object, if available.
(mimetools.Message objects are queried with the getheader() method.)
N( s urljoint urlopent URLopenert FancyURLopenert urlretrievet
urlcleanupt quotet
quote_plust unquotet unquote_plust urlencodet url2pathnamet pathname2urlt splittagt localhostt thishostt ftperrorst basejoint unwrapt splittypet splithostt splitusert splitpasswdt splitportt
splitnportt
splitqueryt splitattrt
splitvaluet splitgophertypet
getproxiess 1.16i
t mac( s url2pathnames pathname2urlt ntt riscosc C s
t | S( s OS-specific conversion from a relative URL of the 'file' scheme
to a file system path; not recommended for general use.N( R t pathname( R ( ( t /usr/lib/python2.4/urllib.pyR
4 s c C s
t | S( s OS-specific conversion from a file system path to a relative URL
of the 'file' scheme; not recommended for general use.N( R R ( R ( ( R! R 9 s c C sr | d j o t d | } n! t p t } | a n t } | d j o | i | Sn | i | | Sd S( s. urlopen(url [, data]) -> open file-like objectt proxiesN( R" t NoneR t openert
_urlopenert datat opent url( R( R& R" R$ ( ( R! R G s
c C s* t p
t a n t i | | | | S( N( R% R t retrieveR( t filenamet
reporthookR& ( R( R* R+ R& ( ( R! R U s
c C s t o t i n d S( N( R% t cleanup( ( ( R! R Z s t ContentTooShortErrorc B s t Z d Z RS( Nc C s t i | | | | _ d S( N( t IOErrort __init__t selft messaget content( R0 R1 R2 ( ( R! R/ ` s ( t __name__t
__module__R/ ( ( ( R! R- _ s c B s t Z d Z e Z d e Z e d Z d Z d Z d Z
d Z e d Z e d Z
e d Z e e e d
Z e d Z e d Z d
Z e e d o e d Z n d Z d Z d Z d Z e d Z RS( s, Class to open URLs.
This is a class rather than just a subroutine because we may need
more than one set of global protocol-specific options.
Note -- this is a base class for those who don't want the
automatic handling of errors type 302 (relocated) and 401
(authorization needed).s Python-urllib/%sc K s | d j o
t } n | | _ | i d | _ | i d | _ d | i f g | _ g | _
t i | _
d | _ t | _ d S( Nt key_filet cert_files
User-agent( R" R# R R0 t x509t getR5 R6 t versiont
addheaderst _URLopener__tempfilest ost unlinkt _URLopener__unlinkt tempcachet ftpcache( R0 R" R7 ( ( R! R/ r s
c C s | i d S( N( R0 t close( R0 ( ( R! t __del__ s c C s | i d S( N( R0 R, ( R0 ( ( R! RA s c C so | i oF x8 | i D]- } y | i | Wq t j
o q Xq W| i 2n | i o | i i n d S( N( R0 R; t fileR> t OSErrorR? t clear( R0 RC ( ( R! R, s
c G s | i i | d S( sd Add a header to be used by the HTTP interface only
e.g. u.addheader('Accept', 'sound/basic')N( R0 R: t appendt args( R0 RG ( ( R! t addheader s c C s t t | } | i oF | | i j o6 | i | \ } }
t | d } t | |
| Sn t
| \ } } | p
d } n | | i
j oA | i
| }
t
|
\ } } t | \ } } | | f } n d }
d | } | | _ | i d d } t | | p2 |
o | i |
| | SqC| i | | Sn y> | d j o t | | | Sn t | | | | SWn4 t i j
o% } t d | f t i! d n Xd S( s6 Use URLopener().open(file) instead of open(file, 'r').t rbRC t open_t -t _s socket errori N(" R t toBytest fullurlR0 R? R* t headersR' t fpt
addinfourlR t urltypeR( R" t proxyt proxyhostR t hostt selectorR# t namet typet replacet hasattrt open_unknown_proxyR& t open_unknownt getattrt sockett errort msgR. t syst exc_info( R0 RN R& RT RR R* R` RP RV RU RS RW R( RO ( ( R! R' s8
c C s( t | \ } } t d d | f d S( s/ Overridable interface to open unknown URL type.s url errors unknown url typeN( R RN RX R( R. ( R0 RN R& R( RX ( ( R! R\ s c C s, t | \ } } t d d | | f d S( s/ Overridable interface to open unknown URL type.s url errors invalid proxy for %sN( R RN RX R( R. RS ( R0 RS RN R& R( RX ( ( R! R[ s c C s t t | } | i o | | i j o | i | Sn t | \ }
} | d j om |
p
|
d j oX y< | i
| } | i } ~ t t | d | f SWq t j
o } q Xn | i | | } | i } | o t | d } n d k }
t | \ } } t | p d \ } } t | p d \ } } t | p d \ } } t i i | d } |
i | \ } } | i i! | t i" | d } | | f } | i d j o | | i | <n d d } d } d } d } | o5 d
| j o t) | d } n | | | | n xh | i& | } | d j o Pn | t+ | 7} | i, | | d 7} | o | | | | q:q:W| i- | i- ~ ~ | d j o* | | j o t. d | | f | n | S(
st retrieve(url) returns (filename, headers) for a local object
or (tempfilename, headers) for a remote object.RC i t wbNt i i ii s content-lengths Content-Lengths1 retrieval incomplete: got only %i out of %i bytes(/ R RM R( R0 R? R RX t url1R* R# t open_local_fileRP t infot hdrsR
R R. R` R' R& RO t tfpt tempfilet garbaget pathR R R< t splitextt suffixt mkstempt fdR; RF t fdopent resultt bst sizet readt blocknumR+ t intt blockt lent writeRA R- ( R0 R( R* R+ R& Rn Rk Rr Rv Rt Rj Re R` RX RP Rh Ru Rp Rs Rl Ri RO Rx ( ( R! R) sn "
c C s d k }
d } t | t oE t | \ } } | o" t | \ } } t
| } n | } n | \ } } t | \ } } | } d } | i d j o
d } ne t | \ } } | o t | \ } } n | o d | | | f } n t | o
| } n | p
t d n | o" d k } | i | i }
n d }
|
i | } | d j o> | i d | | i d d | i d d
t | n | i d | |
o | i d d
|
n | o | i d | n x | i D] } | i | q W| i | d j o | i | n | i! \ } } } | i% } | d j o t' | | d | SnG | d j o | i( | | | | | Sn | i( | | | | | | Sd S( s Use HTTP protocol.Nt https %s://%s%ss
http errors
no host givent POSTs Content-types! application/x-www-form-urlencodeds Content-lengths %dt GETt
Authorizations Basic %st Hosti s http:( s
http errors
no host given() t httplibR# t user_passwdt
isinstanceR( t strR RU RV R R t realhostR RR t restt lowert proxy_bypassR. t base64t encodestringt stript autht HTTPt hR& t
putrequestt putheaderRy R0 R: RG t
endheaderst sendt getreplyt errcodet errmsgRO t getfileRP RQ t
http_error( R0 R( R& R R RR RO R RP RG R RV RU R R R R R ( ( R! t open_http sf
c
C s d | } t | | of t | | } | d j o | | | | | | } n | | | | | | | } | o | Sq n | i
| | | | | S( s Handle http errors.
Derived class can override this, or provide specific handlers
named http_error_DDD where DDD is the 3-digit error code.s
http_error_%dN( R RW RZ R0 R] t methodR& R# R( RP R RO Rr t http_error_default(
R0 R( RP R R RO R& RW R Rr ( ( R! R F s
c C s/ | i } | i t d | | | f d S( s> Default error handler: close the connection and raise IOError.s
http errorN( RP Ru t voidRA R. R R RO ( R0 R( RP R R RO R ( ( R! R U s
t sslc C s d k }
d } t | t oE t | \ } }
| o" t | \ } } t
| } n | } n | \ } }
t |
\ } } | } d } | i d j o
d } nN t | \ } } | o t | \ } } n | o d | | | f }
n | p
t d n | o" d k } | i | i } n d } |
i | d d | i d | i } | d j o> | i d |
| i d
d | i d d
t | n | i d |
| o | i d d | n | o | i d | n x | i D] } | i | qW| i | d j o | i! | n | i" \ } } } | i& } | d j o t( | | d | SnG | d j o | i) | | | | | Sn | i) | | | | | | Sd S( s Use HTTPS protocol.Nt httpss %s://%s%ss https errors
no host giveni R5 R6 R| s Content-types! application/x-www-form-urlencodeds Content-lengths %dR} R~ s Basic %sR i s https:( s https errors
no host given(* R R# R R R( R R RU RV R R R R RR R R R. R R R R t HTTPSR0 R5 R6 R R& R R Ry R: RG R R R R R RO R RP RQ R ( R0 R( R& R R RR RO R RP RG RV RU R R R R R R ( ( R! t
open_https\ sj
c C s d k } t | \ } } | p
t d n t | } t | \ } } t | \ } } t | } | o% t | } | i | | | } n | i
| | } t | t d | S( s Use Gopher protocol.Ns gopher errors
no host givens gopher:( s gopher errors
no host given( t gopherlibR R( RU RV R. R R RX R t queryt
send_queryRP t
send_selectorRQ t noheaders( R0 R( R RU RP R RV RX ( ( R! t open_gopher s
c C sa | d d j o? | d d !d j o+ | d d !i d j o | i | Sn | i | Sd S( s/ Use local file or FTP depending on form of URL.i s //i t /i s
localhost/N( R( R R0 t open_ftpRf ( R0 R( ( ( R! t open_file s ?c C s d k }
d k } d k } y d k l } Wn t j
o d k l } n Xt | \ } } t | } y t
i | } Wn0 t j
o$ }
t |
i |
i |
i n X| i } | i i | i d t } |
i | d } | i | d | p d | | f } | pB | } | d d j o d | } n t" t# | d
| | Sn t$ | \ } } | od t& i' | t( t) f j oB | } | d d j o d | } n t" t# | d
| | Sn t d
d S( s Use local file.N( s StringIOt usegmti s6 Content-Type: %s
Content-Length: %d
Last-modified: %s
s
text/plaini R s file://RI s local file errors not on local host( s local file errors not on local host(* t mimetypest mimetoolst email.Utilst emailt cStringIOt StringIOt ImportErrorR R( RU RC R
t localnameR< t statt statsRD t eR. t errnot strerrorR* t st_sizeRt t Utilst
formatdatet st_mtimet Truet modifiedt
guess_typet mtypet MessageRO t urlfileRQ R' R t portR^ t
gethostbynameR
R ( R0 R( R R RC R R Rt R R R R RU R R R RO ( ( R! Rf s< )*c C s d k } d k } y d k l } Wn t j
o d k l } n Xt | \ } } | p
t d n t
| \ } } t | \ } } | o t | \ } } n d } t | } t | p d } t | p d } t i | } | p d k }
|
i } n
t | } t | \ } } t | } | i d } | d | d } } | o | d o | d } n | o | d o d | d <n | | | d i | f } t | i t! j oP xM | i i" D]8 } | | j o% | i | } | i | =| i% qqWn yK| | i j o# t& | | | | | | i | <n | p
d
} n d } xP | D]H } t) | \ } } | i+ d j o | d j o | i, } qvqvW| i | i- | | \ } } | i0 d | d } d } | o | d | 7} n | d j o | d j o | d | 7} n | i3 | | } t4 | | d | SWn4 t5 j
o% }
t d |
f t7 i8 d n Xd S( s Use FTP protocol.N( s StringIOs ftp errors
no host givenRd R ii i t Dt IRX t at At it ds ftp:s Content-Type: %s
s Content-Length: %d
i ( s ftp errors
no host given( R R R R R R (9 R R R R R R R( RU Rl R. R R R t userR t passwdR# R R^ R t ftplibt FTP_PORTRw R t attrst splitt dirsRC t joint keyRy R0 R@ t MAXFTPCACHEt keyst kt vRA t
ftpwrapperRX t attrR t valueR t uppert retrfileRP t retrlenR R RO R RQ R R` Ra Rb ( R0 R( R R RP R RC R RX R R R R R` RU R R Rl R R R R R RO R ( ( R! R sx
#
c C s d k } y d k l } Wn t j
o d k l } n Xy | i d d \ } } Wn t j
o t d n X| p
d } n | i
d } | d j o- d
| | j o | | d } | | } n d } g }
|
i d t i d
t i t i |
i d | | d j o d k } | i | } n
t | } |
i d t | |
i d |
i | d i |
}
| |
} | i | d } t | | | S( s Use "data" URL.N( s StringIOt ,i s
data errors bad data URLs text/plain;charset=US-ASCIIt ;i t =Rd s Date: %ss %a, %d %b %Y %T GMTs Content-type: %sR s Content-length: %ds
( s
data errors bad data URL( R R R R R( R RX R& t
ValueErrorR. t rfindt semit encodingR` RF t timet strftimet gmtimeR t decodestringR Ry R t fR RO RQ ( R0 R( R& R R R R R RX R R` RO ( ( R! t open_data s@
( R3 R4 t __doc__R# R; t __version__R9 R/ RB RA R, RH R' R\ R[ R) R R R RZ R^ R R R Rf R R ( ( ( R! R e s,
! ? 8 9 # <c B s t Z d Z d Z d Z e d Z d Z e d Z e d Z e d Z
e d Z e d Z e d
Z
d d Z d
Z RS( s? Derived class with handlers for errors we can handle (perhaps).c O s2 t i | | | h | _ d | _ d | _ d S( Ni i
( R R/ R0 RG t kwargst
auth_cachet triest maxtries( R0 RG R ( ( R! R/ ; s c C s t | | d | S( s3 Default error handling -- don't raise an exception.s http:N( RQ RP RO R( ( R0 R( RP R R RO ( ( R! R A s c C s | i d 7_ | i o\ | i | i j oI t | d o
| i } n
| i } d | _ | | | d d | Sn | i
| | | | | | } d | _ | S( s% Error 302 -- relocated (temporarily).i t http_error_500i i s) Internal Server Error: Redirect RecursionN( R0 R R RZ R t methR R( RP RO t redirect_internalR R R& Rr ( R0 R( RP R R RO R& R Rr ( ( R! t http_error_302E s
c
C s d | j o | d } n d | j o | d } n d S| i } | i t | i d | | } | i
} | i d p | i d p
| i d p! t
d | | d | | n | i | S( Nt locationt urit :s http://s https://s ftp://s redirect errors) - Redirection to url '%s' is not allowed( RO t newurlRP Ru R RA R R0 RX R( R t newurl_lowert
startswithR. R R R' (
R0 R( RP R R RO R& R R R ( ( R! R U s
0
c C s | i | | | | | | S( s* Error 301 -- also relocated (permanently).N( R0 R R( RP R R RO R& ( R0 R( RP R R RO R& ( ( R! t http_error_301n s c C s | i | | | | | | S( s; Error 303 -- also relocated (essentially identical to 302).N( R0 R R( RP R R RO R& ( R0 R( RP R R RO R& ( ( R! t http_error_303r s c C sJ | d j o | i | | | | | | Sn | i | | | | | Sd S( s1 Error 307 -- relocated, but turn POST into error.N(
R& R# R0 R R( RP R R RO R ( R0 R( RP R R RO R& ( ( R! t http_error_307v s
c
C s d | j o t i | | | | | | n | d }
d k } | i
d |
} | p t i | | | | | | n | i \ } } | i d j o t i | | | | | | n d | i d } | d j o t | | | | Sn t | | | | | Sd S( s Error 401 -- authentication required.
See this URL for a description of the basic authentication scheme:
http://www.ics.uci.edu/pub/ietf/http/draft-ietf-http-v10-spec-00.txts www-authenticateNs! [ ]*([^ ]+)[ ]+realm="([^"]*)"t basict retry_t _basic_auth( RO R R R0 R( RP R R t stufft ret matcht groupst schemet realmR RX RW R& R# R] (
R0 R( RP R R RO R& R RW R R R R ( ( R! t http_error_401} s$
c
C s t | \ } } | i d d } | | } | i | | | \ } } | p | p d Sn t | d d d t | d d d | } d | | } | d j o | i | Sn | i | | Sd S( Nt @i t safeRd R s http://( R R( RU RV t findR R0 t get_user_passwdR R R R# R R R& R' (
R0 R( R R& R RV R RU R R ( ( R! t retry_http_basic_auth s
.
c
C s t | \ } } | i d d } | | } | i | | | \ } } | p | p d Sn t | d d d t | d d d | } d | | } | i | | S( NR i R Rd R s //( R R( RU RV R R R0 R R R R R# R R R R& (
R0 R( R R& R RV R RU R R ( ( R! t retry_https_basic_auth s
.i c C s | d | i } | | i j o$ | o | i | =qH | i | Sn | i | | \ } } | p | o | | f | i | <n | | f S( NR (
R RU R R R0 R t clear_cachet prompt_user_passwdR R ( R0 RU R R R R R ( ( R! R s c C sh d k } y@ t d | | f } | i d | | | f } | | f SWn t j
o Hd Sn Xd S( s# Override this in a GUI environment!Ns Enter username for %s at %s: s# Enter password for %s in %s at %s: ( NN( t getpasst raw_inputR RU R R t KeyboardInterruptR# ( R0 RU R R R
R ( ( R! R s ( R3 R4 R R/ R R# R R R R R R R R
R R ( ( ( R! R 8 s
c C s$ t d j o t i d a n t S( s8 Return the IP address of the magic hostname 'localhost'.R
N( t
_localhostR# R^ R ( ( ( R! R
s
c C s* t d j o t i t i a n t S( s* Return the IP address of the current host.N( t _thishostR# R^ R t gethostname( ( ( R! R s
c C s' t d j o d k } | i a n t S( s1 Return the set of errors raised by the FTP class.N( t
_ftperrorsR# R t
all_errors( R ( ( R! R s
c C st t d j oc d k } y d k l } Wn t j
o d k l } n X| i | d a t i i n t S( s) Return an empty mimetools.Message object.N( s StringIOi ( t
_noheadersR# R R R R R RP RA ( R R ( ( R! R s
R c B s; t Z d Z d Z d Z d Z d Z d Z RS( s; Class used by open_ftp() for cache of open FTP connections.c C s; | | _ | | _ | | _ | | _ | | _ | i d S( N( R R0 R RU R R t init( R0 R R RU R R ( ( R! R/ s c C s{ d k } d | _ | i | _ | i i | i | i | i i | i | i
x! | i D] } | i i
| q] Wd S( Ni ( R R0 t busyt FTPt ftpt connectRU R t loginR R R t dirt cwd( R0 R R ( ( R! R s
c C s d k } | i | d j o d } d } n d | } d } y | i i | Wn0 | i j
o! | i | i i | n Xd } | o | o y | i i
| Wn4 | i j
o% } t d | f t i d n X| i i | y d | } | i i | } Wqm| i j
o@ } t | d
d j o t d | f t i d qiqmXn | pA | i i d | o d | } n d
} | i i | } n d | _ t | d i d | i | d f S( NR R s TYPE Ai s TYPE i s ftp errori s RETR i t 550s LIST t LISTRI ( R R ( R R0 t endtransferRX t cmdt isdirR t voidcmdR R R# t connRC t nlstt
error_permt reasonR. Ra Rb t ntransfercmdR R t addclosehookt makefile( R0 RC RX R" R R! R' R$ ( ( R! R s@
!
) c C sI | i p d Sn d | _ y | i i Wn t j
o n Xd S( Ni ( R0 R R t voidrespR ( R0 ( ( R! R ) s
c C s8 | i y | i i Wn t j
o n Xd S( N( R0 R R RA R ( R0 ( ( R! RA 2 s
( R3 R4 R R/ R R R RA ( ( ( R! R s % t addbasec B s) t Z d Z d Z d Z d Z RS( s( Base class for addinfo and addclosehook.c C s | | _ | i i | _ | i i | _ t | i d o | i i | _ n t | i d o | i i | _ n
d | _ t | i d o9 | i i | _ t | i d o | i i | _ q n d S( Nt readlinest filenoc C s d S( N( R# ( ( ( R! t <lambda>D s t __iter__t next( RP R0 Ru t readlineRZ R- R. R0 R1 ( R0 RP ( ( R! R/ < s c C s d | i i t | | i f S( Ns <%s at %r whose fp = %r>( R0 t __class__R3 t idRP ( R0 ( ( R! t __repr__J s c C sL d | _ d | _ d | _ d | _ | i o | i i n d | _ d S( N( R# R0 Ru R2 R- R. RP RA ( R0 ( ( R! RA N s
( R3 R4 R R/ R5 RA ( ( ( R! R, 9 s R) c B s t Z d Z d Z d Z RS( s* Class to add a close hook to an open file.c G s& t i | | | | _ | | _ d S( N( R, R/ R0 RP t closehookt hookargs( R0 RP R6 R7 ( ( R! R/ Y s c C sA t i | | i o& | i | i d | _ d | _ n d S( N( R, RA R0 R6 R7 R# ( R0 ( ( R! RA ^ s
( R3 R4 R R/ RA ( ( ( R! R) V s t addinfoc B s t Z d Z d Z d Z RS( s. class to add an info() method to an open file.c C s t i | | | | _ d S( N( R, R/ R0 RP RO ( R0 RP RO ( ( R! R/ h s c C s | i S( N( R0 RO ( R0 ( ( R! Rg l s ( R3 R4 R R/ Rg ( ( ( R! R8 e s RQ c B s) t Z d Z d Z d Z d Z RS( s9 class to add info() and geturl() methods to an open file.c C s& t i | | | | _ | | _ d S( N( R, R/ R0 RP RO R( ( R0 RP RO R( ( ( R! R/ r s c C s | i S( N( R0 RO ( R0 ( ( R! Rg w s c C s | i S( N( R0 R( ( R0 ( ( R! t geturlz s ( R3 R4 R R/ Rg R9 ( ( ( R! RQ o s c C s d S( Ni ( ( t x( ( R! t _is_unicode s c C s
t | t S( N( R R: t unicode( R: ( ( R! R; s c C sX t | oG y | i d } WqT t j
o! t d t | d qT Xn | S( s toBytes(u"URL") --> 'URL'.t ASCIIs URL s contains non-ASCII charactersN( R; R( t encodet UnicodeErrort repr( R( ( ( R! RM s
#c C sn | i } | d d j o( | d d j o | d d !i } n | d d j o | d i } n | S( s8 unwrap('<URL:type://host/path>') --> 'type://host/path'.i t <it >i s URL:N( R( R ( R( ( ( R! R s " c C sz t d j o d k } | i d a n t i | } | o1 | i d } | i | t | d f Sn d | f S( s: splittype('type:opaquestring') --> 'type', 'opaquestring'.Ns
^([^/:]+):i (
t _typeprogR# R t compileR R( t groupR R Ry ( R( R R R ( ( R! R s
"c C s] t d j o d k } | i d a n t i | } | o | i d d Sn d | f S( s; splithost('//host[:port]/path') --> 'host[:port]', '/path'.Ns ^//([^/]*)(.*)$i i ( t _hostprogR# R RD R R( RE ( R( R R ( ( R! R s
c C sf t d j o d k } | i d a n t i | } | o t t | i d d Sn d | f S( sJ splituser('user[:passwd]@host[:port]') --> 'user[:passwd]', 'host[:port]'.Ns ^(.*)@(.*)$i i ( t _userprogR# R RD R RU t mapR RE ( RU R R ( ( R! R s
c C s] t d j o d k } | i d a n t i | } | o | i d d Sn | d f S( s/ splitpasswd('user:passwd') -> 'user', 'passwd'.Ns ^([^:]*):(.*)$i i ( t _passwdprogR# R RD R R RE ( R R R ( ( R! R s
c C s] t d j o d k } | i d a n t i | } | o | i d d Sn | d f S( s* splitport('host:port') --> 'host', 'port'.Ns ^(.*):([0-9]+)$i i ( t _portprogR# R RD R RU RE ( RU R R ( ( R! R s
ic C s t d j o d k } | i d a n t i | } | of | i d d \ } } y$ | p
t d n t | } Wn t j
o
d } n X| | f Sn | | f S( s Split host and port, returning numeric port.
Return given default port if no ':' found; defaults to -1.
Return numerical port if a valid number are found after ':'.
Return None if ':' but not a valid number.Ns ^(.*):(.*)$i i s no digits( t
_nportprogR# R RD R RU RE R R Rw t nportt defport( RU RM RL R R R ( ( R! R s
c C s] t d j o d k } | i d a n t i | } | o | i d d Sn | d f S( s/ splitquery('/path?query') --> '/path', 'query'.Ns ^(.*)\?([^?]*)$i i ( t
_queryprogR# R RD R R( RE ( R( R R ( ( R! R s
c C s] t d j o d k } | i d a n t i | } | o | i d d Sn | d f S( s) splittag('/path#tag') --> '/path', 'tag'.Ns ^(.*)#([^#]*)$i i ( t _tagprogR# R RD R R( RE ( R( R R ( ( R! R
s
c C s! | i d } | d | d f S( sk splitattr('/path;attr1=value1;attr2=value2;...') ->
'/path', ['attr1=value1', 'attr2=value2', ...].R i i N( R( R t words( R( RP ( ( R! R s c C s] t d j o d k } | i d a n t i | } | o | i d d Sn | d f S( s- splitvalue('attr=value') --> 'attr', 'value'.Ns ^([^=]*)=(.*)$i i ( t
_valueprogR# R RD R R RE ( R R R ( ( R! R s
c C s? | d d j o$ | d d !o | d | d f Sn d | f S( s2 splitgophertype('/Xselector') --> 'X', 'selector'.i R i N( RV R# ( RV ( ( R! R * s t 0123456789ABCDEFabcdefc c sF x? | ]8 } x/ t D]' } | | t t | | d f Vq Wq Wd S( i N( t [outmost-iterable]R t _hexdigt bt chrRw ( RS R RU ( ( R! t <generator expression>1 s
c C s | i d } x t d t | D] } | | } y t | d | d | | <Wq% t j
o d | | | <q% t j
o, t
t | d d | d | | <q% Xq% Wd i | S( s" unquote('abc%20def') -> 'abc def'.t %i i i Rd N(
t sR t rest xrangeRy R t itemt _hextochrt KeyErrort UnicodeDecodeErrort unichrRw R ( RY R RZ R\ ( ( R! R 3 s
.c C s | i d d } t | S( s% unquote('%7e/abc+def') -> '~/abc def't +t N( RY RY R ( RY ( ( R! R @ s sA ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_.-R c C s | t f } y t | } Wnr t j
of | t 7} h } xB t d D]4 } t | } | | j o | p d | | | <qH W| t | <n Xt
| i | } d i | S( s quote('abc def') -> 'abc%20def'
Each part of a URL, e.g. the path info, the query, etc., has a
different set of reserved characters that must be quoted.
RFC 2396 Uniform Resource Identifiers (URI): Generic Syntax lists
the following reserved characters.
reserved = ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" |
"$" | ","
Each of these characters is reserved in some component of a URL,
but not necessarily in all of them.
By default, the quote function is intended for quoting the path
section of a URL. Thus, it will not encode '/'. This character
is reserved, but in typical usage the quote function is being
called on a path where the existing slash characters are used as
reserved characters.
i s %%%02XRd N( R t always_safet cachekeyt _safemapst safe_mapR^ t rangeR RV t cRH t __getitem__RY RZ R ( RY R Rh Rd Rf RZ R ( ( R! R J s
&Rd c C sA d | j o' t | | d } | i d d Sn t | | S( s9 Quote the query fragment of a URL; replacing ' ' with '+'Rb Ra N( RY R R RY ( RY R ( ( R! R l s
i c
C s t | d o | i } nh y0 t | o t | d t o
t n Wn4 t j
o( t i \ } } } t d | n Xg } | pT xl| D]E \ } } t t | } t t | } | i | d | q Wn x| D]\ } } t t | } t | t o% t | } | i | d | q t | o1 t | i d d } | i | d | q y t | } Wn: t j
o. t t | } | i | d | q Xx/ | D]' } | i | d t t | qWq Wd i | S( sv Encode a sequence of two-element tuples or dictionary into a URL query string.
If any values in the query arg are sequences and doseq is true, each
sequence element is converted to a separate parameter.
If the query arg is a sequence of two-element tuples, the order of the
parameters in the output will match the order of parameters in the
input.
t itemsi s1 not a valid non-string sequence or mapping objectR R= RY t &N( RZ R Rj Ry R t tuplet TypeErrorRa Rb t tyt vat tbt lt doseqR R R R RF R; R> R: t eltR (
R Rr Ro R: R Rn Rq Rs R Rp ( ( R! R s sD "
)c C st h } xg t i i D]V \ } } | i } | d j o q n | o# | d d j o | | | d <q q W| S( s Return a dictionary of scheme -> proxy server URL mappings.
Scan the environment for variables named <scheme>_proxy;
this seems to be the standard convention. If you need a
different way, you can pass a proxies dictionary to the
[Fancy]URLopener constructor.
t no_proxyit _proxyN( R" R< t environRj RW R R ( R" RW R ( ( R! t getproxies_environment s
c C s t i i d d p t i i d d } | d j o d Sn t | \ } } xF | i d D]5 } | o( | i
| p | i
| o d Sqb qb Wd S( s Test if proxies should not be used for a particular host.
Checks the environment for a variable named no_proxy, which should
be a list of DNS suffixes separated by commas, or '*' for all hosts.
Rt Rd t NO_PROXYt *i R i N( R< Rv R8 Rt R RU t hostonlyR R RW t endswith( RU RW Rt Rz R ( ( R! t proxy_bypass_environment s +
' t darwinc C s y
d k } Wn t j
o h Sn Xy | i } Wn | i j
o h Sn Xh } d | j oD | d o9 y | d } Wn | i j
o q Xd | | d <n | S( s Return a dictionary of scheme -> proxy server URL mappings.
By convention the mac uses Internet Config to store
proxies. An HTTP proxy, for instance, is stored under
the HttpProxy key.
Nt UseHTTPProxyt
HTTPProxyHosts http://%sR{ ( t icR t ICt configR_ R" R ( R R" R R ( ( R! t getproxies_internetconfig s"
c C s t o t | Sn d Sd S( Ni ( Rw R| RU ( RU ( ( R! R s
c C s t p t S( N( Rw R ( ( ( R! R s c C sh h } y
d k } Wn t j
o | Sn Xy| i | i d } | i | d d } | o t | i | d d } d | j op x | i
d D]X } | i
d d \ } } d k } | i d | p d
| | f } n | | | <q Wq:| d d j o | | d
<q:d | | d
<d | | d <n | i Wn t t t f j
o n X| S( sx Return a dictionary of scheme -> proxy server URL mappings.
Win32 uses the registry to store proxies.
Ns; Software\Microsoft\Windows\CurrentVersion\Internet Settingst ProxyEnablei t ProxyServerR R i s ^([^/:]+)://s %s://%si s http:R{ s http://%ss ftp://%sR ( R" t _winregR t OpenKeyt HKEY_CURRENT_USERt internetSettingst QueryValueExt proxyEnableR t proxyServerR t pt protocolt addressR R t Closet WindowsErrorR Rm ( R R R R R R R R" R ( ( R! t getproxies_registry s<
c C s t p t S( s Return a dictionary of scheme -> proxy server URL mappings.
Returns settings gathered from the environment, if specified,
or the registry.
N( Rw R ( ( ( R! R 0 s c
C s y d k } d k } Wn t j
o d Sn XyK | i | i d } | i | d d } t | i | d d } Wn t
j
o d Sn X| p | o d Sn | g } y5 t i
| d } | | j o | i | n Wn t i j
o n X| i d } d } xi | t | j oU | | d j o6 d d t i t i
t i g | | | d +n | d 7} q Wxv | D]n } | i d
d } | i d d
} | i d d
} x/ | D]' } | i | | | i o d SqqWqWd S( Ni s; Software\Microsoft\Windows\CurrentVersion\Internet SettingsR t
ProxyOverrideR s <local>R
s 127.0.0.1i t .s \.Ry s .*t ?( R R R R R R R R R t
proxyOverrideR RU R^ R t addrRF R_ R R Ry R t testRY t valR R (
RU R R R R R R R R R ( ( R! t proxy_bypass_registry9 sP
6 c C s& t o t | Sn t | Sd S( s Return a dictionary of scheme -> proxy server URL mappings.
Returns settings gathered from the environment, if specified,
or the registry.
N( Rw R| RU R ( RU ( ( R! R m s
c C s d } x$ t d D] } | t | } q W| d } t i } t | } t | } t i } | | j o d GHn t | GHt | GHt | GHt | | d Gd GHd S( NRd i i s Wrong!i t sec(
RY Rg R RV R t t0R t qsR t uqst t1R@ t round( R R R R RY R ( ( R! t test1 s
c C s d | | | f GHd S( Ns0 Block number: %d, Block size: %d, Total size: %d( Rv t blocksizet totalsize( Rv R R ( ( R! R+ s c C sA | p: d d d d d g } t t d o | i d qA n z x | D] } d d G| Gd d GHt | d t \ } } | GH| o6 d
GHx% | i
D] } | d G| | GHq Wd
GHn t | d } | i } ~ d
| j o( t i d d } | i | d
} n | GHd \ } } qK Wd d GHWd t Xd S( Ns /etc/passwds file:/etc/passwds file://localhost/etc/passwds& ftp://ftp.python.org/pub/python/READMEs http://www.python.org/index.htmlR s! https://synergy.as.cmu.edu/~geek/RK i
s ======R RI s
Rd i( ( NN( RG RZ R RF R( R R# R+ t fnR R R R' RP Ru R& t stringt maketranst tablet translateR ( RG RP R( R R R& R R ( ( R! R s6
c C s d k } d k } y# | i | i d d \ } } Wn& | i j
o } | GHd GHd Sn Xd } xR | D]J \ } } | d j o | d } n | d j o d GHd Gd GHd Sqk qk W| o& | d j o t
n t | n2 | p d GHn x | D] } t
| i Gq Wd S(
Ni t ths Use -h for helpi s -ts -hs& Usage: python urllib.py [-t] [url ...]s -t runs self-test;s' otherwise, contents of urls are printed( t getoptRa t argvt optsRG R_ R` t tt oR R R R( R Ru ( R R( RG R Ra R R` R R ( ( R! t main s4 #
t __main__(Y R R R^ R< R Ra t urlparset urljoinR t __all__R R RW t macurl2pathR
R t
nturl2patht
rourl2pathR# R% R R R R. R- R@ R R R R
R R R R R R R R, R) R8 RQ R< t NameErrorR; RM R RC R RF R RG R RI R RJ R RK R RN R RO R R RQ R R RT t dictR] R R Rc Re R R R Rw R| t platformR R R R R R R+ R R R3 (= R R R R| R R R, R R@ R8 R- RQ R R R R R R R R R R R RT RM R R R R
R R; R R R) R R Re R R] Ra R R R R R
R R^ R Rw R R Rc R+ R R R R R R R< R ( ( R! R s
]
I
" @
- 4
!