File: //proc/3/task/3/cwd/usr/lib/python2.4/urllib2.pyo
m
=5Dc @ sc d Z d k Z d k Z d k Z d k Z d k Z d k Z d k Z d k Z d k Z d k
Z
d k Z d k Z d k
Z
d k Z d k Z d k Z d k Z d k Z d k Z y d k l Z Wn e j
o d k l Z n Xd k l Z l Z l Z l Z l Z l Z l Z l Z l Z l! Z! l" Z" l# Z# l$ Z$ l% Z% d k l& Z& l' Z' l( Z( l) Z) d Z* d a, d d Z- d Z. d e/ f d YZ0 d
e0 e f d YZ1 d e0 f d
YZ2 d f d YZ3 d f d YZ4 d Z5 d f d YZ6 d e6 f d YZ7 d e6 f d YZ8 d e6 f d YZ9 d Z: d e6 f d YZ; d f d YZ< d e6 f d! YZ= d" f d# YZ> d$ e> f d% YZ? d& f d' YZ@ d( e@ e6 f d) YZA d* e@ e6 f d+ YZB d, ZC d- f d. YZD d/ e6 eD f d0 YZE d1 e6 eD f d2 YZF d3 f d4 YZG d5 e6 f d6 YZH d7 eH f d8 YZI eJ e d9 o d: eH f d; YZK n d< e6 f d= YZL d> e6 f d? YZM d@ ZN dA ZO dB e6 f dC YZP dD e6 f dE YZQ dF eQ f dG YZR dH e6 f dI YZS dJ f dK YZT d S(L s
An extensible library for opening URLs using a variety of protocols
The simplest way to use this module is to call the urlopen function,
which accepts a string containing a URL or a Request object (described
below). It opens the URL and returns the results as file-like
object; the returned object has some extra methods described below.
The OpenerDirector manages a collection of Handler objects that do
all the actual work. Each Handler implements a particular protocol or
option. The OpenerDirector is a composite object that invokes the
Handlers needed to open the requested URL. For example, the
HTTPHandler performs HTTP GET and POST requests and deals with
non-error returns. The HTTPRedirectHandler automatically deals with
HTTP 301, 302, 303 and 307 redirect errors, and the HTTPDigestAuthHandler
deals with digest authentication.
urlopen(url, data=None) -- basic usage is that same as original
urllib. pass the url and optionally data to post to an HTTP URL, and
get a file-like object back. One difference is that you can also pass
a Request instance instead of URL. Raises a URLError (subclass of
IOError); for HTTP errors, raises an HTTPError, which can also be
treated as a valid response.
build_opener -- function that creates a new OpenerDirector instance.
will install the default handlers. accepts one or more Handlers as
arguments, either instances or Handler classes that it will
instantiate. if one of the argument is a subclass of the default
handler, the argument will be installed instead of the default.
install_opener -- installs a new opener as the default opener.
objects of interest:
OpenerDirector --
Request -- an object that encapsulates the state of a request. the
state can be a simple as the URL. it can also include extra HTTP
headers, e.g. a User-Agent.
BaseHandler --
exceptions:
URLError-- a subclass of IOError, individual protocols have their own
specific subclass
HTTPError-- also a valid HTTP response, so you can treat an HTTP error
as an exceptional event or valid response
internals:
BaseHandler and parent
_call_chain conventions
Example usage:
import urllib2
# set up authentication info
authinfo = urllib2.HTTPBasicAuthHandler()
authinfo.add_password('realm', 'host', 'username', 'password')
proxy_support = urllib2.ProxyHandler({"http" : "http://ahad-haam:3128"})
# build a new opener that adds authentication and caching FTP handlers
opener = urllib2.build_opener(proxy_support, authinfo, urllib2.CacheFTPHandler)
# install it
urllib2.install_opener(opener)
f = urllib2.urlopen('http://www.python.org/')
N( s StringIO( s unwraps unquotes splittypes splithosts
addinfourls splitports splitgophertypes
splitquerys splitattrs
ftpwrappers noheaderss splitusers splitpasswds
splitvalue( s localhosts url2pathnames
getproxiess proxy_bypasss 2.4c C s* t d j o
t a n t i | | S( N( t _openert Nonet build_openert opent urlt data( R R ( ( t /usr/lib/python2.4/urllib2.pyt urlopen~ s
c C s
| a d S( N( t openerR ( R ( ( R t install_opener s t URLErrorc B s t Z d Z d Z RS( Nc C s | f | _ | | _ d S( N( t reasont selft args( R R ( ( R t __init__ s c C s d | i S( Ns <urlopen error %s>( R R ( R ( ( R t __str__ s ( t __name__t
__module__R R ( ( ( R R
s t HTTPErrorc B s) t Z d Z e i Z d Z d Z RS( sB Raised when HTTP error occurs, but also acts like non-error returnc C sU | | _ | | _ | | _ | | _ | | _ | d j o | i | | | n d S( N( t codeR t msgt hdrst fpR t filenameR t _HTTPError__super_init( R R R R R R ( ( R R s
c C s d | i | i f S( Ns HTTP Error %s: %s( R R R ( R ( ( R R s ( R R t __doc__t
addinfourlR R R ( ( ( R R s
t GopherErrorc B s t Z RS( N( R R ( ( ( R R s t Requestc B s t Z d h d e d Z d Z d Z d Z d Z d Z d Z
d Z d Z d
Z
d Z d Z d
Z d Z d Z d Z d d Z d Z RS( Nc C s t | | _ d | _ d | _ d | _ | | _ h | _ x* | i
D] \ } } | i
| | qI Wh | _ | d j o t i | } n | | _ | | _ d S( N( t unwrapR R t _Request__originalR t typet hostt portR t headerst itemst keyt valuet
add_headert unredirected_hdrst origin_req_hostt cookielibt request_hostt unverifiable( R R R R" R( R+ R% R$ ( ( R R s
c C se | d d j oG | d } t t d | o% t | d | t | | SqX n t | d S( Ni t _Request__r_t get_( t attrt namet hasattrR t getattrR t AttributeError( R R. R/ ( ( R t __getattr__ s
c C s | i o d Sn d Sd S( Nt POSTt GET( R t has_data( R ( ( R t
get_method s
c C s
| | _ d S( N( R R ( R R ( ( R t add_data s c C s
| i d j S( N( R R R ( R ( ( R R6 s c C s | i S( N( R R ( R ( ( R t get_data s c C s | i S( N( R R ( R ( ( R t get_full_url s c C sZ | i d j oC t | i \ | _ | _ | i d j o t d | i qS n | i S( Ns unknown url type: %s( R R R t splittypeR t _Request__r_typet
ValueError( R ( ( R t get_type s
c C sV | i d j o? t | i \ | _ | _ | i o t | i | _ qO n | i S( N( R R R t splithostR< t _Request__r_hostt unquote( R ( ( R t get_host s
c C s | i S( N( R R@ ( R ( ( R t get_selector s c C s# | | | _ | _ | i | _ d S( N( R R R R R@ ( R R R ( ( R t set_proxy s c C s | i S( N( R R( ( R ( ( R t get_origin_req_host s c C s | i S( N( R R+ ( R ( ( R t is_unverifiable s c C s | | i | i <d S( N( t valR R" R$ t
capitalize( R R$ RG ( ( R R& s c C s | | i | i <d S( N( RG R R' R$ RH ( R R$ RG ( ( R t add_unredirected_header s c C s | | i j p
| | i j S( N( t header_nameR R" R' ( R RJ ( ( R t
has_header s c C s" | i i | | i i | | S( N( R R" t getRJ R' t default( R RJ RM ( ( R t
get_header s c C s) | i i } | i | i | i S( N( R R' t copyR t updateR" R# ( R R ( ( R t header_items s ( R R R t FalseR R3 R7 R8 R6 R9 R: R> RB RC RD RE RF R& RI RK RN RQ ( ( ( R R s$ t OpenerDirectorc B sM t Z d Z d Z d Z d Z d d Z d d Z d Z RS( Nc C sM d t } d | f g | _ g | _ h | _ h | _ h | _ h | _ d S( Ns Python-urllib/%ss
User-agent( t __version__t server_versionR t
addheaderst handlerst handle_opent handle_errort process_responset process_request( R RU ( ( R R s
c C s t } xYt | D]K}
|
i d } |
| } |
| d } | i d oq | i d | d } |
| d } y t | } Wn t
j
o n X| i i | h } | | i | <n^ | d j o | } t | d | } n4 | d d g j o | } t | d | } n q | i | g } | o t i | | n | i | t } q W| o$ t i | i | | i | n d S( Nt _i t errorR t handle_t responset requestt process_( RR t addedt dirt handlert metht findt it protocolt conditiont
startswitht jt kindt intR= R RY RL t lookupR1 t
setdefaultRW t bisectt insortt appendt Truet
add_parent( R Rd Rl Rb RW Rg Rk Rn Ri Rh Re ( ( R t add_handler! s<
c C s d S( N( ( R ( ( R t closeF s c G sW | i | f } x> | D]6 } t | | } | | } | d j o | Sq q Wd S( N( t chainRL Rl RW Rd R1 t meth_namet funcR
t resultR ( R Rw Rl Rx R
RW Rd Rz Ry ( ( R t _call_chainJ s
c C s t | t o t | | } n% | } | d j o | i | n | i } | d } x8 | i i
| g D]! } t | | } | | } qs W| i | | } | d } x; | i i
| g D]$ } t | | } | | | } q W| S( Nt _requestt _response( t
isinstancet fullurlt
basestringR R t reqR R8 R> Rh Rx R R[ RL t processorR1 Re t _openR_ RZ ( R R R R Rh R Rx R_ Re ( ( R R U s$
c C s} | i | i d d | } | o | Sn | i } | i | i | | d | } | o | Sn | i | i d d | S( NRM t default_openR t unknownt unknown_open( R R{ RX R Rz R> Rh ( R R R Rh Rz ( ( R R p s c G s | d d g j o1 | i d } | d } d | } d } | } n | i } | d } d } | | | f | } | i | } | o | Sn | o$ | d d f | } | i | Sn d S(
Nt httpt httpsi s
http_error_%si t _errori RM t http_error_default(
t protoR RY t dictR
Rx t http_errt orig_argsR{ Rz ( R R R
R Rx R Rz R ( ( R R] s
(
R R R Ru Rv R{ R R R R] ( ( ( R RS s
% c G sD t } t t t t t t t t g } t t d o | i
t n g } xu | D]m } xd | D]\ } t i | o% t | | o | i
| q qb t | | o | i
| qb qb WqU Wx | D] } | i | q Wx | D] } | i | q Wx8 | D]0 } t i | o
| } n | i | q W| S( s Create an opener object from a list of handlers.
The opener will use several default handlers, including support
for HTTP and FTP.
If any of the handlers passed as arguments are subclasses of the
default handlers, the default handlers will not be used.
t HTTPSN( RS R t ProxyHandlert UnknownHandlert HTTPHandlert HTTPDefaultErrorHandlert HTTPRedirectHandlert
FTPHandlert FileHandlert HTTPErrorProcessort default_classesR0 t httplibRr t HTTPSHandlert skipt klassRW t checkt inspectt isclasst
issubclassR~ t removeRu t h( RW R R R R R R ( ( R R s6
t BaseHandlerc B s) t Z d Z d Z d Z d Z RS( Ni c C s
| | _ d S( N( t parentR ( R R ( ( R Rt s c C s d S( N( ( R ( ( R Rv s c C s( t | d p t Sn | i | i j S( Nt
handler_order( R0 t otherRs R R ( R R ( ( R t __lt__ s ( R R R Rt Rv R ( ( ( R R s R c B s# t Z d Z d Z d Z e Z RS( s Process HTTP error responses.i c C sV | i | i | i } } } | d j o% | i i d | | | | | } n | S( Ni i R ( i i ( R_ R R t infoR R R R] R` ( R R` R_ R R R ( ( R t
http_response s
%( R R R R R t https_response( ( ( R R s R c B s t Z d Z RS( Nc C s" t | i | | | | d S( N( R R R: R R R R ( R R R R R R ( ( R R s ( R R R ( ( ( R R s R c B s: t Z d Z d Z d Z d Z e Z Z Z d Z RS( Ni i
c C s | i } | d j o
| d
j p | d j o6 | d j o) t | d | i d | i d
t Sn t | i
| | | | d S( s Return a Request or None in response to a redirect.
This is called by the http_error_30x methods when a
redirection response is received. If a redirection should
take place, return a new Request to allow http_error_30x to
perform the redirect. Otherwise, raise HTTPError if no-one
else should try to handle this url. Return None if you can't
but another Handler might.
i- i. i/ i3 R5 t HEADR4 R" R( R+ N( i- i. i/ i3 ( s GETR ( i- i. i/ (
R R7 t mR R t newurlR" RE Rs R R: R R ( R R R R R R" R R ( ( R t redirect_request s 4 c
C s d | j o | i d d } n) d | j o | i d d } n d St i | i | } | i } | i d p | i d p
| i d p$ t
| | | d | | | n | i | | | | | | } | d j o d Sn t | d oo | i } | _ | i | d | i j p t | | i j o) t
| i | | i | | | qn h } | _ | _ | i | d d | | <| i | i | i i | S(
Nt locationi t uris http://s https://s ftp://s) - Redirection to url '%s' is not allowedt
redirect_dicti ( R" t
getheadersR t urlparset urljoinR R: t lowert newurl_lowerRj R R R R R R t newR R0 R t visitedRL t max_repeatst lent max_redirectionst inf_msgt readRv R R (
R R R R R R" R R R R ( ( R t http_error_302 s0
0
2
so The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
(
R R R R R R t http_error_301t http_error_303t http_error_307R ( ( ( R R s 0c
C s d k l } t | \ } } | i d p d } | } n[ | i d p t d | n | i
d d } | d j o
d } n | d | !} t | \ } } | d j o t | \ } } n d } } | | | | f S( s3 Return (scheme, user, password, host/port) given a URL or an authority.
If a URL is supplied, it must have an authority (host:port) component.
According to RFC 3986, having an authority component means the URL must
have two slashes after the scheme:
>>> _parse_proxy('file:/ftp.example.com/')
Traceback (most recent call last):
ValueError: proxy URL with no authority: 'file:/ftp.example.com/'
The first three items of the returned tuple may be None.
Examples of authority parsing:
>>> _parse_proxy('proxy.example.com')
(None, None, None, 'proxy.example.com')
>>> _parse_proxy('proxy.example.com:3128')
(None, None, None, 'proxy.example.com:3128')
The authority component may optionally include userinfo (assumed to be
username:password):
>>> _parse_proxy('joe:password@proxy.example.com')
(None, 'joe', 'password', 'proxy.example.com')
>>> _parse_proxy('joe:password@proxy.example.com:3128')
(None, 'joe', 'password', 'proxy.example.com:3128')
Same examples, but with URLs instead:
>>> _parse_proxy('http://proxy.example.com/')
('http', None, None, 'proxy.example.com')
>>> _parse_proxy('http://proxy.example.com:3128/')
('http', None, None, 'proxy.example.com:3128')
>>> _parse_proxy('http://joe:password@proxy.example.com/')
('http', 'joe', 'password', 'proxy.example.com')
>>> _parse_proxy('http://joe:password@proxy.example.com:3128')
('http', 'joe', 'password', 'proxy.example.com:3128')
Everything after the authority is ignored:
>>> _parse_proxy('ftp://joe:password@proxy.example.com/rubbish:3128')
('ftp', 'joe', 'password', 'proxy.example.com')
Test for no trailing '/' case:
>>> _parse_proxy('http://joe:password@proxy.example.com')
('http', 'joe', 'password', 'proxy.example.com')
( s _splitnetloct /s //s proxy URL with no authority: %ri iN( R t _splitnetlocR; t proxyt schemet r_schemeRj R t authorityR= Rf t endt splitusert userinfot hostportt splitpasswdt usert password(
R R R R R R R R R R ( ( R t _parse_proxy> s" 1
R c B s# t Z d Z d d Z d Z RS( Nid c C sg | d j o
t } n | | _ x= | i D]/ \ } } t | d | | | | i d q0 Wd S( Ns %s_openc C s | | | | S( N( Re t rR R ( R R R Re ( ( R t <lambda> s ( t proxiesR t
getproxiesR R# R R t setattrt
proxy_open( R R R R ( ( R R s
c C s | i } t | \ } }
} } | d j o
| } n | i
| i o t | i o d Sn |
oP | oI d t
|
t
| f } t i | i } | i d d | n t
| } | i | | | | j o d Sn | i i | Sd S( Ns %s:%ss Proxy-authorizations Basic ( R R> t orig_typeR R t
proxy_typeR R R R RB R t proxy_bypassRA t user_passt base64t encodestringt stript credsR& RD R R R ( R R R R R R R R R R R ( ( R R s
( R R R R R R ( ( ( R R s
t CustomProxyc B s) t Z d d d Z d Z d Z RS( Nc C s | | _ | | _ | | _ d S( N( R R Ry t
proxy_addrt addr( R R Ry R ( ( R R s c C s& | i o | i | o d Sn d S( Ni ( R Ry R ( R R ( ( R t handle s c C s | i S( N( R R ( R ( ( R t get_proxy s ( R R R R R R ( ( ( R R s t CustomProxyHandlerc B s2 t Z d Z d Z d Z d Z d Z RS( Nid c G s
h | _ d S( N( R R ( R R ( ( R R s c C s | i } y | i | } Wn t j
o d Sn XxE | D]= } | i | o' | i | i
| i i | Sq> q> Wd S( N(
R R> R R R t KeyErrorR t pR RD R R R ( R R R R R ( ( R R s c C s | i i | S( N( R R R R ( R R R ( ( R t do_proxy s c C sE | i | i j o | i | i i | n | g | i | i <d S( N( t cpoR R R Rr ( R R ( ( R t add_proxy s ( R R R R R R R ( ( ( R R s
t HTTPPasswordMgrc B s5 t Z d Z d Z d Z d Z d Z RS( Nc C s
h | _ d S( N( R t passwd( R ( ( R R s c C sq t | t o
| g } n t t | i | } | | i j o h | i | <n | | f | i | | <d S( N(
R~ R R t tuplet mapR t
reduce_urit realmR R ( R R R R R ( ( R t add_password s
c C sq | i i | h } | i | } xF | i D]8 \ } } x) | D]! } | i | | o | SqD qD Wq1 Wd S( N( NN(
R R RL R t domainsR t authurit iteritemst urist authinfoR t is_suburiR ( R R R R R R R ( ( R t find_user_password s
c C sI t i | } | d o | d | d p d f Sn | d d f Sd S( s9 Accept netloc or URI and extract only the netloc and pathi i R N( R R t parts( R R R ( ( R R s
c C sx | | j o t Sn | d | d j o t Sn t i | d | d f } t | t | d j o t Sn t S( sc Check if test is below base in a URI tree
Both args must be URIs in reduced form.
i i N( t baset testRs RR t posixpatht commonprefixt commonR ( R R R R ( ( R R s
( R R R R R R R ( ( ( R R s
t HTTPPasswordMgrWithDefaultRealmc B s t Z d Z RS( Nc C sI t i | | | \ } } | d j o | | f Sn t i | d | S( N( R R R R R R R R ( R R R R R ( ( R R s
( R R R ( ( ( R R s t AbstractBasicAuthHandlerc B s; t Z e i d e i Z d d Z d Z d Z RS( Ns! [ ]*([^ ]+)[ ]+realm="([^"]*)"c C s6 | d j o
t } n | | _ | i i | _ d S( N( t password_mgrR R R R R ( R R ( ( R R s
c C sz | i | d } | o] t i i | } | o@ | i \ } } | i d j o | i
| | | Sqr qv n d S( Nt basic( R" RL t authreqR R t rxt searcht mot groupsR R R R t retry_http_basic_authR R ( R R R R R" R R R ( ( R t http_error_auth_reqed s c C s | i i | | i \ } } | d j ow d | | f } d t
i | i } | i i | i d | j o d Sn | i | i | | i i | Sn d Sd S( Ns %s:%ss Basic %s( R R R R R R: R t pwR t rawR R R t authR" RL t auth_headerR& R R ( R R R R R R
R R ( ( R R ( s !
(
R R t ret compilet IR R R R R ( ( ( R R s
t HTTPBasicAuthHandlerc B s t Z d Z d Z RS( Nt
Authorizationc C s/ t i | i d } | i d | | | S( Ni s www-authenticate( R R R: R R R R" ( R R R R R R" R ( ( R t http_error_4018 s ( R R R R ( ( ( R R 4 s t ProxyBasicAuthHandlerc B s t Z d Z d Z RS( Ns Proxy-authorizationc C s" | i } | i d | | | S( Ns proxy-authenticate( R RB R R R R" ( R R R R R R" R ( ( R t http_error_407B s ( R R R R ( ( ( R R > s c C s t i i d o- t d } | i | } | i | SnG g } t
| D] } | t t
i d d qQ ~ } d i | Sd S( s Return n random bytes.s /dev/urandomi i t N( t ost patht existsR t fR t nt sRv t _[1]t rangeRg t chrt randomt randranget Lt join( R R Rg R R R ( ( R t randombytesH s
9t AbstractDigestAuthHandlerc B sS t Z d d Z d Z d Z d Z d Z d Z d Z d Z
RS( Nc C sH | d j o
t } n | | _ | i i | _ d | _ d | _ d S( Ni ( R R R R R t retriedt nonce_count( R R ( ( R R a s
c C s
d | _ d S( Ni ( R R$ ( R ( ( R t reset_retry_counti s c C s | i | d } | i d j o" t | i d d | d n | i d 7_ | oK | i
d } | i d j o | i
| | Sq t d | n d S( Ni i s digest auth failedi i t digests/ AbstractDigestAuthHandler doesn't know about %s( R" RL R R R R R$ R R R: t splitR R t retry_http_digest_authR= ( R R R R R" R R ( ( R R l s c C s | i d d \ } } t t | } | i | | } | o^ d | } | i i | i
d | j o d Sn | i | i
| | i i | } | Sn d S( Nt i s Digest %s( R
R( t tokent challenget parse_keqv_listt parse_http_listt chalR t get_authorizationR t auth_valR" RL R R R& R R t resp( R R R
R, R2 R/ R+ R1 ( ( R R) s
c C s< t i d | i | t i t d f i } | d S( Ns %s:%s:%s:%si i (
t shaR R R% t noncet timet ctimeR" t hexdigestt dig( R R4 R8 ( ( R t
get_cnonce s 4c C s\ yK | d } | d } | i d } | i d d } | i d d } Wn t j
o d Sn X| i
| \ }
} |
d j o d Sn | i
i | | i \ } } | d j o d Sn | i o | i | i | } n d } d | | | f } d | i | i f }
| d j oc | i d
7_ d | i } | i | } d | | | | |
|
f } | |
| | } n7 | d j o) | |
| d | |
|
f } n d
| | | | i | f } | o | d | } n | o | d | } n | d j o | d | } n | o | d | | f } n | S( NR R4 t qopt algorithmt MD5t opaques %s:%s:%ss %s:%sR
i s %08xs %s:%s:%s:%s:%ss> username="%s", realm="%s", nonce="%s", uri="%s", response="%s"s
, opaque="%s"s
, digest="%s"s , algorithm="%s"s , qop=auth, nc=%s, cnonce="%s"(" R/ R R4 RL R: R; R R= R R t get_algorithm_implst Ht KDR R R R: R R R6 t get_entity_digestR9 t entdigt A1R7 RC t A2R% t ncvalueR9 t cnoncet noncebitt respdigR ( R R R/ R: RB R R R= RH RC RD RG R4 R? RF RE R R R; R@ ( ( R R0 sL
!
)
c sJ | d j o
d n | d j o
d n d } | f S( NR< c C s t i | i S( N( t md5R t xR7 ( RJ ( ( R R s t SHAc C s t i | i S( N( R3 R RJ R7 ( RJ ( ( R R s c s d | | f S( Ns %s:%s( R? R t d( R RL ( R? ( R R s ( R; R? R@ ( R R; R? R@ ( ( R? R R> s
c C s d S( N( R ( R R R/ ( ( R RA s ( R R R R R& R R) R9 R0 R> RA ( ( ( R R# V s
9
t HTTPDigestAuthHandlerc B s t Z d Z d Z d Z RS( s An authentication protocol defined by RFC 2069
Digest authentication improves on basic authentication because it
does not transmit passwords in the clear.
R c C s? t i | i d } | i d | | | } | i | S( Ni s www-authenticate( R R R: R R R R" t retryR& ( R R R R R R" R RN ( ( R R s
( R R R R R ( ( ( R RM s t ProxyDigestAuthHandlerc B s t Z d Z d Z RS( Ns Proxy-Authorizationc C s2 | i } | i d | | | } | i | S( Ns proxy-authenticate( R RB R R R R" RN R& ( R R R R R R" R RN ( ( R R s
( R R R R ( ( ( R RO s t _WrapForRecvc B s t Z d Z d Z d Z RS( se Simple wrapper for urllib2 so we don't have to do r.read = r.recv and
create circular refs. c C s
| | _ d S( N( t objR t _WrapForRecv__obj( R RQ ( ( R R s c C s' | d j o
d } n t | i | S( Nt recvR ( R/ R1 R RR ( R R/ ( ( R R3 s
( R R R R R3 ( ( ( R RP s t AbstractHTTPHandlerc B s/ t Z d d Z d Z d Z d Z RS( Ni c C s
| | _ d S( N( t
debuglevelR t _debuglevel( R RU ( ( R R s c C s
| | _ d S( N( t levelR RV ( R RW ( ( R t set_http_debuglevel s c
C s8 | i } | p t d n | i ob | i } | i d p | i d d n | i d p | i d d t | q n t
| i \ } } t | \ } } | i d p | i d | p | n xJ | i i D]< \ } } | i } | i | p | i | | q q W| S( Ns
no host givens Content-types! application/x-www-form-urlencodeds Content-lengths %dt Host( R` RB R R
R6 R9 R RK RI R R; RC R t selR? t sel_hostt sel_pathR R RV R/ R% RH (
R R` RZ R/ R R% R\ R R R[ ( ( R t do_request_ s(
"
c
C s | i } | p t d n | | } | i | i t | i
} | i | i d | d <y5 | i
| i | i | i | | i } Wn$ t i j
o } t | n Xt | } t i | } t | | i | i } | i | _ | i | _ | S( s Return an addinfourl object for the request, using http_class.
http_class must implement the HTTPConnection API from httplib.
The addinfourl return value is a file-like object. It also
has methods and attributes including:
- info(): return a mimetools.Message object for the headers
- geturl(): return the original request URL
- code: HTTP status code
s
no host givenRv t
ConnectionN( R RB R R
t
http_classR t set_debuglevelR RV R R" RP R' R` R7 RC R t getresponseR t socketR] t errRP t _fileobjectR R R R: R2 t statusR R (
R R_ R R Rc R R" R R R2 ( ( R t do_open& s(
% ( R R R RX R] Rf ( ( ( R RT s R c B s t Z d Z e i Z RS( Nc C s | i t i | S( N( R Rf R t HTTPConnectionR ( R R ( ( R t http_open] s ( R R Rh RT R] t http_request( ( ( R R [ s R R c B s t Z d Z e i Z RS( Nc C s | i t i | S( N( R Rf R t HTTPSConnectionR ( R R ( ( R t
https_opene s ( R R Rk RT R] t
https_request( ( ( R R c s t HTTPCookieProcessorc B s2 t Z d d Z d Z d Z e Z e Z RS( Nc C s* | d j o t i } n | | _ d S( N( t cookiejarR R) t CookieJarR ( R Rn ( ( R R k s
c C s | i i | | S( N( R Rn t add_cookie_headerR` ( R R` ( ( R Ri p s c C s | i i | | | S( N( R Rn t extract_cookiesR_ R` ( R R` R_ ( ( R R t s ( R R R R Ri R Rl R ( ( ( R Rm j s
R c B s t Z d Z RS( Nc C s | i } t d | d S( Ns unknown url type: %s( R R> R R
( R R R ( ( R R | s ( R R R ( ( ( R R { s c C sp h } xc | D][ } | i d d \ } } | d d j o" | d d j o | d d !} n | | | <q
W| S( s> Parse list of key=value strings where keys are not duplicated.t =i i t "iN( t parsedt lt eltR( t kt v( Ru Rw Rx Rv Rt ( ( R R- s "c C s g } d } t } } x | D] } | o | | 7} t } q n | oB | d j o
t } q n | d j o
t } n | | 7} q n | d j o | i | d } q n | d j o
t } n | | 7} q W| o | i | n g } | D] } | | i
q ~ S( sp Parse lists as described by RFC 2068 Section 2.
In particular, parse comma-separated lists where the elements of
the list may include quoted-strings. A quoted-string could
contain a comma. A non-quoted string could have quotes in the
middle. Neither commas nor quotes count if they are escaped.
Only double-quotes count, not single-quotes.
R s \Rs t ,N( t rest partRR t escapet quoteR t curRs Rr R R ( R R~ Rz R| R R{ R} ( ( R R. s8
R c B s) t Z d Z d Z d Z d Z RS( Nc C s_ | i } | d d j o1 | d d !d j o d | _ | i i | Sn | i | Sd S( Ni s //i R t ftp( R RC R R R R R t open_local_file( R R R ( ( R t file_open s
% c C sB t i d j o+ t i d t i t i f t _ n t i S( Nt localhost( R t namesR Rb t
gethostbynamet gethostname( R ( ( R t get_names s +c C s d k } | i } | i } t | } t i
| } | i } | i i | i d t }
t i | d } t i t d | p d | |
f } | o t | \ } } n | p$ | o= t i | | i j o! t t! | d | d | Sn t" d d S( Nt usegmti s6 Content-type: %s
Content-length: %d
Last-modified: %s
s
text/plaint rbs file:s file not on local host(# t email.Utilst emailR RB R RC t filet url2pathnamet localfileR t statt statst st_sizet sizet Utilst
formatdatet st_mtimeRs t modifiedt mimetypest
guess_typet mtypet mimetoolst Messaget StringIOR" t splitportR! Rb R R R R R R
( R R R R R! R R R R R R R" ( ( R R s ),( R R R R R R R ( ( ( R R s R c B s t Z d Z d Z RS( Nc C s | i } | p
t d n t | \ } } | d j o
t i } n
t | } t
| \ }
} |
o t |
\ }
}
n d }
t | } t |
p d }
t |
p d }
y t i | } Wn$ t i j
o } t | n Xt | i \ } } | i d } t t | } | d | d } } | o | d o | d } n y3| i |
|
| | | } | o d p d } xP | D]H } t! | \ } } | i# d
j o | d j o | i$ } qqW| i% | | \ } } d } t) i* | i+ d } | o | d | 7} n | d j o | d j o | d | 7} n t- | } t/ i0 | } t1 | | | i+ SWn4 t i2 j
o% } t d | f t3 i4 d n Xd S( Ns ftp errors
no host givenR R ii i R t DR t at ARg RL s Content-type: %s
s Content-length: %d
i ( s ftp errors
no host given( R R Rg R RL R (5 R RB R t IOErrorR R! R t ftplibt FTP_PORTRm R R R R RA Rb R R] R R
t splitattrRC R t attrsR( t dirsR R R t connect_ftpt fwR R. t
splitvalueR% R t uppert retrfileR t retrlenR" R R R: R R t sfR R R t
all_errorst syst exc_info( R R R R R R R! R R R R R R R R R R. R% R" R ( ( R t ftp_open sX
c C s t | | | | | } | S( N( t
ftpwrapperR R R R! R R ( R R R R R! R R ( ( R R s ( R R R R ( ( ( R R s 3t CacheFTPHandlerc B s5 t Z d Z d Z d Z d Z d Z RS( Nc C s1 h | _ h | _ d | _ d | _ d | _ d S( Ni i< i ( R t cachet timeoutt soonestt delayt max_conns( R ( ( R R s
c C s
| | _ d S( N( t tR R ( R R ( ( R t
setTimeout s c C s
| | _ d S( N( R R R ( R R ( ( R t setMaxConns" s c C s | | | d i | f } | | i j o t i | i | i
| <n: t | | | | | | i | <t i | i | i
| <| i
| i | S( NR ( R R R! R! R R$ R R R5 R R R R t check_cache( R R R R R! R R$ ( ( R R % s
c C s t i } | i | j oZ xW | i i D]B \ } } | | j o) | i | i | i | =| i | =q, q, Wn t
| i i | _ t | i | i
j oe xF | i i D]5 \ } } | | i j o | i | =| i | =Pq q Wt
| i i | _ n d S( N( R5 R R R R R# Rw Rx R Rv t mint valuesR R ( R Rw R Rx ( ( R R / s"
( R R R R R R R ( ( ( R R s
t
GopherHandlerc B s t Z d Z RS( Nc C s | i } | p t d n t | } | i } t | \ } } t | \ } } t | } | o% t | } t i | | | } n t i | | } t | t | i S( Ns
no host given( R RB R R RA RC t selectort splitgophertypeR t
splitqueryt queryt gopherlibt
send_queryR t
send_selectorR t noheadersR: ( R R R R R R R ( ( R t gopher_openD s ( R R R ( ( ( R R C s t
OpenerFactoryc B sG t Z e e e e e e g Z g Z g Z
d Z d Z d Z
RS( Nc C s | i | g | _ d S( N( R RW R ( R R ( ( R Ru ] s c C s d S( N( ( R R ( ( R t replace_handler` s c C sK t } x; | i D]0 } t i | o
| } n | i | q Wd S( N( RS R R t default_handlerst phR R Ru ( R R R ( ( R R c s
( R R R R R R R R R RW t replacement_handlersRu R R ( ( ( R R U s (U R R R R R R RI R R R R R R R3 Rb R R5 R Rp R) t cStringIOR t ImportErrort urllibR RA R; R? R R R R R R R R R R R R R R RT R R R R R R
R R R RS R R R R R R R R R R R R R R R" R# RM RO RP RT R R0 R Rm R R- R. R R R R R (M R RA R R- R R R R R? R R R RM R R R R R R R R R; R R R R R Rm R R R R R R R R RP R R
R R RO R R R# R R R. R" R R RS R R R R R) RI R R RT R Rb R R Rp R R R R3 R R R R5 RT R R ( ( R t ?G s [
e $\ I( -
$
W
+'9.