Lines Matching +full:decode +full:- +full:uri +full:- +full:component
1 :mod:`urllib.request` --- Extensible library for opening URLs
13 --------------
16 opening URLs (mostly HTTP) in a complex world --- basic and digest
22 is recommended for a higher-level HTTP client interface.
24 .. include:: ../includes/wasm-notavail.rst
66 attribute --- the reason phrase returned by server --- instead of
91 .. audit-event:: urllib.Request fullurl,data,headers,method urllib.request.urlopen
158 the path component of a URL. This does not produce a complete URL. The return
164 Convert the path component *path* from a percent-encoded URL to the local syntax for a
166 :func:`~urllib.parse.unquote` to decode *path*.
200 types include bytes, file-like objects, and iterables of bytes-like objects.
201 If no ``Content-Length`` nor ``Transfer-Encoding`` header field
203 to the type of *data*. ``Content-Length`` will be used to send
204 bytes objects, while ``Transfer-Encoding: chunked`` as specified in
208 standard :mimetype:`application/x-www-form-urlencoded` format. The
210 of 2-tuples and returns an ASCII string in this format. It should
215 This is often used to "spoof" the ``User-Agent`` header value, which is
216 used by a browser to identify itself -- some HTTP servers only
221 ``"Python-urllib/2.6"`` (on Python 2.6).
224 An appropriate ``Content-Type`` header should be included if the *data*
226 is not None, ``Content-Type: application/x-www-form-urlencoded`` will
230 of third-party HTTP cookies:
232 *origin_req_host* should be the request-host of the origin
237 this should be the request-host of the request for the page
260 a 100-continue expectation in the library.
269 Do not raise an error if the ``Content-Length`` has not been
281 This is the base class for all registered handlers --- and handles only the
314 which shouldn't be reached via proxy; if set, it should be a comma-separated
326 Keep a database of ``(realm, uri) -> (user, password)`` mappings.
331 Keep a database of ``(realm, uri) -> (user, password)`` mappings. A realm of
332 ``None`` is considered a catch-all realm, which is searched if no other realm
339 database of ``uri -> is_authenticated`` mappings. Can be used by a
351 :ref:`http-password-mgr` for information on the interface that must be
354 :ref:`http-password-mgr-with-prior-auth`), then the handler will use the
355 ``is_authenticated`` result for a given URI to determine whether or not to
357 returns ``True`` for the URI, credentials are sent. If ``is_authenticated``
359 received the request is re-sent with the authentication credentials. If
361 ``is_authenticated`` ``True`` for the URI, so that subsequent requests to
362 the URI or any of its super-URIs will automatically include the
373 section :ref:`http-password-mgr` for information on the interface that must
382 :ref:`http-password-mgr` for information on the interface that must be
391 :ref:`http-password-mgr` for information on the interface that must be
399 section :ref:`http-password-mgr` for information on the interface that must
416 :ref:`http-password-mgr` for information on the interface that must be
456 A catch-all class to handle unknown URLs.
464 .. _request-objects:
467 ---------------
486 The URI scheme.
490 The URI authority, typically a host, but may also contain a port
499 The URI path. If the :class:`Request` uses a proxy, then selector
507 Changing value of :attr:`Request.data` now deletes "Content-Length"
550 meaning when used more than once have a (header-specific) way of gaining the
606 .. _opener-director-objects:
609 ----------------------
624 * :meth:`<protocol>_open` --- signal that the handler knows how to open *protocol*
629 * :meth:`http_error_\<type\>` --- signal that the handler knows how to handle HTTP
634 * :meth:`<protocol>_error` --- signal that the handler knows how to handle errors
635 from (non-\ ``http``) *protocol*.
637 * :meth:`<protocol>_request` --- signal that the handler knows how to pre-process
642 * :meth:`<protocol>_response` --- signal that the handler knows how to
643 post-process *protocol* responses.
680 method called to pre-process the request.
683 the request. This stage ends when a handler either returns a non-\ :const:`None`
698 method called to post-process the response.
701 .. _base-handler-objects:
704 -------------------
742 :class:`OpenerDirector`. It should return a file-like object as described in
748 This method will be called before any protocol-specific open method.
776 override it if they intend to provide a catch-all for otherwise unhandled HTTP
780 *req* will be a :class:`Request` object, *fp* will be a file-like object with
781 the HTTP error body, *code* will be the three-digit code of the error, *msg*
782 will be the user-visible explanation of the code and *hdrs* will be a mapping
792 *nnn* should be a three-digit HTTP error code. This method is also not defined
807 define it if they want to pre-process requests of the given protocol.
819 define it if they want to post-process responses of the given protocol.
828 .. _http-redirect-handler:
831 ---------------------------
865 Redirect to the ``Location:`` or ``URI:`` URL. This method is called by the
895 .. _http-cookie-processor:
898 ---------------------------
907 .. _proxy-handler:
910 --------------------
923 .. _http-password-mgr:
926 -----------------------
932 .. method:: HTTPPasswordMgr.add_password(realm, uri, user, passwd)
934 *uri* can be either a single URI, or a sequence of URIs. *realm*, *user* and
936 authentication tokens when authentication for *realm* and a super-URI of any of
942 Get user/password for given realm and URI, if any. This method will return
949 .. _http-password-mgr-with-prior-auth:
952 ------------------------------------
958 .. method:: HTTPPasswordMgrWithPriorAuth.add_password(realm, uri, user, \
961 *realm*, *uri*, *user*, *passwd* are as for
963 value of the ``is_authenticated`` flag for the given URI or list of URIs.
972 .. method:: HTTPPasswordMgrWithPriorAuth.update_authenticated(self, uri, \
975 Update the ``is_authenticated`` flag for the given *uri* or list
982 the given URI.
985 .. _abstract-basic-auth-handler:
988 --------------------------------
993 Handle an authentication request by getting a user/password pair, and re-trying
1000 authority component (e.g. ``"http://python.org/"``). In either case, the
1001 authority must not contain a userinfo component (so, ``"python.org"`` and
1005 .. _http-basic-auth-handler:
1008 ----------------------------
1016 .. _proxy-basic-auth-handler:
1019 -----------------------------
1027 .. _abstract-digest-auth-handler:
1030 ---------------------------------
1041 .. _http-digest-auth-handler:
1044 -----------------------------
1052 .. _proxy-digest-auth-handler:
1055 ------------------------------
1063 .. _http-handler-objects:
1066 -------------------
1075 .. _https-handler-objects:
1078 --------------------
1087 .. _file-handler-objects:
1090 -------------------
1103 .. _data-handler-objects:
1106 -------------------
1118 .. _ftp-handler-objects:
1121 ------------------
1130 .. _cacheftp-handler-objects:
1133 -----------------------
1149 .. _unknown-handler-objects:
1152 ----------------------
1160 .. _http-error-processor-objects:
1163 --------------------------
1171 For non-200 error codes, this simply passes the job on to the
1184 .. _urllib-request-examples:
1187 --------
1190 :ref:`urllib-howto`.
1199 b'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
1200 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n\n\n<html
1202 <meta http-equiv="content-type" content="text/html; charset=utf-8" />\n
1207 it receives from the HTTP server. In general, a program will decode
1211 The following W3C document, https://www.w3.org/International/O-charset\ , lists
1215 As the python.org website uses *utf-8* encoding as specified in its meta tag, we
1219 ... print(f.read(100).decode('utf-8'))
1221 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
1229 >>> print(f.read(100).decode('utf-8'))
1230 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
1233 In the following example, we are sending a data-stream to the stdin of a CGI
1238 >>> req = urllib.request.Request(url='https://localhost/cgi-bin/test.cgi',
1241 ... print(f.read().decode('utf-8'))
1250 print('Content-type: text/plain\n\nGot Data: "%s"' % data)
1268 uri='https://mahler:8092/site-updates.py',
1301 # Customize the default User-Agent header value:
1302 req.add_header('User-Agent', 'urllib-example/0.1 (Contact: . . .)')
1305 :class:`OpenerDirector` automatically adds a :mailheader:`User-Agent` header to
1310 opener.addheaders = [('User-agent', 'Mozilla/5.0')]
1313 Also, remember that a few standard headers (:mailheader:`Content-Length`,
1314 :mailheader:`Content-Type` and :mailheader:`Host`)
1318 .. _urllib-examples:
1326 >>> url = "http://www.musi-cal.com/cgi-bin/query?%s" % params
1328 ... print(f.read().decode('utf-8'))
1339 ... print(f.read().decode('utf-8'))
1349 ... f.read().decode('utf-8')
1357 ... f.read().decode('utf-8')
1362 ----------------
1383 third argument may be ``-1`` on older FTP servers which do not return a file
1396 :mimetype:`application/x-www-form-urlencoded` format; see the
1401 size reported by a *Content-Length* header). This can occur, for example, when
1404 The *Content-Length* is treated as a lower bound: if there's more data to read,
1411 If no *Content-Length* header was supplied, urlretrieve can not check the size
1428 By default, the :class:`URLopener` class sends a :mailheader:`User-Agent` header
1430 Applications can define their own :mailheader:`User-Agent` header by subclassing
1469 local file, the input filename is returned. If the URL is non-local and
1471 with a suffix that matches the suffix of the last path component of the input
1474 …(-1 if unknown). It will be called once at the start and after each chunk of data is read from the
1480 :mimetype:`application/x-www-form-urlencoded` format; see the
1537 ----------------------------------
1554 file can't be opened, the URL is re-interpreted using the FTP protocol. This
1569 header, which can be inspected by looking at the :mailheader:`Content-Type`
1585 listing for the unreadable file. If fine-grained control is needed, consider
1591 :mod:`urllib.response` --- Response classes used by urllib
1598 minimal file-like interface, including ``read()`` and ``readline()``.