1:mod:`!http.client` --- HTTP protocol client 2============================================ 3 4.. module:: http.client 5 :synopsis: HTTP and HTTPS protocol client (requires sockets). 6 7**Source code:** :source:`Lib/http/client.py` 8 9.. index:: 10 pair: HTTP; protocol 11 single: HTTP; http.client (standard module) 12 13.. index:: pair: module; urllib.request 14 15-------------- 16 17This module defines classes that implement the client side of the HTTP and 18HTTPS protocols. It is normally not used directly --- the module 19:mod:`urllib.request` uses it to handle URLs that use HTTP and HTTPS. 20 21.. seealso:: 22 23 The `Requests package <https://requests.readthedocs.io/en/latest/>`_ 24 is recommended for a higher-level HTTP client interface. 25 26.. note:: 27 28 HTTPS support is only available if Python was compiled with SSL support 29 (through the :mod:`ssl` module). 30 31.. include:: ../includes/wasm-notavail.rst 32 33The module provides the following classes: 34 35 36.. class:: HTTPConnection(host, port=None[, timeout], source_address=None, \ 37 blocksize=8192) 38 39 An :class:`HTTPConnection` instance represents one transaction with an HTTP 40 server. It should be instantiated by passing it a host and optional port 41 number. If no port number is passed, the port is extracted from the host 42 string if it has the form ``host:port``, else the default HTTP port (80) is 43 used. If the optional *timeout* parameter is given, blocking 44 operations (like connection attempts) will timeout after that many seconds 45 (if it is not given, the global default timeout setting is used). 46 The optional *source_address* parameter may be a tuple of a (host, port) 47 to use as the source address the HTTP connection is made from. 48 The optional *blocksize* parameter sets the buffer size in bytes for 49 sending a file-like message body. 50 51 For example, the following calls all create instances that connect to the server 52 at the same host and port:: 53 54 >>> h1 = http.client.HTTPConnection('www.python.org') 55 >>> h2 = http.client.HTTPConnection('www.python.org:80') 56 >>> h3 = http.client.HTTPConnection('www.python.org', 80) 57 >>> h4 = http.client.HTTPConnection('www.python.org', 80, timeout=10) 58 59 .. versionchanged:: 3.2 60 *source_address* was added. 61 62 .. versionchanged:: 3.4 63 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are 64 no longer supported. 65 66 .. versionchanged:: 3.7 67 *blocksize* parameter was added. 68 69 70.. class:: HTTPSConnection(host, port=None, *[, timeout], \ 71 source_address=None, context=None, \ 72 blocksize=8192) 73 74 A subclass of :class:`HTTPConnection` that uses SSL for communication with 75 secure servers. Default port is ``443``. If *context* is specified, it 76 must be a :class:`ssl.SSLContext` instance describing the various SSL 77 options. 78 79 Please read :ref:`ssl-security` for more information on best practices. 80 81 .. versionchanged:: 3.2 82 *source_address*, *context* and *check_hostname* were added. 83 84 .. versionchanged:: 3.2 85 This class now supports HTTPS virtual hosts if possible (that is, 86 if :const:`ssl.HAS_SNI` is true). 87 88 .. versionchanged:: 3.4 89 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are 90 no longer supported. 91 92 .. versionchanged:: 3.4.3 93 This class now performs all the necessary certificate and hostname checks 94 by default. To revert to the previous, unverified, behavior 95 :func:`!ssl._create_unverified_context` can be passed to the *context* 96 parameter. 97 98 .. versionchanged:: 3.8 99 This class now enables TLS 1.3 100 :attr:`ssl.SSLContext.post_handshake_auth` for the default *context* or 101 when *cert_file* is passed with a custom *context*. 102 103 .. versionchanged:: 3.10 104 This class now sends an ALPN extension with protocol indicator 105 ``http/1.1`` when no *context* is given. Custom *context* should set 106 ALPN protocols with :meth:`~ssl.SSLContext.set_alpn_protocols`. 107 108 .. versionchanged:: 3.12 109 The deprecated *key_file*, *cert_file* and *check_hostname* parameters 110 have been removed. 111 112 113.. class:: HTTPResponse(sock, debuglevel=0, method=None, url=None) 114 115 Class whose instances are returned upon successful connection. Not 116 instantiated directly by user. 117 118 .. versionchanged:: 3.4 119 The *strict* parameter was removed. HTTP 0.9 style "Simple Responses" are 120 no longer supported. 121 122This module provides the following function: 123 124.. function:: parse_headers(fp) 125 126 Parse the headers from a file pointer *fp* representing a HTTP 127 request/response. The file has to be a :class:`~io.BufferedIOBase` reader 128 (i.e. not text) and must provide a valid :rfc:`2822` style header. 129 130 This function returns an instance of :class:`http.client.HTTPMessage` 131 that holds the header fields, but no payload 132 (the same as :attr:`HTTPResponse.msg` 133 and :attr:`http.server.BaseHTTPRequestHandler.headers`). 134 After returning, the file pointer *fp* is ready to read the HTTP body. 135 136 .. note:: 137 :meth:`parse_headers` does not parse the start-line of a HTTP message; 138 it only parses the ``Name: value`` lines. The file has to be ready to 139 read these field lines, so the first line should already be consumed 140 before calling the function. 141 142The following exceptions are raised as appropriate: 143 144 145.. exception:: HTTPException 146 147 The base class of the other exceptions in this module. It is a subclass of 148 :exc:`Exception`. 149 150 151.. exception:: NotConnected 152 153 A subclass of :exc:`HTTPException`. 154 155 156.. exception:: InvalidURL 157 158 A subclass of :exc:`HTTPException`, raised if a port is given and is either 159 non-numeric or empty. 160 161 162.. exception:: UnknownProtocol 163 164 A subclass of :exc:`HTTPException`. 165 166 167.. exception:: UnknownTransferEncoding 168 169 A subclass of :exc:`HTTPException`. 170 171 172.. exception:: UnimplementedFileMode 173 174 A subclass of :exc:`HTTPException`. 175 176 177.. exception:: IncompleteRead 178 179 A subclass of :exc:`HTTPException`. 180 181 182.. exception:: ImproperConnectionState 183 184 A subclass of :exc:`HTTPException`. 185 186 187.. exception:: CannotSendRequest 188 189 A subclass of :exc:`ImproperConnectionState`. 190 191 192.. exception:: CannotSendHeader 193 194 A subclass of :exc:`ImproperConnectionState`. 195 196 197.. exception:: ResponseNotReady 198 199 A subclass of :exc:`ImproperConnectionState`. 200 201 202.. exception:: BadStatusLine 203 204 A subclass of :exc:`HTTPException`. Raised if a server responds with a HTTP 205 status code that we don't understand. 206 207 208.. exception:: LineTooLong 209 210 A subclass of :exc:`HTTPException`. Raised if an excessively long line 211 is received in the HTTP protocol from the server. 212 213 214.. exception:: RemoteDisconnected 215 216 A subclass of :exc:`ConnectionResetError` and :exc:`BadStatusLine`. Raised 217 by :meth:`HTTPConnection.getresponse` when the attempt to read the response 218 results in no data read from the connection, indicating that the remote end 219 has closed the connection. 220 221 .. versionadded:: 3.5 222 Previously, :exc:`BadStatusLine`\ ``('')`` was raised. 223 224 225The constants defined in this module are: 226 227.. data:: HTTP_PORT 228 229 The default port for the HTTP protocol (always ``80``). 230 231.. data:: HTTPS_PORT 232 233 The default port for the HTTPS protocol (always ``443``). 234 235.. data:: responses 236 237 This dictionary maps the HTTP 1.1 status codes to the W3C names. 238 239 Example: ``http.client.responses[http.client.NOT_FOUND]`` is ``'Not Found'``. 240 241See :ref:`http-status-codes` for a list of HTTP status codes that are 242available in this module as constants. 243 244 245.. _httpconnection-objects: 246 247HTTPConnection Objects 248---------------------- 249 250:class:`HTTPConnection` instances have the following methods: 251 252 253.. method:: HTTPConnection.request(method, url, body=None, headers={}, *, \ 254 encode_chunked=False) 255 256 This will send a request to the server using the HTTP request 257 method *method* and the request URI *url*. The provided *url* must be 258 an absolute path to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>` 259 (unless connecting to an HTTP proxy server or using the ``OPTIONS`` or 260 ``CONNECT`` methods). 261 262 If *body* is specified, the specified data is sent after the headers are 263 finished. It may be a :class:`str`, a :term:`bytes-like object`, an 264 open :term:`file object`, or an iterable of :class:`bytes`. If *body* 265 is a string, it is encoded as ISO-8859-1, the default for HTTP. If it 266 is a bytes-like object, the bytes are sent as is. If it is a :term:`file 267 object`, the contents of the file is sent; this file object should 268 support at least the ``read()`` method. If the file object is an 269 instance of :class:`io.TextIOBase`, the data returned by the ``read()`` 270 method will be encoded as ISO-8859-1, otherwise the data returned by 271 ``read()`` is sent as is. If *body* is an iterable, the elements of the 272 iterable are sent as is until the iterable is exhausted. 273 274 The *headers* argument should be a mapping of extra HTTP headers to send 275 with the request. A :rfc:`Host header <2616#section-14.23>` 276 must be provided to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>` 277 (unless connecting to an HTTP proxy server or using the ``OPTIONS`` or 278 ``CONNECT`` methods). 279 280 If *headers* contains neither Content-Length nor Transfer-Encoding, 281 but there is a request body, one of those 282 header fields will be added automatically. If 283 *body* is ``None``, the Content-Length header is set to ``0`` for 284 methods that expect a body (``PUT``, ``POST``, and ``PATCH``). If 285 *body* is a string or a bytes-like object that is not also a 286 :term:`file <file object>`, the Content-Length header is 287 set to its length. Any other type of *body* (files 288 and iterables in general) will be chunk-encoded, and the 289 Transfer-Encoding header will automatically be set instead of 290 Content-Length. 291 292 The *encode_chunked* argument is only relevant if Transfer-Encoding is 293 specified in *headers*. If *encode_chunked* is ``False``, the 294 HTTPConnection object assumes that all encoding is handled by the 295 calling code. If it is ``True``, the body will be chunk-encoded. 296 297 For example, to perform a ``GET`` request to ``https://docs.python.org/3/``:: 298 299 >>> import http.client 300 >>> host = "docs.python.org" 301 >>> conn = http.client.HTTPSConnection(host) 302 >>> conn.request("GET", "/3/", headers={"Host": host}) 303 >>> response = conn.getresponse() 304 >>> print(response.status, response.reason) 305 200 OK 306 307 .. note:: 308 Chunked transfer encoding has been added to the HTTP protocol 309 version 1.1. Unless the HTTP server is known to handle HTTP 1.1, 310 the caller must either specify the Content-Length, or must pass a 311 :class:`str` or bytes-like object that is not also a file as the 312 body representation. 313 314 .. versionchanged:: 3.2 315 *body* can now be an iterable. 316 317 .. versionchanged:: 3.6 318 If neither Content-Length nor Transfer-Encoding are set in 319 *headers*, file and iterable *body* objects are now chunk-encoded. 320 The *encode_chunked* argument was added. 321 No attempt is made to determine the Content-Length for file 322 objects. 323 324.. method:: HTTPConnection.getresponse() 325 326 Should be called after a request is sent to get the response from the server. 327 Returns an :class:`HTTPResponse` instance. 328 329 .. note:: 330 331 Note that you must have read the whole response before you can send a new 332 request to the server. 333 334 .. versionchanged:: 3.5 335 If a :exc:`ConnectionError` or subclass is raised, the 336 :class:`HTTPConnection` object will be ready to reconnect when 337 a new request is sent. 338 339 340.. method:: HTTPConnection.set_debuglevel(level) 341 342 Set the debugging level. The default debug level is ``0``, meaning no 343 debugging output is printed. Any value greater than ``0`` will cause all 344 currently defined debug output to be printed to stdout. The ``debuglevel`` 345 is passed to any new :class:`HTTPResponse` objects that are created. 346 347 .. versionadded:: 3.1 348 349 350.. method:: HTTPConnection.set_tunnel(host, port=None, headers=None) 351 352 Set the host and the port for HTTP Connect Tunnelling. This allows running 353 the connection through a proxy server. 354 355 The *host* and *port* arguments specify the endpoint of the tunneled connection 356 (i.e. the address included in the CONNECT request, *not* the address of the 357 proxy server). 358 359 The *headers* argument should be a mapping of extra HTTP headers to send with 360 the CONNECT request. 361 362 As HTTP/1.1 is used for HTTP CONNECT tunnelling request, `as per the RFC 363 <https://datatracker.ietf.org/doc/html/rfc7231#section-4.3.6>`_, a HTTP ``Host:`` 364 header must be provided, matching the authority-form of the request target 365 provided as the destination for the CONNECT request. If a HTTP ``Host:`` 366 header is not provided via the headers argument, one is generated and 367 transmitted automatically. 368 369 For example, to tunnel through a HTTPS proxy server running locally on port 370 8080, we would pass the address of the proxy to the :class:`HTTPSConnection` 371 constructor, and the address of the host that we eventually want to reach to 372 the :meth:`~HTTPConnection.set_tunnel` method:: 373 374 >>> import http.client 375 >>> conn = http.client.HTTPSConnection("localhost", 8080) 376 >>> conn.set_tunnel("www.python.org") 377 >>> conn.request("HEAD","/index.html") 378 379 .. versionadded:: 3.2 380 381 .. versionchanged:: 3.12 382 HTTP CONNECT tunnelling requests use protocol HTTP/1.1, upgraded from 383 protocol HTTP/1.0. ``Host:`` HTTP headers are mandatory for HTTP/1.1, so 384 one will be automatically generated and transmitted if not provided in 385 the headers argument. 386 387 388.. method:: HTTPConnection.get_proxy_response_headers() 389 390 Returns a dictionary with the headers of the response received from 391 the proxy server to the CONNECT request. 392 393 If the CONNECT request was not sent, the method returns ``None``. 394 395 .. versionadded:: 3.12 396 397 398.. method:: HTTPConnection.connect() 399 400 Connect to the server specified when the object was created. By default, 401 this is called automatically when making a request if the client does not 402 already have a connection. 403 404 .. audit-event:: http.client.connect self,host,port http.client.HTTPConnection.connect 405 406 407.. method:: HTTPConnection.close() 408 409 Close the connection to the server. 410 411 412.. attribute:: HTTPConnection.blocksize 413 414 Buffer size in bytes for sending a file-like message body. 415 416 .. versionadded:: 3.7 417 418 419As an alternative to using the :meth:`~HTTPConnection.request` method described above, you can 420also send your request step by step, by using the four functions below. 421 422 423.. method:: HTTPConnection.putrequest(method, url, skip_host=False, \ 424 skip_accept_encoding=False) 425 426 This should be the first call after the connection to the server has been 427 made. It sends a line to the server consisting of the *method* string, 428 the *url* string, and the HTTP version (``HTTP/1.1``). To disable automatic 429 sending of ``Host:`` or ``Accept-Encoding:`` headers (for example to accept 430 additional content encodings), specify *skip_host* or *skip_accept_encoding* 431 with non-False values. 432 433 434.. method:: HTTPConnection.putheader(header, argument[, ...]) 435 436 Send an :rfc:`822`\ -style header to the server. It sends a line to the server 437 consisting of the header, a colon and a space, and the first argument. If more 438 arguments are given, continuation lines are sent, each consisting of a tab and 439 an argument. 440 441 442.. method:: HTTPConnection.endheaders(message_body=None, *, encode_chunked=False) 443 444 Send a blank line to the server, signalling the end of the headers. The 445 optional *message_body* argument can be used to pass a message body 446 associated with the request. 447 448 If *encode_chunked* is ``True``, the result of each iteration of 449 *message_body* will be chunk-encoded as specified in :rfc:`7230`, 450 Section 3.3.1. How the data is encoded is dependent on the type of 451 *message_body*. If *message_body* implements the :ref:`buffer interface 452 <bufferobjects>` the encoding will result in a single chunk. 453 If *message_body* is a :class:`collections.abc.Iterable`, each iteration 454 of *message_body* will result in a chunk. If *message_body* is a 455 :term:`file object`, each call to ``.read()`` will result in a chunk. 456 The method automatically signals the end of the chunk-encoded data 457 immediately after *message_body*. 458 459 .. note:: Due to the chunked encoding specification, empty chunks 460 yielded by an iterator body will be ignored by the chunk-encoder. 461 This is to avoid premature termination of the read of the request by 462 the target server due to malformed encoding. 463 464 .. versionchanged:: 3.6 465 Added chunked encoding support and the *encode_chunked* parameter. 466 467 468.. method:: HTTPConnection.send(data) 469 470 Send data to the server. This should be used directly only after the 471 :meth:`endheaders` method has been called and before :meth:`getresponse` is 472 called. 473 474 .. audit-event:: http.client.send self,data http.client.HTTPConnection.send 475 476 477.. _httpresponse-objects: 478 479HTTPResponse Objects 480-------------------- 481 482An :class:`HTTPResponse` instance wraps the HTTP response from the 483server. It provides access to the request headers and the entity 484body. The response is an iterable object and can be used in a with 485statement. 486 487.. versionchanged:: 3.5 488 The :class:`io.BufferedIOBase` interface is now implemented and 489 all of its reader operations are supported. 490 491 492.. method:: HTTPResponse.read([amt]) 493 494 Reads and returns the response body, or up to the next *amt* bytes. 495 496.. method:: HTTPResponse.readinto(b) 497 498 Reads up to the next len(b) bytes of the response body into the buffer *b*. 499 Returns the number of bytes read. 500 501 .. versionadded:: 3.3 502 503.. method:: HTTPResponse.getheader(name, default=None) 504 505 Return the value of the header *name*, or *default* if there is no header 506 matching *name*. If there is more than one header with the name *name*, 507 return all of the values joined by ', '. If *default* is any iterable other 508 than a single string, its elements are similarly returned joined by commas. 509 510.. method:: HTTPResponse.getheaders() 511 512 Return a list of (header, value) tuples. 513 514.. method:: HTTPResponse.fileno() 515 516 Return the ``fileno`` of the underlying socket. 517 518.. attribute:: HTTPResponse.msg 519 520 A :class:`http.client.HTTPMessage` instance containing the response 521 headers. :class:`http.client.HTTPMessage` is a subclass of 522 :class:`email.message.Message`. 523 524.. attribute:: HTTPResponse.version 525 526 HTTP protocol version used by server. 10 for HTTP/1.0, 11 for HTTP/1.1. 527 528.. attribute:: HTTPResponse.url 529 530 URL of the resource retrieved, commonly used to determine if a redirect was followed. 531 532.. attribute:: HTTPResponse.headers 533 534 Headers of the response in the form of an :class:`email.message.EmailMessage` instance. 535 536.. attribute:: HTTPResponse.status 537 538 Status code returned by server. 539 540.. attribute:: HTTPResponse.reason 541 542 Reason phrase returned by server. 543 544.. attribute:: HTTPResponse.debuglevel 545 546 A debugging hook. If :attr:`debuglevel` is greater than zero, messages 547 will be printed to stdout as the response is read and parsed. 548 549.. attribute:: HTTPResponse.closed 550 551 Is ``True`` if the stream is closed. 552 553.. method:: HTTPResponse.geturl() 554 555 .. deprecated:: 3.9 556 Deprecated in favor of :attr:`~HTTPResponse.url`. 557 558.. method:: HTTPResponse.info() 559 560 .. deprecated:: 3.9 561 Deprecated in favor of :attr:`~HTTPResponse.headers`. 562 563.. method:: HTTPResponse.getcode() 564 565 .. deprecated:: 3.9 566 Deprecated in favor of :attr:`~HTTPResponse.status`. 567 568Examples 569-------- 570 571Here is an example session that uses the ``GET`` method:: 572 573 >>> import http.client 574 >>> conn = http.client.HTTPSConnection("www.python.org") 575 >>> conn.request("GET", "/") 576 >>> r1 = conn.getresponse() 577 >>> print(r1.status, r1.reason) 578 200 OK 579 >>> data1 = r1.read() # This will return entire content. 580 >>> # The following example demonstrates reading data in chunks. 581 >>> conn.request("GET", "/") 582 >>> r1 = conn.getresponse() 583 >>> while chunk := r1.read(200): 584 ... print(repr(chunk)) 585 b'<!doctype html>\n<!--[if"... 586 ... 587 >>> # Example of an invalid request 588 >>> conn = http.client.HTTPSConnection("docs.python.org") 589 >>> conn.request("GET", "/parrot.spam") 590 >>> r2 = conn.getresponse() 591 >>> print(r2.status, r2.reason) 592 404 Not Found 593 >>> data2 = r2.read() 594 >>> conn.close() 595 596Here is an example session that uses the ``HEAD`` method. Note that the 597``HEAD`` method never returns any data. :: 598 599 >>> import http.client 600 >>> conn = http.client.HTTPSConnection("www.python.org") 601 >>> conn.request("HEAD", "/") 602 >>> res = conn.getresponse() 603 >>> print(res.status, res.reason) 604 200 OK 605 >>> data = res.read() 606 >>> print(len(data)) 607 0 608 >>> data == b'' 609 True 610 611Here is an example session that uses the ``POST`` method:: 612 613 >>> import http.client, urllib.parse 614 >>> params = urllib.parse.urlencode({'@number': 12524, '@type': 'issue', '@action': 'show'}) 615 >>> headers = {"Content-type": "application/x-www-form-urlencoded", 616 ... "Accept": "text/plain"} 617 >>> conn = http.client.HTTPConnection("bugs.python.org") 618 >>> conn.request("POST", "", params, headers) 619 >>> response = conn.getresponse() 620 >>> print(response.status, response.reason) 621 302 Found 622 >>> data = response.read() 623 >>> data 624 b'Redirecting to <a href="https://bugs.python.org/issue12524">https://bugs.python.org/issue12524</a>' 625 >>> conn.close() 626 627Client side HTTP ``PUT`` requests are very similar to ``POST`` requests. The 628difference lies only on the server side where HTTP servers will allow resources to 629be created via ``PUT`` requests. It should be noted that custom HTTP methods 630are also handled in :class:`urllib.request.Request` by setting the appropriate 631method attribute. Here is an example session that uses the ``PUT`` method:: 632 633 >>> # This creates an HTTP request 634 >>> # with the content of BODY as the enclosed representation 635 >>> # for the resource http://localhost:8080/file 636 ... 637 >>> import http.client 638 >>> BODY = "***filecontents***" 639 >>> conn = http.client.HTTPConnection("localhost", 8080) 640 >>> conn.request("PUT", "/file", BODY) 641 >>> response = conn.getresponse() 642 >>> print(response.status, response.reason) 643 200, OK 644 645.. _httpmessage-objects: 646 647HTTPMessage Objects 648------------------- 649 650.. class:: HTTPMessage(email.message.Message) 651 652An :class:`http.client.HTTPMessage` instance holds the headers from an HTTP 653response. It is implemented using the :class:`email.message.Message` class. 654 655.. XXX Define the methods that clients can depend upon between versions. 656