• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1:mod:`http.client` --- HTTP protocol client
2===========================================
3
4.. module:: http.client
5   :synopsis: HTTP and HTTPS protocol client (requires sockets).
6
7**Source code:** :source:`Lib/http/client.py`
8
9.. index::
10   pair: HTTP; protocol
11   single: HTTP; http.client (standard module)
12
13.. index:: module: urllib.request
14
15--------------
16
17This module defines classes which implement the client side of the HTTP and
18HTTPS protocols.  It is normally not used directly --- the module
19:mod:`urllib.request` uses it to handle URLs that use HTTP and HTTPS.
20
21.. seealso::
22
23    The `Requests package <https://requests.readthedocs.io/en/master/>`_
24    is recommended for a higher-level HTTP client interface.
25
26.. note::
27
28   HTTPS support is only available if Python was compiled with SSL support
29   (through the :mod:`ssl` module).
30
31The module provides the following classes:
32
33
34.. class:: HTTPConnection(host, port=None[, timeout], source_address=None, \
35                          blocksize=8192)
36
37   An :class:`HTTPConnection` instance represents one transaction with an HTTP
38   server.  It should be instantiated passing it a host and optional port
39   number.  If no port number is passed, the port is extracted from the host
40   string if it has the form ``host:port``, else the default HTTP port (80) is
41   used.  If the optional *timeout* parameter is given, blocking
42   operations (like connection attempts) will timeout after that many seconds
43   (if it is not given, the global default timeout setting is used).
44   The optional *source_address* parameter may be a tuple of a (host, port)
45   to use as the source address the HTTP connection is made from.
46   The optional *blocksize* parameter sets the buffer size in bytes for
47   sending a file-like message body.
48
49   For example, the following calls all create instances that connect to the server
50   at the same host and port::
51
52      >>> h1 = http.client.HTTPConnection('www.python.org')
53      >>> h2 = http.client.HTTPConnection('www.python.org:80')
54      >>> h3 = http.client.HTTPConnection('www.python.org', 80)
55      >>> h4 = http.client.HTTPConnection('www.python.org', 80, timeout=10)
56
57   .. versionchanged:: 3.2
58      *source_address* was added.
59
60   .. versionchanged:: 3.4
61      The  *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are
62      not longer supported.
63
64   .. versionchanged:: 3.7
65      *blocksize* parameter was added.
66
67
68.. class:: HTTPSConnection(host, port=None, key_file=None, \
69                           cert_file=None[, timeout], \
70                           source_address=None, *, context=None, \
71                           check_hostname=None, blocksize=8192)
72
73   A subclass of :class:`HTTPConnection` that uses SSL for communication with
74   secure servers.  Default port is ``443``.  If *context* is specified, it
75   must be a :class:`ssl.SSLContext` instance describing the various SSL
76   options.
77
78   Please read :ref:`ssl-security` for more information on best practices.
79
80   .. versionchanged:: 3.2
81      *source_address*, *context* and *check_hostname* were added.
82
83   .. versionchanged:: 3.2
84      This class now supports HTTPS virtual hosts if possible (that is,
85      if :data:`ssl.HAS_SNI` is true).
86
87   .. versionchanged:: 3.4
88      The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are
89      no longer supported.
90
91   .. versionchanged:: 3.4.3
92      This class now performs all the necessary certificate and hostname checks
93      by default. To revert to the previous, unverified, behavior
94      :func:`ssl._create_unverified_context` can be passed to the *context*
95      parameter.
96
97   .. versionchanged:: 3.8
98      This class now enables TLS 1.3
99      :attr:`ssl.SSLContext.post_handshake_auth` for the default *context* or
100      when *cert_file* is passed with a custom *context*.
101
102   .. versionchanged:: 3.10
103      This class now sends an ALPN extension with protocol indicator
104      ``http/1.1`` when no *context* is given. Custom *context* should set
105      ALPN protocols with :meth:`~ssl.SSLContext.set_alpn_protocol`.
106
107   .. deprecated:: 3.6
108
109       *key_file* and *cert_file* are deprecated in favor of *context*.
110       Please use :meth:`ssl.SSLContext.load_cert_chain` instead, or let
111       :func:`ssl.create_default_context` select the system's trusted CA
112       certificates for you.
113
114       The *check_hostname* parameter is also deprecated; the
115       :attr:`ssl.SSLContext.check_hostname` attribute of *context* should
116       be used instead.
117
118
119.. class:: HTTPResponse(sock, debuglevel=0, method=None, url=None)
120
121   Class whose instances are returned upon successful connection.  Not
122   instantiated directly by user.
123
124   .. versionchanged:: 3.4
125      The *strict* parameter was removed. HTTP 0.9 style "Simple Responses" are
126      no longer supported.
127
128This module provides the following function:
129
130.. function:: parse_headers(fp)
131
132   Parse the headers from a file pointer *fp* representing a HTTP
133   request/response. The file has to be a :class:`BufferedIOBase` reader
134   (i.e. not text) and must provide a valid :rfc:`2822` style header.
135
136   This function returns an instance of :class:`http.client.HTTPMessage`
137   that holds the header fields, but no payload
138   (the same as :attr:`HTTPResponse.msg`
139   and :attr:`http.server.BaseHTTPRequestHandler.headers`).
140   After returning, the file pointer *fp* is ready to read the HTTP body.
141
142   .. note::
143      :meth:`parse_headers` does not parse the start-line of a HTTP message;
144      it only parses the ``Name: value`` lines. The file has to be ready to
145      read these field lines, so the first line should already be consumed
146      before calling the function.
147
148The following exceptions are raised as appropriate:
149
150
151.. exception:: HTTPException
152
153   The base class of the other exceptions in this module.  It is a subclass of
154   :exc:`Exception`.
155
156
157.. exception:: NotConnected
158
159   A subclass of :exc:`HTTPException`.
160
161
162.. exception:: InvalidURL
163
164   A subclass of :exc:`HTTPException`, raised if a port is given and is either
165   non-numeric or empty.
166
167
168.. exception:: UnknownProtocol
169
170   A subclass of :exc:`HTTPException`.
171
172
173.. exception:: UnknownTransferEncoding
174
175   A subclass of :exc:`HTTPException`.
176
177
178.. exception:: UnimplementedFileMode
179
180   A subclass of :exc:`HTTPException`.
181
182
183.. exception:: IncompleteRead
184
185   A subclass of :exc:`HTTPException`.
186
187
188.. exception:: ImproperConnectionState
189
190   A subclass of :exc:`HTTPException`.
191
192
193.. exception:: CannotSendRequest
194
195   A subclass of :exc:`ImproperConnectionState`.
196
197
198.. exception:: CannotSendHeader
199
200   A subclass of :exc:`ImproperConnectionState`.
201
202
203.. exception:: ResponseNotReady
204
205   A subclass of :exc:`ImproperConnectionState`.
206
207
208.. exception:: BadStatusLine
209
210   A subclass of :exc:`HTTPException`.  Raised if a server responds with a HTTP
211   status code that we don't understand.
212
213
214.. exception:: LineTooLong
215
216   A subclass of :exc:`HTTPException`.  Raised if an excessively long line
217   is received in the HTTP protocol from the server.
218
219
220.. exception:: RemoteDisconnected
221
222   A subclass of :exc:`ConnectionResetError` and :exc:`BadStatusLine`.  Raised
223   by :meth:`HTTPConnection.getresponse` when the attempt to read the response
224   results in no data read from the connection, indicating that the remote end
225   has closed the connection.
226
227   .. versionadded:: 3.5
228      Previously, :exc:`BadStatusLine`\ ``('')`` was raised.
229
230
231The constants defined in this module are:
232
233.. data:: HTTP_PORT
234
235   The default port for the HTTP protocol (always ``80``).
236
237.. data:: HTTPS_PORT
238
239   The default port for the HTTPS protocol (always ``443``).
240
241.. data:: responses
242
243   This dictionary maps the HTTP 1.1 status codes to the W3C names.
244
245   Example: ``http.client.responses[http.client.NOT_FOUND]`` is ``'Not Found'``.
246
247See :ref:`http-status-codes` for a list of HTTP status codes that are
248available in this module as constants.
249
250
251.. _httpconnection-objects:
252
253HTTPConnection Objects
254----------------------
255
256:class:`HTTPConnection` instances have the following methods:
257
258
259.. method:: HTTPConnection.request(method, url, body=None, headers={}, *, \
260            encode_chunked=False)
261
262   This will send a request to the server using the HTTP request
263   method *method* and the selector *url*.
264
265   If *body* is specified, the specified data is sent after the headers are
266   finished.  It may be a :class:`str`, a :term:`bytes-like object`, an
267   open :term:`file object`, or an iterable of :class:`bytes`.  If *body*
268   is a string, it is encoded as ISO-8859-1, the default for HTTP.  If it
269   is a bytes-like object, the bytes are sent as is.  If it is a :term:`file
270   object`, the contents of the file is sent; this file object should
271   support at least the ``read()`` method.  If the file object is an
272   instance of :class:`io.TextIOBase`, the data returned by the ``read()``
273   method will be encoded as ISO-8859-1, otherwise the data returned by
274   ``read()`` is sent as is.  If *body* is an iterable, the elements of the
275   iterable are sent as is until the iterable is exhausted.
276
277   The *headers* argument should be a mapping of extra HTTP headers to send
278   with the request.
279
280   If *headers* contains neither Content-Length nor Transfer-Encoding,
281   but there is a request body, one of those
282   header fields will be added automatically.  If
283   *body* is ``None``, the Content-Length header is set to ``0`` for
284   methods that expect a body (``PUT``, ``POST``, and ``PATCH``).  If
285   *body* is a string or a bytes-like object that is not also a
286   :term:`file <file object>`, the Content-Length header is
287   set to its length.  Any other type of *body* (files
288   and iterables in general) will be chunk-encoded, and the
289   Transfer-Encoding header will automatically be set instead of
290   Content-Length.
291
292   The *encode_chunked* argument is only relevant if Transfer-Encoding is
293   specified in *headers*.  If *encode_chunked* is ``False``, the
294   HTTPConnection object assumes that all encoding is handled by the
295   calling code.  If it is ``True``, the body will be chunk-encoded.
296
297   .. note::
298      Chunked transfer encoding has been added to the HTTP protocol
299      version 1.1.  Unless the HTTP server is known to handle HTTP 1.1,
300      the caller must either specify the Content-Length, or must pass a
301      :class:`str` or bytes-like object that is not also a file as the
302      body representation.
303
304   .. versionadded:: 3.2
305      *body* can now be an iterable.
306
307   .. versionchanged:: 3.6
308      If neither Content-Length nor Transfer-Encoding are set in
309      *headers*, file and iterable *body* objects are now chunk-encoded.
310      The *encode_chunked* argument was added.
311      No attempt is made to determine the Content-Length for file
312      objects.
313
314.. method:: HTTPConnection.getresponse()
315
316   Should be called after a request is sent to get the response from the server.
317   Returns an :class:`HTTPResponse` instance.
318
319   .. note::
320
321      Note that you must have read the whole response before you can send a new
322      request to the server.
323
324   .. versionchanged:: 3.5
325      If a :exc:`ConnectionError` or subclass is raised, the
326      :class:`HTTPConnection` object will be ready to reconnect when
327      a new request is sent.
328
329
330.. method:: HTTPConnection.set_debuglevel(level)
331
332   Set the debugging level.  The default debug level is ``0``, meaning no
333   debugging output is printed.  Any value greater than ``0`` will cause all
334   currently defined debug output to be printed to stdout.  The ``debuglevel``
335   is passed to any new :class:`HTTPResponse` objects that are created.
336
337   .. versionadded:: 3.1
338
339
340.. method:: HTTPConnection.set_tunnel(host, port=None, headers=None)
341
342   Set the host and the port for HTTP Connect Tunnelling. This allows running
343   the connection through a proxy server.
344
345   The host and port arguments specify the endpoint of the tunneled connection
346   (i.e. the address included in the CONNECT request, *not* the address of the
347   proxy server).
348
349   The headers argument should be a mapping of extra HTTP headers to send with
350   the CONNECT request.
351
352   For example, to tunnel through a HTTPS proxy server running locally on port
353   8080, we would pass the address of the proxy to the :class:`HTTPSConnection`
354   constructor, and the address of the host that we eventually want to reach to
355   the :meth:`~HTTPConnection.set_tunnel` method::
356
357      >>> import http.client
358      >>> conn = http.client.HTTPSConnection("localhost", 8080)
359      >>> conn.set_tunnel("www.python.org")
360      >>> conn.request("HEAD","/index.html")
361
362   .. versionadded:: 3.2
363
364
365.. method:: HTTPConnection.connect()
366
367   Connect to the server specified when the object was created.  By default,
368   this is called automatically when making a request if the client does not
369   already have a connection.
370
371   .. audit-event:: http.client.connect self,host,port http.client.HTTPConnection.connect
372
373
374.. method:: HTTPConnection.close()
375
376   Close the connection to the server.
377
378
379.. attribute:: HTTPConnection.blocksize
380
381   Buffer size in bytes for sending a file-like message body.
382
383   .. versionadded:: 3.7
384
385
386As an alternative to using the :meth:`request` method described above, you can
387also send your request step by step, by using the four functions below.
388
389
390.. method:: HTTPConnection.putrequest(method, url, skip_host=False, \
391                                      skip_accept_encoding=False)
392
393   This should be the first call after the connection to the server has been
394   made. It sends a line to the server consisting of the *method* string,
395   the *url* string, and the HTTP version (``HTTP/1.1``).  To disable automatic
396   sending of ``Host:`` or ``Accept-Encoding:`` headers (for example to accept
397   additional content encodings), specify *skip_host* or *skip_accept_encoding*
398   with non-False values.
399
400
401.. method:: HTTPConnection.putheader(header, argument[, ...])
402
403   Send an :rfc:`822`\ -style header to the server.  It sends a line to the server
404   consisting of the header, a colon and a space, and the first argument.  If more
405   arguments are given, continuation lines are sent, each consisting of a tab and
406   an argument.
407
408
409.. method:: HTTPConnection.endheaders(message_body=None, *, encode_chunked=False)
410
411   Send a blank line to the server, signalling the end of the headers. The
412   optional *message_body* argument can be used to pass a message body
413   associated with the request.
414
415   If *encode_chunked* is ``True``, the result of each iteration of
416   *message_body* will be chunk-encoded as specified in :rfc:`7230`,
417   Section 3.3.1.  How the data is encoded is dependent on the type of
418   *message_body*.  If *message_body* implements the :ref:`buffer interface
419   <bufferobjects>` the encoding will result in a single chunk.
420   If *message_body* is a :class:`collections.abc.Iterable`, each iteration
421   of *message_body* will result in a chunk.  If *message_body* is a
422   :term:`file object`, each call to ``.read()`` will result in a chunk.
423   The method automatically signals the end of the chunk-encoded data
424   immediately after *message_body*.
425
426   .. note:: Due to the chunked encoding specification, empty chunks
427      yielded by an iterator body will be ignored by the chunk-encoder.
428      This is to avoid premature termination of the read of the request by
429      the target server due to malformed encoding.
430
431   .. versionadded:: 3.6
432      Chunked encoding support.  The *encode_chunked* parameter was
433      added.
434
435
436.. method:: HTTPConnection.send(data)
437
438   Send data to the server.  This should be used directly only after the
439   :meth:`endheaders` method has been called and before :meth:`getresponse` is
440   called.
441
442   .. audit-event:: http.client.send self,data http.client.HTTPConnection.send
443
444
445.. _httpresponse-objects:
446
447HTTPResponse Objects
448--------------------
449
450An :class:`HTTPResponse` instance wraps the HTTP response from the
451server.  It provides access to the request headers and the entity
452body.  The response is an iterable object and can be used in a with
453statement.
454
455.. versionchanged:: 3.5
456   The :class:`io.BufferedIOBase` interface is now implemented and
457   all of its reader operations are supported.
458
459
460.. method:: HTTPResponse.read([amt])
461
462   Reads and returns the response body, or up to the next *amt* bytes.
463
464.. method:: HTTPResponse.readinto(b)
465
466   Reads up to the next len(b) bytes of the response body into the buffer *b*.
467   Returns the number of bytes read.
468
469   .. versionadded:: 3.3
470
471.. method:: HTTPResponse.getheader(name, default=None)
472
473   Return the value of the header *name*, or *default* if there is no header
474   matching *name*.  If there is more than one  header with the name *name*,
475   return all of the values joined by ', '.  If 'default' is any iterable other
476   than a single string, its elements are similarly returned joined by commas.
477
478.. method:: HTTPResponse.getheaders()
479
480   Return a list of (header, value) tuples.
481
482.. method:: HTTPResponse.fileno()
483
484   Return the ``fileno`` of the underlying socket.
485
486.. attribute:: HTTPResponse.msg
487
488   A :class:`http.client.HTTPMessage` instance containing the response
489   headers.  :class:`http.client.HTTPMessage` is a subclass of
490   :class:`email.message.Message`.
491
492.. attribute:: HTTPResponse.version
493
494   HTTP protocol version used by server.  10 for HTTP/1.0, 11 for HTTP/1.1.
495
496.. attribute:: HTTPResponse.url
497
498   URL of the resource retrieved, commonly used to determine if a redirect was followed.
499
500.. attribute:: HTTPResponse.headers
501
502   Headers of the response in the form of an :class:`email.message.EmailMessage` instance.
503
504.. attribute:: HTTPResponse.status
505
506   Status code returned by server.
507
508.. attribute:: HTTPResponse.reason
509
510   Reason phrase returned by server.
511
512.. attribute:: HTTPResponse.debuglevel
513
514   A debugging hook.  If :attr:`debuglevel` is greater than zero, messages
515   will be printed to stdout as the response is read and parsed.
516
517.. attribute:: HTTPResponse.closed
518
519   Is ``True`` if the stream is closed.
520
521.. method:: HTTPResponse.geturl()
522
523   .. deprecated:: 3.9
524      Deprecated in favor of :attr:`~HTTPResponse.url`.
525
526.. method:: HTTPResponse.info()
527
528   .. deprecated:: 3.9
529      Deprecated in favor of :attr:`~HTTPResponse.headers`.
530
531.. method:: HTTPResponse.getstatus()
532
533   .. deprecated:: 3.9
534      Deprecated in favor of :attr:`~HTTPResponse.status`.
535
536Examples
537--------
538
539Here is an example session that uses the ``GET`` method::
540
541   >>> import http.client
542   >>> conn = http.client.HTTPSConnection("www.python.org")
543   >>> conn.request("GET", "/")
544   >>> r1 = conn.getresponse()
545   >>> print(r1.status, r1.reason)
546   200 OK
547   >>> data1 = r1.read()  # This will return entire content.
548   >>> # The following example demonstrates reading data in chunks.
549   >>> conn.request("GET", "/")
550   >>> r1 = conn.getresponse()
551   >>> while chunk := r1.read(200):
552   ...     print(repr(chunk))
553   b'<!doctype html>\n<!--[if"...
554   ...
555   >>> # Example of an invalid request
556   >>> conn = http.client.HTTPSConnection("docs.python.org")
557   >>> conn.request("GET", "/parrot.spam")
558   >>> r2 = conn.getresponse()
559   >>> print(r2.status, r2.reason)
560   404 Not Found
561   >>> data2 = r2.read()
562   >>> conn.close()
563
564Here is an example session that uses the ``HEAD`` method.  Note that the
565``HEAD`` method never returns any data. ::
566
567   >>> import http.client
568   >>> conn = http.client.HTTPSConnection("www.python.org")
569   >>> conn.request("HEAD", "/")
570   >>> res = conn.getresponse()
571   >>> print(res.status, res.reason)
572   200 OK
573   >>> data = res.read()
574   >>> print(len(data))
575   0
576   >>> data == b''
577   True
578
579Here is an example session that shows how to ``POST`` requests::
580
581   >>> import http.client, urllib.parse
582   >>> params = urllib.parse.urlencode({'@number': 12524, '@type': 'issue', '@action': 'show'})
583   >>> headers = {"Content-type": "application/x-www-form-urlencoded",
584   ...            "Accept": "text/plain"}
585   >>> conn = http.client.HTTPConnection("bugs.python.org")
586   >>> conn.request("POST", "", params, headers)
587   >>> response = conn.getresponse()
588   >>> print(response.status, response.reason)
589   302 Found
590   >>> data = response.read()
591   >>> data
592   b'Redirecting to <a href="http://bugs.python.org/issue12524">http://bugs.python.org/issue12524</a>'
593   >>> conn.close()
594
595Client side ``HTTP PUT`` requests are very similar to ``POST`` requests. The
596difference lies only the server side where HTTP server will allow resources to
597be created via ``PUT`` request. It should be noted that custom HTTP methods
598are also handled in :class:`urllib.request.Request` by setting the appropriate
599method attribute. Here is an example session that shows how to send a ``PUT``
600request using http.client::
601
602    >>> # This creates an HTTP message
603    >>> # with the content of BODY as the enclosed representation
604    >>> # for the resource http://localhost:8080/file
605    ...
606    >>> import http.client
607    >>> BODY = "***filecontents***"
608    >>> conn = http.client.HTTPConnection("localhost", 8080)
609    >>> conn.request("PUT", "/file", BODY)
610    >>> response = conn.getresponse()
611    >>> print(response.status, response.reason)
612    200, OK
613
614.. _httpmessage-objects:
615
616HTTPMessage Objects
617-------------------
618
619An :class:`http.client.HTTPMessage` instance holds the headers from an HTTP
620response.  It is implemented using the :class:`email.message.Message` class.
621
622.. XXX Define the methods that clients can depend upon between versions.
623