1.. _devmode: 2 3Python Development Mode 4======================= 5 6.. versionadded:: 3.7 7 8The Python Development Mode introduces additional runtime checks that are too 9expensive to be enabled by default. It should not be more verbose than the 10default if the code is correct; new warnings are only emitted when an issue is 11detected. 12 13It can be enabled using the :option:`-X dev <-X>` command line option or by 14setting the :envvar:`PYTHONDEVMODE` environment variable to ``1``. 15 16Effects of the Python Development Mode 17====================================== 18 19Enabling the Python Development Mode is similar to the following command, but 20with additional effects described below:: 21 22 PYTHONMALLOC=debug PYTHONASYNCIODEBUG=1 python3 -W default -X faulthandler 23 24Effects of the Python Development Mode: 25 26* Add ``default`` :ref:`warning filter <describing-warning-filters>`. The 27 following warnings are shown: 28 29 * :exc:`DeprecationWarning` 30 * :exc:`ImportWarning` 31 * :exc:`PendingDeprecationWarning` 32 * :exc:`ResourceWarning` 33 34 Normally, the above warnings are filtered by the default :ref:`warning 35 filters <describing-warning-filters>`. 36 37 It behaves as if the :option:`-W default <-W>` command line option is used. 38 39 Use the :option:`-W error <-W>` command line option or set the 40 :envvar:`PYTHONWARNINGS` environment variable to ``error`` to treat warnings 41 as errors. 42 43* Install debug hooks on memory allocators to check for: 44 45 * Buffer underflow 46 * Buffer overflow 47 * Memory allocator API violation 48 * Unsafe usage of the GIL 49 50 See the :c:func:`PyMem_SetupDebugHooks` C function. 51 52 It behaves as if the :envvar:`PYTHONMALLOC` environment variable is set to 53 ``debug``. 54 55 To enable the Python Development Mode without installing debug hooks on 56 memory allocators, set the :envvar:`PYTHONMALLOC` environment variable to 57 ``default``. 58 59* Call :func:`faulthandler.enable` at Python startup to install handlers for 60 the :const:`SIGSEGV`, :const:`SIGFPE`, :const:`SIGABRT`, :const:`SIGBUS` and 61 :const:`SIGILL` signals to dump the Python traceback on a crash. 62 63 It behaves as if the :option:`-X faulthandler <-X>` command line option is 64 used or if the :envvar:`PYTHONFAULTHANDLER` environment variable is set to 65 ``1``. 66 67* Enable :ref:`asyncio debug mode <asyncio-debug-mode>`. For example, 68 :mod:`asyncio` checks for coroutines that were not awaited and logs them. 69 70 It behaves as if the :envvar:`PYTHONASYNCIODEBUG` environment variable is set 71 to ``1``. 72 73* Check the *encoding* and *errors* arguments for string encoding and decoding 74 operations. Examples: :func:`open`, :meth:`str.encode` and 75 :meth:`bytes.decode`. 76 77 By default, for best performance, the *errors* argument is only checked at 78 the first encoding/decoding error and the *encoding* argument is sometimes 79 ignored for empty strings. 80 81* The :class:`io.IOBase` destructor logs ``close()`` exceptions. 82* Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to 83 ``True``. 84 85The Python Development Mode does not enable the :mod:`tracemalloc` module by 86default, because the overhead cost (to performance and memory) would be too 87large. Enabling the :mod:`tracemalloc` module provides additional information 88on the origin of some errors. For example, :exc:`ResourceWarning` logs the 89traceback where the resource was allocated, and a buffer overflow error logs 90the traceback where the memory block was allocated. 91 92The Python Development Mode does not prevent the :option:`-O` command line 93option from removing :keyword:`assert` statements nor from setting 94:const:`__debug__` to ``False``. 95 96.. versionchanged:: 3.8 97 The :class:`io.IOBase` destructor now logs ``close()`` exceptions. 98 99.. versionchanged:: 3.9 100 The *encoding* and *errors* arguments are now checked for string encoding 101 and decoding operations. 102 103 104ResourceWarning Example 105======================= 106 107Example of a script counting the number of lines of the text file specified in 108the command line:: 109 110 import sys 111 112 def main(): 113 fp = open(sys.argv[1]) 114 nlines = len(fp.readlines()) 115 print(nlines) 116 # The file is closed implicitly 117 118 if __name__ == "__main__": 119 main() 120 121The script does not close the file explicitly. By default, Python does not emit 122any warning. Example using README.txt, which has 269 lines: 123 124.. code-block:: shell-session 125 126 $ python3 script.py README.txt 127 269 128 129Enabling the Python Development Mode displays a :exc:`ResourceWarning` warning: 130 131.. code-block:: shell-session 132 133 $ python3 -X dev script.py README.txt 134 269 135 script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'> 136 main() 137 ResourceWarning: Enable tracemalloc to get the object allocation traceback 138 139In addition, enabling :mod:`tracemalloc` shows the line where the file was 140opened: 141 142.. code-block:: shell-session 143 144 $ python3 -X dev -X tracemalloc=5 script.py README.rst 145 269 146 script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'> 147 main() 148 Object allocated at (most recent call last): 149 File "script.py", lineno 10 150 main() 151 File "script.py", lineno 4 152 fp = open(sys.argv[1]) 153 154The fix is to close explicitly the file. Example using a context manager:: 155 156 def main(): 157 # Close the file explicitly when exiting the with block 158 with open(sys.argv[1]) as fp: 159 nlines = len(fp.readlines()) 160 print(nlines) 161 162Not closing a resource explicitly can leave a resource open for way longer than 163expected; it can cause severe issues upon exiting Python. It is bad in 164CPython, but it is even worse in PyPy. Closing resources explicitly makes an 165application more deterministic and more reliable. 166 167 168Bad file descriptor error example 169================================= 170 171Script displaying the first line of itself:: 172 173 import os 174 175 def main(): 176 fp = open(__file__) 177 firstline = fp.readline() 178 print(firstline.rstrip()) 179 os.close(fp.fileno()) 180 # The file is closed implicitly 181 182 main() 183 184By default, Python does not emit any warning: 185 186.. code-block:: shell-session 187 188 $ python3 script.py 189 import os 190 191The Python Development Mode shows a :exc:`ResourceWarning` and logs a "Bad file 192descriptor" error when finalizing the file object: 193 194.. code-block:: shell-session 195 196 $ python3 script.py 197 import os 198 script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> 199 main() 200 ResourceWarning: Enable tracemalloc to get the object allocation traceback 201 Exception ignored in: <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> 202 Traceback (most recent call last): 203 File "script.py", line 10, in <module> 204 main() 205 OSError: [Errno 9] Bad file descriptor 206 207``os.close(fp.fileno())`` closes the file descriptor. When the file object 208finalizer tries to close the file descriptor again, it fails with the ``Bad 209file descriptor`` error. A file descriptor must be closed only once. In the 210worst case scenario, closing it twice can lead to a crash (see :issue:`18748` 211for an example). 212 213The fix is to remove the ``os.close(fp.fileno())`` line, or open the file with 214``closefd=False``. 215