1.. _devmode: 2 3Python Development Mode 4======================= 5 6.. versionadded:: 3.7 7 8The Python Development Mode introduces additional runtime checks that are too 9expensive to be enabled by default. It should not be more verbose than the 10default if the code is correct; new warnings are only emitted when an issue is 11detected. 12 13It can be enabled using the :option:`-X dev <-X>` command line option or by 14setting the :envvar:`PYTHONDEVMODE` environment variable to ``1``. 15 16See also :ref:`Python debug build <debug-build>`. 17 18Effects of the Python Development Mode 19-------------------------------------- 20 21Enabling the Python Development Mode is similar to the following command, but 22with additional effects described below:: 23 24 PYTHONMALLOC=debug PYTHONASYNCIODEBUG=1 python -W default -X faulthandler 25 26Effects of the Python Development Mode: 27 28* Add ``default`` :ref:`warning filter <describing-warning-filters>`. The 29 following warnings are shown: 30 31 * :exc:`DeprecationWarning` 32 * :exc:`ImportWarning` 33 * :exc:`PendingDeprecationWarning` 34 * :exc:`ResourceWarning` 35 36 Normally, the above warnings are filtered by the default :ref:`warning 37 filters <describing-warning-filters>`. 38 39 It behaves as if the :option:`-W default <-W>` command line option is used. 40 41 Use the :option:`-W error <-W>` command line option or set the 42 :envvar:`PYTHONWARNINGS` environment variable to ``error`` to treat warnings 43 as errors. 44 45* Install debug hooks on memory allocators to check for: 46 47 * Buffer underflow 48 * Buffer overflow 49 * Memory allocator API violation 50 * Unsafe usage of the GIL 51 52 See the :c:func:`PyMem_SetupDebugHooks` C function. 53 54 It behaves as if the :envvar:`PYTHONMALLOC` environment variable is set to 55 ``debug``. 56 57 To enable the Python Development Mode without installing debug hooks on 58 memory allocators, set the :envvar:`PYTHONMALLOC` environment variable to 59 ``default``. 60 61* Call :func:`faulthandler.enable` at Python startup to install handlers for 62 the :const:`~signal.SIGSEGV`, :const:`~signal.SIGFPE`, 63 :const:`~signal.SIGABRT`, :const:`~signal.SIGBUS` and 64 :const:`~signal.SIGILL` signals to dump the Python traceback on a crash. 65 66 It behaves as if the :option:`-X faulthandler <-X>` command line option is 67 used or if the :envvar:`PYTHONFAULTHANDLER` environment variable is set to 68 ``1``. 69 70* Enable :ref:`asyncio debug mode <asyncio-debug-mode>`. For example, 71 :mod:`asyncio` checks for coroutines that were not awaited and logs them. 72 73 It behaves as if the :envvar:`PYTHONASYNCIODEBUG` environment variable is set 74 to ``1``. 75 76* Check the *encoding* and *errors* arguments for string encoding and decoding 77 operations. Examples: :func:`open`, :meth:`str.encode` and 78 :meth:`bytes.decode`. 79 80 By default, for best performance, the *errors* argument is only checked at 81 the first encoding/decoding error and the *encoding* argument is sometimes 82 ignored for empty strings. 83 84* The :class:`io.IOBase` destructor logs ``close()`` exceptions. 85* Set the :attr:`~sys.flags.dev_mode` attribute of :data:`sys.flags` to 86 ``True``. 87 88The Python Development Mode does not enable the :mod:`tracemalloc` module by 89default, because the overhead cost (to performance and memory) would be too 90large. Enabling the :mod:`tracemalloc` module provides additional information 91on the origin of some errors. For example, :exc:`ResourceWarning` logs the 92traceback where the resource was allocated, and a buffer overflow error logs 93the traceback where the memory block was allocated. 94 95The Python Development Mode does not prevent the :option:`-O` command line 96option from removing :keyword:`assert` statements nor from setting 97:const:`__debug__` to ``False``. 98 99The Python Development Mode can only be enabled at the Python startup. Its 100value can be read from :data:`sys.flags.dev_mode <sys.flags>`. 101 102.. versionchanged:: 3.8 103 The :class:`io.IOBase` destructor now logs ``close()`` exceptions. 104 105.. versionchanged:: 3.9 106 The *encoding* and *errors* arguments are now checked for string encoding 107 and decoding operations. 108 109 110ResourceWarning Example 111----------------------- 112 113Example of a script counting the number of lines of the text file specified in 114the command line:: 115 116 import sys 117 118 def main(): 119 fp = open(sys.argv[1]) 120 nlines = len(fp.readlines()) 121 print(nlines) 122 # The file is closed implicitly 123 124 if __name__ == "__main__": 125 main() 126 127The script does not close the file explicitly. By default, Python does not emit 128any warning. Example using README.txt, which has 269 lines: 129 130.. code-block:: shell-session 131 132 $ python script.py README.txt 133 269 134 135Enabling the Python Development Mode displays a :exc:`ResourceWarning` warning: 136 137.. code-block:: shell-session 138 139 $ python -X dev script.py README.txt 140 269 141 script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'> 142 main() 143 ResourceWarning: Enable tracemalloc to get the object allocation traceback 144 145In addition, enabling :mod:`tracemalloc` shows the line where the file was 146opened: 147 148.. code-block:: shell-session 149 150 $ python -X dev -X tracemalloc=5 script.py README.rst 151 269 152 script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'> 153 main() 154 Object allocated at (most recent call last): 155 File "script.py", lineno 10 156 main() 157 File "script.py", lineno 4 158 fp = open(sys.argv[1]) 159 160The fix is to close explicitly the file. Example using a context manager:: 161 162 def main(): 163 # Close the file explicitly when exiting the with block 164 with open(sys.argv[1]) as fp: 165 nlines = len(fp.readlines()) 166 print(nlines) 167 168Not closing a resource explicitly can leave a resource open for way longer than 169expected; it can cause severe issues upon exiting Python. It is bad in 170CPython, but it is even worse in PyPy. Closing resources explicitly makes an 171application more deterministic and more reliable. 172 173 174Bad file descriptor error example 175--------------------------------- 176 177Script displaying the first line of itself:: 178 179 import os 180 181 def main(): 182 fp = open(__file__) 183 firstline = fp.readline() 184 print(firstline.rstrip()) 185 os.close(fp.fileno()) 186 # The file is closed implicitly 187 188 main() 189 190By default, Python does not emit any warning: 191 192.. code-block:: shell-session 193 194 $ python script.py 195 import os 196 197The Python Development Mode shows a :exc:`ResourceWarning` and logs a "Bad file 198descriptor" error when finalizing the file object: 199 200.. code-block:: shell-session 201 202 $ python -X dev script.py 203 import os 204 script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> 205 main() 206 ResourceWarning: Enable tracemalloc to get the object allocation traceback 207 Exception ignored in: <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> 208 Traceback (most recent call last): 209 File "script.py", line 10, in <module> 210 main() 211 OSError: [Errno 9] Bad file descriptor 212 213``os.close(fp.fileno())`` closes the file descriptor. When the file object 214finalizer tries to close the file descriptor again, it fails with the ``Bad 215file descriptor`` error. A file descriptor must be closed only once. In the 216worst case scenario, closing it twice can lead to a crash (see :issue:`18748` 217for an example). 218 219The fix is to remove the ``os.close(fp.fileno())`` line, or open the file with 220``closefd=False``. 221