1:mod:`!compileall` --- Byte-compile Python libraries 2==================================================== 3 4.. module:: compileall 5 :synopsis: Tools for byte-compiling all Python source files in a directory tree. 6 7**Source code:** :source:`Lib/compileall.py` 8 9-------------- 10 11This module provides some utility functions to support installing Python 12libraries. These functions compile Python source files in a directory tree. 13This module can be used to create the cached byte-code files at library 14installation time, which makes them available for use even by users who don't 15have write permission to the library directories. 16 17.. include:: ../includes/wasm-notavail.rst 18 19.. _compileall-cli: 20 21Command-line use 22---------------- 23 24This module can work as a script (using :program:`python -m compileall`) to 25compile Python sources. 26 27.. program:: compileall 28 29.. option:: directory ... 30 file ... 31 32 Positional arguments are files to compile or directories that contain 33 source files, traversed recursively. If no argument is given, behave as if 34 the command line was :samp:`-l {<directories from sys.path>}`. 35 36.. option:: -l 37 38 Do not recurse into subdirectories, only compile source code files directly 39 contained in the named or implied directories. 40 41.. option:: -f 42 43 Force rebuild even if timestamps are up-to-date. 44 45.. option:: -q 46 47 Do not print the list of files compiled. If passed once, error messages will 48 still be printed. If passed twice (``-qq``), all output is suppressed. 49 50.. option:: -d destdir 51 52 Directory prepended to the path to each file being compiled. This will 53 appear in compilation time tracebacks, and is also compiled in to the 54 byte-code file, where it will be used in tracebacks and other messages in 55 cases where the source file does not exist at the time the byte-code file is 56 executed. 57 58.. option:: -s strip_prefix 59.. option:: -p prepend_prefix 60 61 Remove (``-s``) or append (``-p``) the given prefix of paths 62 recorded in the ``.pyc`` files. 63 Cannot be combined with ``-d``. 64 65.. option:: -x regex 66 67 regex is used to search the full path to each file considered for 68 compilation, and if the regex produces a match, the file is skipped. 69 70.. option:: -i list 71 72 Read the file ``list`` and add each line that it contains to the list of 73 files and directories to compile. If ``list`` is ``-``, read lines from 74 ``stdin``. 75 76.. option:: -b 77 78 Write the byte-code files to their legacy locations and names, which may 79 overwrite byte-code files created by another version of Python. The default 80 is to write files to their :pep:`3147` locations and names, which allows 81 byte-code files from multiple versions of Python to coexist. 82 83.. option:: -r 84 85 Control the maximum recursion level for subdirectories. 86 If this is given, then ``-l`` option will not be taken into account. 87 :program:`python -m compileall <directory> -r 0` is equivalent to 88 :program:`python -m compileall <directory> -l`. 89 90.. option:: -j N 91 92 Use *N* workers to compile the files within the given directory. 93 If ``0`` is used, then the result of :func:`os.process_cpu_count` 94 will be used. 95 96.. option:: --invalidation-mode [timestamp|checked-hash|unchecked-hash] 97 98 Control how the generated byte-code files are invalidated at runtime. 99 The ``timestamp`` value, means that ``.pyc`` files with the source timestamp 100 and size embedded will be generated. The ``checked-hash`` and 101 ``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based 102 pycs embed a hash of the source file contents rather than a timestamp. See 103 :ref:`pyc-invalidation` for more information on how Python validates 104 bytecode cache files at runtime. 105 The default is ``timestamp`` if the :envvar:`SOURCE_DATE_EPOCH` environment 106 variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH`` 107 environment variable is set. 108 109.. option:: -o level 110 111 Compile with the given optimization level. May be used multiple times 112 to compile for multiple levels at a time (for example, 113 ``compileall -o 1 -o 2``). 114 115.. option:: -e dir 116 117 Ignore symlinks pointing outside the given directory. 118 119.. option:: --hardlink-dupes 120 121 If two ``.pyc`` files with different optimization level have 122 the same content, use hard links to consolidate duplicate files. 123 124.. versionchanged:: 3.2 125 Added the ``-i``, ``-b`` and ``-h`` options. 126 127.. versionchanged:: 3.5 128 Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option 129 was changed to a multilevel value. ``-b`` will always produce a 130 byte-code file ending in ``.pyc``, never ``.pyo``. 131 132.. versionchanged:: 3.7 133 Added the ``--invalidation-mode`` option. 134 135.. versionchanged:: 3.9 136 Added the ``-s``, ``-p``, ``-e`` and ``--hardlink-dupes`` options. 137 Raised the default recursion limit from 10 to 138 :py:func:`sys.getrecursionlimit()`. 139 Added the possibility to specify the ``-o`` option multiple times. 140 141 142There is no command-line option to control the optimization level used by the 143:func:`compile` function, because the Python interpreter itself already 144provides the option: :program:`python -O -m compileall`. 145 146Similarly, the :func:`compile` function respects the :data:`sys.pycache_prefix` 147setting. The generated bytecode cache will only be useful if :func:`compile` is 148run with the same :data:`sys.pycache_prefix` (if any) that will be used at 149runtime. 150 151Public functions 152---------------- 153 154.. function:: compile_dir(dir, maxlevels=sys.getrecursionlimit(), ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False) 155 156 Recursively descend the directory tree named by *dir*, compiling all :file:`.py` 157 files along the way. Return a true value if all the files compiled successfully, 158 and a false value otherwise. 159 160 The *maxlevels* parameter is used to limit the depth of the recursion; it 161 defaults to ``sys.getrecursionlimit()``. 162 163 If *ddir* is given, it is prepended to the path to each file being compiled 164 for use in compilation time tracebacks, and is also compiled in to the 165 byte-code file, where it will be used in tracebacks and other messages in 166 cases where the source file does not exist at the time the byte-code file is 167 executed. 168 169 If *force* is true, modules are re-compiled even if the timestamps are up to 170 date. 171 172 If *rx* is given, its ``search`` method is called on the complete path to each 173 file considered for compilation, and if it returns a true value, the file 174 is skipped. This can be used to exclude files matching a regular expression, 175 given as a :ref:`re.Pattern <re-objects>` object. 176 177 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 178 information are printed to standard out. Set to ``1``, only errors are 179 printed. Set to ``2``, all output is suppressed. 180 181 If *legacy* is true, byte-code files are written to their legacy locations 182 and names, which may overwrite byte-code files created by another version of 183 Python. The default is to write files to their :pep:`3147` locations and 184 names, which allows byte-code files from multiple versions of Python to 185 coexist. 186 187 *optimize* specifies the optimization level for the compiler. It is passed to 188 the built-in :func:`compile` function. Accepts also a sequence of optimization 189 levels which lead to multiple compilations of one :file:`.py` file in one call. 190 191 The argument *workers* specifies how many workers are used to 192 compile files in parallel. The default is to not use multiple workers. 193 If the platform can't use multiple workers and *workers* argument is given, 194 then sequential compilation will be used as a fallback. If *workers* 195 is 0, the number of cores in the system is used. If *workers* is 196 lower than ``0``, a :exc:`ValueError` will be raised. 197 198 *invalidation_mode* should be a member of the 199 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 200 pycs are invalidated at runtime. 201 202 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to 203 the ``-s``, ``-p`` and ``-e`` options described above. 204 They may be specified as ``str`` or :py:class:`os.PathLike`. 205 206 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization 207 level have the same content, use hard links to consolidate duplicate files. 208 209 .. versionchanged:: 3.2 210 Added the *legacy* and *optimize* parameter. 211 212 .. versionchanged:: 3.5 213 Added the *workers* parameter. 214 215 .. versionchanged:: 3.5 216 *quiet* parameter was changed to a multilevel value. 217 218 .. versionchanged:: 3.5 219 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 220 no matter what the value of *optimize* is. 221 222 .. versionchanged:: 3.6 223 Accepts a :term:`path-like object`. 224 225 .. versionchanged:: 3.7 226 The *invalidation_mode* parameter was added. 227 228 .. versionchanged:: 3.7.2 229 The *invalidation_mode* parameter's default value is updated to ``None``. 230 231 .. versionchanged:: 3.8 232 Setting *workers* to 0 now chooses the optimal number of cores. 233 234 .. versionchanged:: 3.9 235 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments. 236 Default value of *maxlevels* was changed from ``10`` to ``sys.getrecursionlimit()`` 237 238.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False) 239 240 Compile the file with path *fullname*. Return a true value if the file 241 compiled successfully, and a false value otherwise. 242 243 If *ddir* is given, it is prepended to the path to the file being compiled 244 for use in compilation time tracebacks, and is also compiled in to the 245 byte-code file, where it will be used in tracebacks and other messages in 246 cases where the source file does not exist at the time the byte-code file is 247 executed. 248 249 If *rx* is given, its ``search`` method is passed the full path name to the 250 file being compiled, and if it returns a true value, the file is not 251 compiled and ``True`` is returned. This can be used to exclude files matching 252 a regular expression, given as a :ref:`re.Pattern <re-objects>` object. 253 254 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 255 information are printed to standard out. Set to ``1``, only errors are 256 printed. Set to ``2``, all output is suppressed. 257 258 If *legacy* is true, byte-code files are written to their legacy locations 259 and names, which may overwrite byte-code files created by another version of 260 Python. The default is to write files to their :pep:`3147` locations and 261 names, which allows byte-code files from multiple versions of Python to 262 coexist. 263 264 *optimize* specifies the optimization level for the compiler. It is passed to 265 the built-in :func:`compile` function. Accepts also a sequence of optimization 266 levels which lead to multiple compilations of one :file:`.py` file in one call. 267 268 *invalidation_mode* should be a member of the 269 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 270 pycs are invalidated at runtime. 271 272 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to 273 the ``-s``, ``-p`` and ``-e`` options described above. 274 They may be specified as ``str`` or :py:class:`os.PathLike`. 275 276 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization 277 level have the same content, use hard links to consolidate duplicate files. 278 279 .. versionadded:: 3.2 280 281 .. versionchanged:: 3.5 282 *quiet* parameter was changed to a multilevel value. 283 284 .. versionchanged:: 3.5 285 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 286 no matter what the value of *optimize* is. 287 288 .. versionchanged:: 3.7 289 The *invalidation_mode* parameter was added. 290 291 .. versionchanged:: 3.7.2 292 The *invalidation_mode* parameter's default value is updated to ``None``. 293 294 .. versionchanged:: 3.9 295 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments. 296 297.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) 298 299 Byte-compile all the :file:`.py` files found along ``sys.path``. Return a 300 true value if all the files compiled successfully, and a false value otherwise. 301 302 If *skip_curdir* is true (the default), the current directory is not included 303 in the search. All other parameters are passed to the :func:`compile_dir` 304 function. Note that unlike the other compile functions, ``maxlevels`` 305 defaults to ``0``. 306 307 .. versionchanged:: 3.2 308 Added the *legacy* and *optimize* parameter. 309 310 .. versionchanged:: 3.5 311 *quiet* parameter was changed to a multilevel value. 312 313 .. versionchanged:: 3.5 314 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 315 no matter what the value of *optimize* is. 316 317 .. versionchanged:: 3.7 318 The *invalidation_mode* parameter was added. 319 320 .. versionchanged:: 3.7.2 321 The *invalidation_mode* parameter's default value is updated to ``None``. 322 323To force a recompile of all the :file:`.py` files in the :file:`Lib/` 324subdirectory and all its subdirectories:: 325 326 import compileall 327 328 compileall.compile_dir('Lib/', force=True) 329 330 # Perform same compilation, excluding files in .svn directories. 331 import re 332 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True) 333 334 # pathlib.Path objects can also be used. 335 import pathlib 336 compileall.compile_dir(pathlib.Path('Lib/'), force=True) 337 338.. seealso:: 339 340 Module :mod:`py_compile` 341 Byte-compile a single source file. 342