1:mod:`compileall` --- Byte-compile Python libraries 2=================================================== 3 4.. module:: compileall 5 :synopsis: Tools for byte-compiling all Python source files in a directory tree. 6 7**Source code:** :source:`Lib/compileall.py` 8 9-------------- 10 11This module provides some utility functions to support installing Python 12libraries. These functions compile Python source files in a directory tree. 13This module can be used to create the cached byte-code files at library 14installation time, which makes them available for use even by users who don't 15have write permission to the library directories. 16 17 18Command-line use 19---------------- 20 21This module can work as a script (using :program:`python -m compileall`) to 22compile Python sources. 23 24.. program:: compileall 25 26.. cmdoption:: directory ... 27 file ... 28 29 Positional arguments are files to compile or directories that contain 30 source files, traversed recursively. If no argument is given, behave as if 31 the command line was ``-l <directories from sys.path>``. 32 33.. cmdoption:: -l 34 35 Do not recurse into subdirectories, only compile source code files directly 36 contained in the named or implied directories. 37 38.. cmdoption:: -f 39 40 Force rebuild even if timestamps are up-to-date. 41 42.. cmdoption:: -q 43 44 Do not print the list of files compiled. If passed once, error messages will 45 still be printed. If passed twice (``-qq``), all output is suppressed. 46 47.. cmdoption:: -d destdir 48 49 Directory prepended to the path to each file being compiled. This will 50 appear in compilation time tracebacks, and is also compiled in to the 51 byte-code file, where it will be used in tracebacks and other messages in 52 cases where the source file does not exist at the time the byte-code file is 53 executed. 54 55.. cmdoption:: -s strip_prefix 56.. cmdoption:: -p prepend_prefix 57 58 Remove (``-s``) or append (``-p``) the given prefix of paths 59 recorded in the ``.pyc`` files. 60 Cannot be combined with ``-d``. 61 62.. cmdoption:: -x regex 63 64 regex is used to search the full path to each file considered for 65 compilation, and if the regex produces a match, the file is skipped. 66 67.. cmdoption:: -i list 68 69 Read the file ``list`` and add each line that it contains to the list of 70 files and directories to compile. If ``list`` is ``-``, read lines from 71 ``stdin``. 72 73.. cmdoption:: -b 74 75 Write the byte-code files to their legacy locations and names, which may 76 overwrite byte-code files created by another version of Python. The default 77 is to write files to their :pep:`3147` locations and names, which allows 78 byte-code files from multiple versions of Python to coexist. 79 80.. cmdoption:: -r 81 82 Control the maximum recursion level for subdirectories. 83 If this is given, then ``-l`` option will not be taken into account. 84 :program:`python -m compileall <directory> -r 0` is equivalent to 85 :program:`python -m compileall <directory> -l`. 86 87.. cmdoption:: -j N 88 89 Use *N* workers to compile the files within the given directory. 90 If ``0`` is used, then the result of :func:`os.cpu_count()` 91 will be used. 92 93.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash] 94 95 Control how the generated byte-code files are invalidated at runtime. 96 The ``timestamp`` value, means that ``.pyc`` files with the source timestamp 97 and size embedded will be generated. The ``checked-hash`` and 98 ``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based 99 pycs embed a hash of the source file contents rather than a timestamp. See 100 :ref:`pyc-invalidation` for more information on how Python validates 101 bytecode cache files at runtime. 102 The default is ``timestamp`` if the :envvar:`SOURCE_DATE_EPOCH` environment 103 variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH`` 104 environment variable is set. 105 106.. cmdoption:: -o level 107 108 Compile with the given optimization level. May be used multiple times 109 to compile for multiple levels at a time (for example, 110 ``compileall -o 1 -o 2``). 111 112.. cmdoption:: -e dir 113 114 Ignore symlinks pointing outside the given directory. 115 116.. cmdoption:: --hardlink-dupes 117 118 If two ``.pyc`` files with different optimization level have 119 the same content, use hard links to consolidate duplicate files. 120 121.. versionchanged:: 3.2 122 Added the ``-i``, ``-b`` and ``-h`` options. 123 124.. versionchanged:: 3.5 125 Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option 126 was changed to a multilevel value. ``-b`` will always produce a 127 byte-code file ending in ``.pyc``, never ``.pyo``. 128 129.. versionchanged:: 3.7 130 Added the ``--invalidation-mode`` option. 131 132.. versionchanged:: 3.9 133 Added the ``-s``, ``-p``, ``-e`` and ``--hardlink-dupes`` options. 134 Raised the default recursion limit from 10 to 135 :py:func:`sys.getrecursionlimit()`. 136 Added the possibility to specify the ``-o`` option multiple times. 137 138 139There is no command-line option to control the optimization level used by the 140:func:`compile` function, because the Python interpreter itself already 141provides the option: :program:`python -O -m compileall`. 142 143Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix` 144setting. The generated bytecode cache will only be useful if :func:`compile` is 145run with the same :attr:`sys.pycache_prefix` (if any) that will be used at 146runtime. 147 148Public functions 149---------------- 150 151.. function:: compile_dir(dir, maxlevels=sys.getrecursionlimit(), ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None, \*, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False) 152 153 Recursively descend the directory tree named by *dir*, compiling all :file:`.py` 154 files along the way. Return a true value if all the files compiled successfully, 155 and a false value otherwise. 156 157 The *maxlevels* parameter is used to limit the depth of the recursion; it 158 defaults to ``sys.getrecursionlimit()``. 159 160 If *ddir* is given, it is prepended to the path to each file being compiled 161 for use in compilation time tracebacks, and is also compiled in to the 162 byte-code file, where it will be used in tracebacks and other messages in 163 cases where the source file does not exist at the time the byte-code file is 164 executed. 165 166 If *force* is true, modules are re-compiled even if the timestamps are up to 167 date. 168 169 If *rx* is given, its search method is called on the complete path to each 170 file considered for compilation, and if it returns a true value, the file 171 is skipped. 172 173 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 174 information are printed to standard out. Set to ``1``, only errors are 175 printed. Set to ``2``, all output is suppressed. 176 177 If *legacy* is true, byte-code files are written to their legacy locations 178 and names, which may overwrite byte-code files created by another version of 179 Python. The default is to write files to their :pep:`3147` locations and 180 names, which allows byte-code files from multiple versions of Python to 181 coexist. 182 183 *optimize* specifies the optimization level for the compiler. It is passed to 184 the built-in :func:`compile` function. Accepts also a sequence of optimization 185 levels which lead to multiple compilations of one :file:`.py` file in one call. 186 187 The argument *workers* specifies how many workers are used to 188 compile files in parallel. The default is to not use multiple workers. 189 If the platform can't use multiple workers and *workers* argument is given, 190 then sequential compilation will be used as a fallback. If *workers* 191 is 0, the number of cores in the system is used. If *workers* is 192 lower than ``0``, a :exc:`ValueError` will be raised. 193 194 *invalidation_mode* should be a member of the 195 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 196 pycs are invalidated at runtime. 197 198 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to 199 the ``-s``, ``-p`` and ``-e`` options described above. 200 They may be specified as ``str``, ``bytes`` or :py:class:`os.PathLike`. 201 202 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization 203 level have the same content, use hard links to consolidate duplicate files. 204 205 .. versionchanged:: 3.2 206 Added the *legacy* and *optimize* parameter. 207 208 .. versionchanged:: 3.5 209 Added the *workers* parameter. 210 211 .. versionchanged:: 3.5 212 *quiet* parameter was changed to a multilevel value. 213 214 .. versionchanged:: 3.5 215 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 216 no matter what the value of *optimize* is. 217 218 .. versionchanged:: 3.6 219 Accepts a :term:`path-like object`. 220 221 .. versionchanged:: 3.7 222 The *invalidation_mode* parameter was added. 223 224 .. versionchanged:: 3.7.2 225 The *invalidation_mode* parameter's default value is updated to None. 226 227 .. versionchanged:: 3.8 228 Setting *workers* to 0 now chooses the optimal number of cores. 229 230 .. versionchanged:: 3.9 231 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments. 232 Default value of *maxlevels* was changed from ``10`` to ``sys.getrecursionlimit()`` 233 234.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None, \*, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False) 235 236 Compile the file with path *fullname*. Return a true value if the file 237 compiled successfully, and a false value otherwise. 238 239 If *ddir* is given, it is prepended to the path to the file being compiled 240 for use in compilation time tracebacks, and is also compiled in to the 241 byte-code file, where it will be used in tracebacks and other messages in 242 cases where the source file does not exist at the time the byte-code file is 243 executed. 244 245 If *rx* is given, its search method is passed the full path name to the 246 file being compiled, and if it returns a true value, the file is not 247 compiled and ``True`` is returned. 248 249 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 250 information are printed to standard out. Set to ``1``, only errors are 251 printed. Set to ``2``, all output is suppressed. 252 253 If *legacy* is true, byte-code files are written to their legacy locations 254 and names, which may overwrite byte-code files created by another version of 255 Python. The default is to write files to their :pep:`3147` locations and 256 names, which allows byte-code files from multiple versions of Python to 257 coexist. 258 259 *optimize* specifies the optimization level for the compiler. It is passed to 260 the built-in :func:`compile` function. Accepts also a sequence of optimization 261 levels which lead to multiple compilations of one :file:`.py` file in one call. 262 263 *invalidation_mode* should be a member of the 264 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 265 pycs are invalidated at runtime. 266 267 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to 268 the ``-s``, ``-p`` and ``-e`` options described above. 269 They may be specified as ``str``, ``bytes`` or :py:class:`os.PathLike`. 270 271 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization 272 level have the same content, use hard links to consolidate duplicate files. 273 274 .. versionadded:: 3.2 275 276 .. versionchanged:: 3.5 277 *quiet* parameter was changed to a multilevel value. 278 279 .. versionchanged:: 3.5 280 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 281 no matter what the value of *optimize* is. 282 283 .. versionchanged:: 3.7 284 The *invalidation_mode* parameter was added. 285 286 .. versionchanged:: 3.7.2 287 The *invalidation_mode* parameter's default value is updated to None. 288 289 .. versionchanged:: 3.9 290 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments. 291 292.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) 293 294 Byte-compile all the :file:`.py` files found along ``sys.path``. Return a 295 true value if all the files compiled successfully, and a false value otherwise. 296 297 If *skip_curdir* is true (the default), the current directory is not included 298 in the search. All other parameters are passed to the :func:`compile_dir` 299 function. Note that unlike the other compile functions, ``maxlevels`` 300 defaults to ``0``. 301 302 .. versionchanged:: 3.2 303 Added the *legacy* and *optimize* parameter. 304 305 .. versionchanged:: 3.5 306 *quiet* parameter was changed to a multilevel value. 307 308 .. versionchanged:: 3.5 309 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 310 no matter what the value of *optimize* is. 311 312 .. versionchanged:: 3.7 313 The *invalidation_mode* parameter was added. 314 315 .. versionchanged:: 3.7.2 316 The *invalidation_mode* parameter's default value is updated to None. 317 318To force a recompile of all the :file:`.py` files in the :file:`Lib/` 319subdirectory and all its subdirectories:: 320 321 import compileall 322 323 compileall.compile_dir('Lib/', force=True) 324 325 # Perform same compilation, excluding files in .svn directories. 326 import re 327 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True) 328 329 # pathlib.Path objects can also be used. 330 import pathlib 331 compileall.compile_dir(pathlib.Path('Lib/'), force=True) 332 333.. seealso:: 334 335 Module :mod:`py_compile` 336 Byte-compile a single source file. 337