1:mod:`compileall` --- Byte-compile Python libraries 2=================================================== 3 4.. module:: compileall 5 :synopsis: Tools for byte-compiling all Python source files in a directory tree. 6 7**Source code:** :source:`Lib/compileall.py` 8 9-------------- 10 11This module provides some utility functions to support installing Python 12libraries. These functions compile Python source files in a directory tree. 13This module can be used to create the cached byte-code files at library 14installation time, which makes them available for use even by users who don't 15have write permission to the library directories. 16 17 18Command-line use 19---------------- 20 21This module can work as a script (using :program:`python -m compileall`) to 22compile Python sources. 23 24.. program:: compileall 25 26.. cmdoption:: directory ... 27 file ... 28 29 Positional arguments are files to compile or directories that contain 30 source files, traversed recursively. If no argument is given, behave as if 31 the command line was ``-l <directories from sys.path>``. 32 33.. cmdoption:: -l 34 35 Do not recurse into subdirectories, only compile source code files directly 36 contained in the named or implied directories. 37 38.. cmdoption:: -f 39 40 Force rebuild even if timestamps are up-to-date. 41 42.. cmdoption:: -q 43 44 Do not print the list of files compiled. If passed once, error messages will 45 still be printed. If passed twice (``-qq``), all output is suppressed. 46 47.. cmdoption:: -d destdir 48 49 Directory prepended to the path to each file being compiled. This will 50 appear in compilation time tracebacks, and is also compiled in to the 51 byte-code file, where it will be used in tracebacks and other messages in 52 cases where the source file does not exist at the time the byte-code file is 53 executed. 54 55.. cmdoption:: -s strip_prefix 56.. cmdoption:: -p prepend_prefix 57 58 Remove (``-s``) or append (``-p``) the given prefix of paths 59 recorded in the ``.pyc`` files. 60 Cannot be combined with ``-d``. 61 62.. cmdoption:: -x regex 63 64 regex is used to search the full path to each file considered for 65 compilation, and if the regex produces a match, the file is skipped. 66 67.. cmdoption:: -i list 68 69 Read the file ``list`` and add each line that it contains to the list of 70 files and directories to compile. If ``list`` is ``-``, read lines from 71 ``stdin``. 72 73.. cmdoption:: -b 74 75 Write the byte-code files to their legacy locations and names, which may 76 overwrite byte-code files created by another version of Python. The default 77 is to write files to their :pep:`3147` locations and names, which allows 78 byte-code files from multiple versions of Python to coexist. 79 80.. cmdoption:: -r 81 82 Control the maximum recursion level for subdirectories. 83 If this is given, then ``-l`` option will not be taken into account. 84 :program:`python -m compileall <directory> -r 0` is equivalent to 85 :program:`python -m compileall <directory> -l`. 86 87.. cmdoption:: -j N 88 89 Use *N* workers to compile the files within the given directory. 90 If ``0`` is used, then the result of :func:`os.cpu_count()` 91 will be used. 92 93.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash] 94 95 Control how the generated byte-code files are invalidated at runtime. 96 The ``timestamp`` value, means that ``.pyc`` files with the source timestamp 97 and size embedded will be generated. The ``checked-hash`` and 98 ``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based 99 pycs embed a hash of the source file contents rather than a timestamp. See 100 :ref:`pyc-invalidation` for more information on how Python validates 101 bytecode cache files at runtime. 102 The default is ``timestamp`` if the :envvar:`SOURCE_DATE_EPOCH` environment 103 variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH`` 104 environment variable is set. 105 106.. cmdoption:: -o level 107 108 Compile with the given optimization level. May be used multiple times 109 to compile for multiple levels at a time (for example, 110 ``compileall -o 1 -o 2``). 111 112.. cmdoption:: -e dir 113 114 Ignore symlinks pointing outside the given directory. 115 116.. cmdoption:: --hardlink-dupes 117 118 If two ``.pyc`` files with different optimization level have 119 the same content, use hard links to consolidate duplicate files. 120 121.. versionchanged:: 3.2 122 Added the ``-i``, ``-b`` and ``-h`` options. 123 124.. versionchanged:: 3.5 125 Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option 126 was changed to a multilevel value. ``-b`` will always produce a 127 byte-code file ending in ``.pyc``, never ``.pyo``. 128 129.. versionchanged:: 3.7 130 Added the ``--invalidation-mode`` option. 131 132.. versionchanged:: 3.9 133 Added the ``-s``, ``-p``, ``-e`` and ``--hardlink-dupes`` options. 134 Raised the default recursion limit from 10 to 135 :py:func:`sys.getrecursionlimit()`. 136 Added the possibility to specify the ``-o`` option multiple times. 137 138 139There is no command-line option to control the optimization level used by the 140:func:`compile` function, because the Python interpreter itself already 141provides the option: :program:`python -O -m compileall`. 142 143Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix` 144setting. The generated bytecode cache will only be useful if :func:`compile` is 145run with the same :attr:`sys.pycache_prefix` (if any) that will be used at 146runtime. 147 148Public functions 149---------------- 150 151.. function:: compile_dir(dir, maxlevels=sys.getrecursionlimit(), ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False) 152 153 Recursively descend the directory tree named by *dir*, compiling all :file:`.py` 154 files along the way. Return a true value if all the files compiled successfully, 155 and a false value otherwise. 156 157 The *maxlevels* parameter is used to limit the depth of the recursion; it 158 defaults to ``sys.getrecursionlimit()``. 159 160 If *ddir* is given, it is prepended to the path to each file being compiled 161 for use in compilation time tracebacks, and is also compiled in to the 162 byte-code file, where it will be used in tracebacks and other messages in 163 cases where the source file does not exist at the time the byte-code file is 164 executed. 165 166 If *force* is true, modules are re-compiled even if the timestamps are up to 167 date. 168 169 If *rx* is given, its ``search`` method is called on the complete path to each 170 file considered for compilation, and if it returns a true value, the file 171 is skipped. This can be used to exclude files matching a regular expression, 172 given as a :ref:`re.Pattern <re-objects>` object. 173 174 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 175 information are printed to standard out. Set to ``1``, only errors are 176 printed. Set to ``2``, all output is suppressed. 177 178 If *legacy* is true, byte-code files are written to their legacy locations 179 and names, which may overwrite byte-code files created by another version of 180 Python. The default is to write files to their :pep:`3147` locations and 181 names, which allows byte-code files from multiple versions of Python to 182 coexist. 183 184 *optimize* specifies the optimization level for the compiler. It is passed to 185 the built-in :func:`compile` function. Accepts also a sequence of optimization 186 levels which lead to multiple compilations of one :file:`.py` file in one call. 187 188 The argument *workers* specifies how many workers are used to 189 compile files in parallel. The default is to not use multiple workers. 190 If the platform can't use multiple workers and *workers* argument is given, 191 then sequential compilation will be used as a fallback. If *workers* 192 is 0, the number of cores in the system is used. If *workers* is 193 lower than ``0``, a :exc:`ValueError` will be raised. 194 195 *invalidation_mode* should be a member of the 196 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 197 pycs are invalidated at runtime. 198 199 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to 200 the ``-s``, ``-p`` and ``-e`` options described above. 201 They may be specified as ``str``, ``bytes`` or :py:class:`os.PathLike`. 202 203 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization 204 level have the same content, use hard links to consolidate duplicate files. 205 206 .. versionchanged:: 3.2 207 Added the *legacy* and *optimize* parameter. 208 209 .. versionchanged:: 3.5 210 Added the *workers* parameter. 211 212 .. versionchanged:: 3.5 213 *quiet* parameter was changed to a multilevel value. 214 215 .. versionchanged:: 3.5 216 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 217 no matter what the value of *optimize* is. 218 219 .. versionchanged:: 3.6 220 Accepts a :term:`path-like object`. 221 222 .. versionchanged:: 3.7 223 The *invalidation_mode* parameter was added. 224 225 .. versionchanged:: 3.7.2 226 The *invalidation_mode* parameter's default value is updated to None. 227 228 .. versionchanged:: 3.8 229 Setting *workers* to 0 now chooses the optimal number of cores. 230 231 .. versionchanged:: 3.9 232 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments. 233 Default value of *maxlevels* was changed from ``10`` to ``sys.getrecursionlimit()`` 234 235.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False) 236 237 Compile the file with path *fullname*. Return a true value if the file 238 compiled successfully, and a false value otherwise. 239 240 If *ddir* is given, it is prepended to the path to the file being compiled 241 for use in compilation time tracebacks, and is also compiled in to the 242 byte-code file, where it will be used in tracebacks and other messages in 243 cases where the source file does not exist at the time the byte-code file is 244 executed. 245 246 If *rx* is given, its ``search`` method is passed the full path name to the 247 file being compiled, and if it returns a true value, the file is not 248 compiled and ``True`` is returned. This can be used to exclude files matching 249 a regular expression, given as a :ref:`re.Pattern <re-objects>` object. 250 251 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 252 information are printed to standard out. Set to ``1``, only errors are 253 printed. Set to ``2``, all output is suppressed. 254 255 If *legacy* is true, byte-code files are written to their legacy locations 256 and names, which may overwrite byte-code files created by another version of 257 Python. The default is to write files to their :pep:`3147` locations and 258 names, which allows byte-code files from multiple versions of Python to 259 coexist. 260 261 *optimize* specifies the optimization level for the compiler. It is passed to 262 the built-in :func:`compile` function. Accepts also a sequence of optimization 263 levels which lead to multiple compilations of one :file:`.py` file in one call. 264 265 *invalidation_mode* should be a member of the 266 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 267 pycs are invalidated at runtime. 268 269 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to 270 the ``-s``, ``-p`` and ``-e`` options described above. 271 They may be specified as ``str``, ``bytes`` or :py:class:`os.PathLike`. 272 273 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization 274 level have the same content, use hard links to consolidate duplicate files. 275 276 .. versionadded:: 3.2 277 278 .. versionchanged:: 3.5 279 *quiet* parameter was changed to a multilevel value. 280 281 .. versionchanged:: 3.5 282 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 283 no matter what the value of *optimize* is. 284 285 .. versionchanged:: 3.7 286 The *invalidation_mode* parameter was added. 287 288 .. versionchanged:: 3.7.2 289 The *invalidation_mode* parameter's default value is updated to None. 290 291 .. versionchanged:: 3.9 292 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments. 293 294.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) 295 296 Byte-compile all the :file:`.py` files found along ``sys.path``. Return a 297 true value if all the files compiled successfully, and a false value otherwise. 298 299 If *skip_curdir* is true (the default), the current directory is not included 300 in the search. All other parameters are passed to the :func:`compile_dir` 301 function. Note that unlike the other compile functions, ``maxlevels`` 302 defaults to ``0``. 303 304 .. versionchanged:: 3.2 305 Added the *legacy* and *optimize* parameter. 306 307 .. versionchanged:: 3.5 308 *quiet* parameter was changed to a multilevel value. 309 310 .. versionchanged:: 3.5 311 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 312 no matter what the value of *optimize* is. 313 314 .. versionchanged:: 3.7 315 The *invalidation_mode* parameter was added. 316 317 .. versionchanged:: 3.7.2 318 The *invalidation_mode* parameter's default value is updated to None. 319 320To force a recompile of all the :file:`.py` files in the :file:`Lib/` 321subdirectory and all its subdirectories:: 322 323 import compileall 324 325 compileall.compile_dir('Lib/', force=True) 326 327 # Perform same compilation, excluding files in .svn directories. 328 import re 329 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True) 330 331 # pathlib.Path objects can also be used. 332 import pathlib 333 compileall.compile_dir(pathlib.Path('Lib/'), force=True) 334 335.. seealso:: 336 337 Module :mod:`py_compile` 338 Byte-compile a single source file. 339