Lines Matching +full:is +full:- +full:module
10 Python code in one :term:`module` gains access to the code in another module
11 by the process of :term:`importing` it. The :keyword:`import` statement is
12 the most common way of invoking the import machinery, but it is not the only
13 way. Functions such as :func:`importlib.import_module` and built-in
17 named module, then it binds the results of that search to a name in the local
18 scope. The search operation of the :keyword:`!import` statement is defined as
20 The return value of :func:`__import__` is used to perform the name
25 A direct call to :func:`__import__` performs only the module search and, if
26 found, the module creation operation. While certain side-effects may occur,
31 When an :keyword:`import` statement is executed, the standard builtin
32 :func:`__import__` function is called. Other mechanisms for invoking the
36 When a module is first imported, Python searches for the module and if found,
37 it creates a module object [#fnmo]_, initializing it. If the named module
38 cannot be found, a :exc:`ModuleNotFoundError` is raised. Python implements various
39 strategies to search for the named module when the import machinery is
45 of :pep:`302`. There is no longer any implicit import machinery - the full
46 import system is exposed through :data:`sys.meta_path`. In addition,
53 The :mod:`importlib` module provides a rich API for interacting with the
55 recommended, simpler API than built-in :func:`__import__` for invoking the
67 Python has only one type of module object, and all modules are of this type,
68 regardless of whether the module is implemented in Python, C, or something
82 module. Specifically, any module that contains a ``__path__`` attribute is
88 called :mod:`email.mime` and a module within that subpackage called
93 ----------------
101 A regular package is typically implemented as a directory containing an
102 ``__init__.py`` file. When a regular package is imported, this
103 ``__init__.py`` file is implicitly executed, and the objects it defines are
105 contain the same Python code that any other module can contain, and Python
106 will add some additional attributes to the module when it is imported.
127 ------------------
133 A namespace package is a composite of various :term:`portions <portion>`,
147 With namespace packages, there is no ``parent/__init__.py`` file. In fact,
149 each one is provided by a different portion. Thus ``parent/one`` may not be
151 namespace package for the top-level ``parent`` package whenever it or one of
152 its subpackages is imported.
161 name of the module (or package, but for the purposes of this discussion, the
162 difference is immaterial) being imported. This name may come from various
169 If any of the intermediate imports fail, a :exc:`ModuleNotFoundError` is raised.
172 The module cache
173 ----------------
178 The first place checked during import search is :data:`sys.modules`. This
182 and ``foo.bar.baz``. Each key will have as its value the corresponding module
185 During import, the module name is looked up in :data:`sys.modules` and if
186 present, the associated value is the module satisfying the import, and the
187 process completes. However, if the value is ``None``, then a
188 :exc:`ModuleNotFoundError` is raised. If the module name is missing, Python will
189 continue searching for the module.
191 :data:`sys.modules` is writable. Deleting a key may not destroy the
192 associated module (as other modules may hold references to it),
193 but it will invalidate the cache entry for the named module, causing
194 Python to search anew for the named module upon its next
196 of the module to result in a :exc:`ModuleNotFoundError`.
198 Beware though, as if you keep a reference to the module object,
199 invalidate its cache entry in :data:`sys.modules`, and then re-import the
200 named module, the two module objects will *not* be the same. By contrast,
201 :func:`importlib.reload` will reuse the *same* module object, and simply
202 reinitialise the module contents by rerunning the module's code.
205 .. _finders-and-loaders:
208 -------------------
213 single: module spec
215 If the named module is not found in :data:`sys.modules`, then Python's import
216 protocol is invoked to find and load the module. This protocol consists of
218 A finder's job is to determine whether it can find the named module using
220 interfaces are referred to as :term:`importers <importer>` - they return
221 themselves when they find that they can load the requested module.
224 knows how to locate built-in modules, and the second knows how to locate
226 for modules. The :term:`import path` is a list of locations that may
230 The import machinery is extensible, so new finders can be added to extend the
231 range and scope of module searching.
233 Finders do not actually load modules. If they can find the named module, they
234 return a :dfn:`module spec`, an encapsulation of the module's import-related
235 information, which the import machinery then uses when loading the module.
243 directly, whereas now they return module specs which *contain* loaders.
247 ------------
257 The import machinery is designed to be extensible; the primary mechanism for
264 modules, or even built-in modules. Meta hooks are registered by adding new
269 item is encountered. Import path hooks are registered by adding new callables
274 -------------
280 When the named module is not found in :data:`sys.modules`, Python next
283 the named module. Meta path finders must implement a method called
285 a name, an import path, and (optionally) a target module. The meta path
287 the named module or not.
289 If the meta path finder knows how to handle the named module, it returns a
290 spec object. If it cannot handle the named module, it returns ``None``. If
292 a spec, then a :exc:`ModuleNotFoundError` is raised. Any other exceptions
296 finders is called with two or three arguments. The first is the fully
297 qualified name of the module being imported, for example ``foo.bar.baz``.
298 The second argument is the path entries to use for the module search. For
299 top-level modules, the second argument is ``None``, but for submodules or
300 subpackages, the second argument is the value of the parent package's
302 be accessed, a :exc:`ModuleNotFoundError` is raised. The third argument
303 is an existing module object that will be the target of loading later.
304 The import system passes in a target module only during reload.
317 always return ``None`` when anything other than ``None`` is passed as the
321 knows how to import built-in modules, one that knows how to import frozen
328 is now deprecated. While it will continue to work without change, the
340 If and when a module spec is found, the import machinery will use it (and
341 the loader it contains) when loading the module. Here is an approximation
344 module = None
345 if spec.loader is not None and hasattr(spec.loader, 'create_module'):
346 # It is assumed 'exec_module' will also be defined on the loader.
347 module = spec.loader.create_module(spec)
348 if module is None:
349 module = ModuleType(spec.name)
350 # The import-related module attributes get set here:
351 _init_module_attrs(spec, module)
353 if spec.loader is None:
356 if spec.origin is None and spec.submodule_search_locations is not None:
358 sys.modules[spec.name] = module
360 module = spec.loader.load_module(spec.name)
363 sys.modules[spec.name] = module
365 spec.loader.exec_module(module)
376 * If there is an existing module object with the given name in
379 * The module will exist in :data:`sys.modules` before the loader
380 executes the module code. This is crucial because the module code may
385 * If loading fails, the failing module -- and only the failing module --
386 gets removed from :data:`sys.modules`. Any module already in the
387 :data:`sys.modules` cache, and any module that was successfully loaded
388 as a side-effect, must remain in the cache. This contrasts with
389 reloading where even the failing module is left in :data:`sys.modules`.
391 * After the module is created but before execution, the import machinery
392 sets the import-related module attributes ("_init_module_attrs" in
393 the pseudo-code example above), as summarized in a
394 :ref:`later section <import-mod-attrs>`.
396 * Module execution is the key moment of loading in which the module's
397 namespace gets populated. Execution is entirely delegated to the
400 * The module created during loading and passed to exec_module() may
409 -------
411 Module loaders provide the critical function of loading: module execution.
413 method with a single argument, the module object to execute. Any value
414 returned from :meth:`~importlib.abc.Loader.exec_module` is ignored.
418 * If the module is a Python module (as opposed to a built-in module or a
419 dynamically loaded extension), the loader should execute the module's code
420 in the module's global name space (``module.__dict__``).
422 * If the loader cannot execute the module, it should raise an
430 Module loaders may opt in to creating the module object during loading
432 It takes one argument, the module spec, and returns the new module object
434 on the module object. If the method returns ``None``, the
435 import machinery will create the new module itself.
451 functionality described above in addition to executing the module. All
454 * If there is an existing module object with the given name in
455 :data:`sys.modules`, the loader must use that existing module.
457 named module does not exist in :data:`sys.modules`, the loader
458 must create a new module object and add it to :data:`sys.modules`.
460 * The module *must* exist in :data:`sys.modules` before the loader
461 executes the module code, to prevent unbounded recursion or multiple
466 module(s), and only if the loader itself has loaded the module(s)
470 A :exc:`DeprecationWarning` is raised when ``exec_module()`` is defined but
471 ``create_module()`` is not.
474 An :exc:`ImportError` is raised when ``exec_module()`` is defined but
475 ``create_module()`` is not.
481 ----------
483 When a submodule is loaded using any mechanism (e.g. ``importlib`` APIs, the
484 ``import`` or ``import-from`` statements, or built-in ``__import__()``) a
485 binding is placed in the parent module's namespace to the submodule object.
487 ``spam.foo``, ``spam`` will have an attribute ``foo`` which is bound to the
499 ``spam`` module::
503 <module 'spam.foo' from '/tmp/imports/spam/foo.py'>
509 holding is that if you have ``sys.modules['spam']`` and
513 Module spec
514 -----------
516 The import machinery uses a variety of information about each module
517 during import, especially before loading. Most of the information is
518 common to all modules. The purpose of a module's spec is to encapsulate
519 this import-related information on a per-module basis.
522 system components, e.g. between the finder that creates the module spec
525 whereas without a module spec the loader had that responsibility.
527 The module's spec is exposed as the ``__spec__`` attribute on a module object.
529 the module spec.
533 .. _import-mod-attrs:
535 Import-related module attributes
536 --------------------------------
538 The import machinery fills in these attributes on each module object
539 during loading, based on the module's spec, before the loader executes
540 the module.
545 the module. This name is used to uniquely identify the module in
551 the import machinery used when loading the module. This is mostly
552 for introspection, but can be used for additional loader-specific
557 The module's ``__package__`` attribute must be set. Its value must
559 the module is a package, its ``__package__`` value should be set to
560 its ``__name__``. When the module is not a package, ``__package__``
561 should be set to the empty string for top-level modules, or for
565 This attribute is used instead of ``__name__`` to calculate explicit
566 relative imports for main modules, as defined in :pep:`366`. It is
570 The value of ``__package__`` is expected to be the same as
575 The ``__spec__`` attribute must be set to the module spec that was
576 used when importing the module. Setting ``__spec__``
578 interpreter startup <programs>`. The one exception is ``__main__``,
579 where ``__spec__`` is :ref:`set to None in some cases <main_spec>`.
581 When ``__package__`` is not defined, ``__spec__.parent`` is used as
587 ``__spec__.parent`` is used as a fallback when ``__package__`` is
592 If the module is a package (either regular or namespace), the module
595 If ``__path__`` is not empty, it must produce strings when iterated
597 :ref:`below <package-path-rules>`.
599 Non-package modules should not have a ``__path__`` attribute.
604 ``__file__`` is optional (if set, value must be a string). It indicates
605 the pathname of the file from which the module was loaded (if
611 meaning (e.g. a module loaded from a database).
613 If ``__file__`` is set then the ``__cached__`` attribute might also
614 be set, which is the path to any compiled version of
615 the code (e.g. byte-compiled file). The file does not need to exist
619 Note that ``__cached__`` may be set even if ``__file__`` is not
620 set. However, that scenario is quite atypical. Ultimately, the
621 loader is what makes use of the module spec provided by the finder
623 if a loader can load from a cached module but otherwise does not load
626 .. _package-path-rules:
628 module.__path__
629 ---------------
631 By definition, if a module has a ``__path__`` attribute, it is a package.
633 A package's ``__path__`` attribute is used during imports of its subpackages.
636 However, ``__path__`` is typically much more constrained than
651 Module reprs
652 ------------
655 attributes set above, and in the module's spec, you can more explicitly
656 control the repr of module objects.
658 If the module has a spec (``__spec__``), the import machinery will try
659 to generate a repr from it. If that fails or there is no spec, the import
660 system will craft a default repr using whatever information is available
661 on the module. It will try to use the ``module.__name__``,
662 ``module.__file__``, and ``module.__loader__`` as input into the repr,
663 with defaults for whatever information is missing.
667 * If the module has a ``__spec__`` attribute, the information in the spec
668 is used to generate the repr. The "name", "loader", "origin", and
671 * If the module has a ``__file__`` attribute, this is used as part of the
672 module's repr.
674 * If the module has no ``__file__`` but does have a ``__loader__`` that is not
675 ``None``, then the loader's repr is used as part of the module's repr.
677 * Otherwise, just use the module's ``__name__`` in the repr.
681 has been deprecated and the module spec is now used by the import
682 machinery to generate a module repr.
684 For backward compatibility with Python 3.3, the module repr will be
687 trying either approach described above. However, the method is deprecated.
692 use a module's ``__spec__`` attribute but before falling back on
693 ``__file__``. Use of :meth:`~importlib.abc.Loader.module_repr` is slated to
696 .. _pyc-invalidation:
699 ----------------------------
702 cache is up-to-date with the source ``.py`` file. By default, Python does this
703 by storing the source's last-modified timestamp and size in the cache file when
708 Python also supports "hash-based" cache files, which store a hash of the source
709 file's contents rather than its metadata. There are two variants of hash-based
710 ``.pyc`` files: checked and unchecked. For checked hash-based ``.pyc`` files,
712 resulting hash with the hash in the cache file. If a checked hash-based cache
713 file is found to be invalid, Python regenerates it and writes a new checked
714 hash-based cache file. For unchecked hash-based ``.pyc`` files, Python simply
715 assumes the cache file is valid if it exists. Hash-based ``.pyc`` files
716 validation behavior may be overridden with the :option:`--check-hash-based-pycs`
720 Added hash-based ``.pyc`` files. Previously, Python only supported
721 timestamp-based invalidation of bytecode caches.
744 module in the standard library, the default path entry finders also handle
756 described below, which was then used to get a loader for the module from the
774 ------------------
782 The :term:`path based finder` is responsible for finding and loading
783 Python modules and packages whose location is specified with a string
798 modules and packages. It is initialized from the :data:`PYTHONPATH`
799 environment variable and various other installation- and
800 implementation-specific defaults. Entries in :data:`sys.path` can name
802 (see the :mod:`site` module) that should be searched for modules, such as
806 The :term:`path based finder` is a :term:`meta path finder`, so the import
810 :meth:`~importlib.machinery.PathFinder.find_spec` is given, it will be a
811 list of string paths to traverse - typically a package's ``__path__``
812 attribute for an import within that package. If the ``path`` argument is
813 ``None``, this indicates a top level import and :data:`sys.path` is used.
820 a cache mapping path entries to path entry finders. This cache is maintained
824 location's :term:`path entry finder` need only be done once. User code is
828 If the path entry is not present in the cache, the path based finder iterates
830 hooks <path entry hook>` in this list is called with a single argument, the
833 :exc:`ImportError`. An :exc:`ImportError` is used by the path based finder to
836 exception is ignored and :term:`import path` iteration continues. The hook
838 is up to the hook (e.g. it may be a file system encoding, UTF-8, or something
845 in :data:`sys.path_importer_cache` (to indicate that there is no finder for
847 :term:`meta path finder` could not find the module.
849 If a :term:`path entry finder` *is* returned by one of the :term:`path entry
850 hook` callables on :data:`sys.path_hooks`, then the following protocol is used
851 to ask the finder for a module spec, which is then used when loading the
852 module.
854 The current working directory -- denoted by an empty string -- is handled
856 current working directory is found to not exist, no value is stored in
858 directory is looked up fresh for each module lookup. Third, the path used for
864 --------------------------
871 fully qualified name of the module being imported, and the (optional) target
872 module. ``find_spec()`` returns a fully populated spec for the module.
883 are now deprecated, but will be used if ``find_spec()`` is not defined.
887 sake of backward compatibility. However, if ``find_spec()`` is
891 fully qualified name of the module being imported. ``find_loader()``
892 returns a 2-tuple where the first item is the loader and the second item
893 is a namespace :term:`portion`.
902 The ``find_module()`` method on path entry finders is deprecated,
917 The most reliable mechanism for replacing the entire import system is to
921 If it is acceptable to only alter the behaviour of import statements
924 may also be employed at the module level to only alter the behaviour of
925 import statements within that module.
929 it is sufficient to raise :exc:`ModuleNotFoundError` directly from
967 for this is that::
971 should expose ``XXX.YYY.ZZZ`` as a usable expression, but .moduleY is
975 .. _import-dunder-main:
980 The :mod:`__main__` module is a special case relative to Python's import
981 system. As noted :ref:`elsewhere <programs>`, the ``__main__`` module
982 is directly initialized at interpreter startup, much like :mod:`sys` and
984 qualify as a built-in module. This is because the manner in which
985 ``__main__`` is initialized depends on the flags and other options with
986 which the interpreter is invoked.
991 -----------------
993 Depending on how :mod:`__main__` is initialized, ``__main__.__spec__``
996 When Python is started with the :option:`-m` option, ``__spec__`` is set
997 to the module spec of the corresponding module or package. ``__spec__`` is
998 also populated when the ``__main__`` module is loaded as part of executing a
1001 In :ref:`the remaining cases <using-on-interface-options>`
1002 ``__main__.__spec__`` is set to ``None``, as the code used to populate the
1003 :mod:`__main__` does not correspond directly with an importable module:
1005 - interactive prompt
1006 - :option:`-c` option
1007 - running from stdin
1008 - running directly from a source or bytecode file
1010 Note that ``__main__.__spec__`` is always ``None`` in the last case,
1011 *even if* the file could technically be imported directly as a module
1012 instead. Use the :option:`-m` switch if valid module metadata is desired
1015 Note also that even when ``__main__`` corresponds with an importable module
1016 and ``__main__.__spec__`` is set accordingly, they're still considered
1017 *distinct* modules. This is due to the fact that blocks guarded by
1018 ``if __name__ == "__main__":`` checks only execute when the module is used
1027 <https://www.python.org/doc/essays/packages/>`_ is still available to read,
1046 :pep:`451` adds the encapsulation of per-module import state in spec
1047 objects. It also off-loads most of the boilerplate responsibilities of
1057 directly. Instead, it gets the module object by looking the module name up
1058 in :data:`sys.modules`. The indirect effect of this is that an imported
1059 module may replace itself in :data:`sys.modules`. This is
1060 implementation-specific behavior that is not guaranteed to work in other
1063 .. [#fnpic] In legacy code, it is possible to find instances of
1065 is recommended that code be changed to use ``None`` instead. See