Lines Matching +full:import +full:- +full:module
5 The import system
8 .. index:: single: import machinery
10 Python code in one :term:`module` gains access to the code in another module
11 by the process of :term:`importing` it. The :keyword:`import` statement is
12 the most common way of invoking the import machinery, but it is not the only
13 way. Functions such as :func:`importlib.import_module` and built-in
14 :func:`__import__` can also be used to invoke the import machinery.
16 The :keyword:`import` statement combines two operations; it searches for the
17 named module, then it binds the results of that search to a name in the local
18 scope. The search operation of the :keyword:`!import` statement is defined as
21 binding operation of the :keyword:`!import` statement. See the
22 :keyword:`!import` statement for the exact details of that name binding
25 A direct call to :func:`__import__` performs only the module search and, if
26 found, the module creation operation. While certain side-effects may occur,
28 (including :data:`sys.modules`), only the :keyword:`import` statement performs
31 When an :keyword:`import` statement is executed, the standard builtin
33 import system (such as :func:`importlib.import_module`) may choose to bypass
34 :func:`__import__` and use their own solutions to implement import semantics.
36 When a module is first imported, Python searches for the module and if found,
37 it creates a module object [#fnmo]_, initializing it. If the named module
39 strategies to search for the named module when the import machinery is
44 The import system has been updated to fully implement the second phase
45 of :pep:`302`. There is no longer any implicit import machinery - the full
46 import system is exposed through :data:`sys.meta_path`. In addition,
53 The :mod:`importlib` module provides a rich API for interacting with the
54 import system. For example :func:`importlib.import_module` provides a
55 recommended, simpler API than built-in :func:`__import__` for invoking the
56 import machinery. Refer to the :mod:`importlib` library documentation for
67 Python has only one type of module object, and all modules are of this type,
68 regardless of whether the module is implemented in Python, C, or something
82 module. Specifically, any module that contains a ``__path__`` attribute is
88 called :mod:`email.mime` and a module within that subpackage called
93 ----------------
105 contain the same Python code that any other module can contain, and Python
106 will add some additional attributes to the module when it is imported.
127 ------------------
137 during import. Namespace packages may or may not correspond directly to
143 perform a new search for package portions on the next import attempt within
148 there may be multiple ``parent`` directories found during import search, where
151 namespace package for the top-level ``parent`` package whenever it or one of
161 name of the module (or package, but for the purposes of this discussion, the
163 arguments to the :keyword:`import` statement, or from the parameters to the
166 This name will be used in various phases of the import search, and it may be
168 first tries to import ``foo``, then ``foo.bar``, and finally ``foo.bar.baz``.
172 The module cache
173 ----------------
178 The first place checked during import search is :data:`sys.modules`. This
182 and ``foo.bar.baz``. Each key will have as its value the corresponding module
185 During import, the module name is looked up in :data:`sys.modules` and if
186 present, the associated value is the module satisfying the import, and the
188 :exc:`ModuleNotFoundError` is raised. If the module name is missing, Python will
189 continue searching for the module.
192 associated module (as other modules may hold references to it),
193 but it will invalidate the cache entry for the named module, causing
194 Python to search anew for the named module upon its next
195 import. The key can also be assigned to ``None``, forcing the next import
196 of the module to result in a :exc:`ModuleNotFoundError`.
198 Beware though, as if you keep a reference to the module object,
199 invalidate its cache entry in :data:`sys.modules`, and then re-import the
200 named module, the two module objects will *not* be the same. By contrast,
201 :func:`importlib.reload` will reuse the *same* module object, and simply
202 reinitialise the module contents by rerunning the module's code.
205 .. _finders-and-loaders:
208 -------------------
213 single: module spec
215 If the named module is not found in :data:`sys.modules`, then Python's import
216 protocol is invoked to find and load the module. This protocol consists of
218 A finder's job is to determine whether it can find the named module using
220 interfaces are referred to as :term:`importers <importer>` - they return
221 themselves when they find that they can load the requested module.
224 knows how to locate built-in modules, and the second knows how to locate
225 frozen modules. A third default finder searches an :term:`import path`
226 for modules. The :term:`import path` is a list of locations that may
230 The import machinery is extensible, so new finders can be added to extend the
231 range and scope of module searching.
233 Finders do not actually load modules. If they can find the named module, they
234 return a :dfn:`module spec`, an encapsulation of the module's import-related
235 information, which the import machinery then uses when loading the module.
239 import machinery.
243 directly, whereas now they return module specs which *contain* loaders.
244 Loaders are still used during import but have fewer responsibilities.
246 Import hooks
247 ------------
250 single: import hooks
253 pair: hooks; import
257 The import machinery is designed to be extensible; the primary mechanism for
258 this are the *import hooks*. There are two types of import hooks: *meta
259 hooks* and *import path hooks*.
261 Meta hooks are called at the start of import processing, before any other
262 import processing has occurred, other than :data:`sys.modules` cache look up.
264 modules, or even built-in modules. Meta hooks are registered by adding new
267 Import path hooks are called as part of :data:`sys.path` (or
269 item is encountered. Import path hooks are registered by adding new callables
274 -------------
280 When the named module is not found in :data:`sys.modules`, Python next
283 the named module. Meta path finders must implement a method called
285 a name, an import path, and (optionally) a target module. The meta path
287 the named module or not.
289 If the meta path finder knows how to handle the named module, it returns a
290 spec object. If it cannot handle the named module, it returns ``None``. If
293 raised are simply propagated up, aborting the import process.
297 qualified name of the module being imported, for example ``foo.bar.baz``.
298 The second argument is the path entries to use for the module search. For
299 top-level modules, the second argument is ``None``, but for submodules or
303 is an existing module object that will be the target of loading later.
304 The import system passes in a target module only during reload.
306 The meta path may be traversed multiple times for a single import request.
308 importing ``foo.bar.baz`` will first perform a top level import, calling
321 knows how to import built-in modules, one that knows how to import frozen
322 modules, and one that knows how to import modules from an :term:`import path`
329 import machinery will try it only if the finder does not implement
333 Use of :meth:`~importlib.abc.MetaPathFinder.find_module` by the import system
340 If and when a module spec is found, the import machinery will use it (and
341 the loader it contains) when loading the module. Here is an approximation
342 of what happens during the loading portion of import::
344 module = None
347 module = spec.loader.create_module(spec)
348 if module is None:
349 module = ModuleType(spec.name)
350 # The import-related module attributes get set here:
351 _init_module_attrs(spec, module)
358 sys.modules[spec.name] = module
360 module = spec.loader.load_module(spec.name)
363 sys.modules[spec.name] = module
365 spec.loader.exec_module(module)
376 * If there is an existing module object with the given name in
377 :data:`sys.modules`, import will have already returned it.
379 * The module will exist in :data:`sys.modules` before the loader
380 executes the module code. This is crucial because the module code may
381 (directly or indirectly) import itself; adding it to :data:`sys.modules`
385 * If loading fails, the failing module -- and only the failing module --
386 gets removed from :data:`sys.modules`. Any module already in the
387 :data:`sys.modules` cache, and any module that was successfully loaded
388 as a side-effect, must remain in the cache. This contrasts with
389 reloading where even the failing module is left in :data:`sys.modules`.
391 * After the module is created but before execution, the import machinery
392 sets the import-related module attributes ("_init_module_attrs" in
393 the pseudo-code example above), as summarized in a
394 :ref:`later section <import-mod-attrs>`.
396 * Module execution is the key moment of loading in which the module's
400 * The module created during loading and passed to exec_module() may
401 not be the one returned at the end of import [#fnlo]_.
404 The import system has taken over the boilerplate responsibilities of
409 -------
411 Module loaders provide the critical function of loading: module execution.
412 The import machinery calls the :meth:`importlib.abc.Loader.exec_module`
413 method with a single argument, the module object to execute. Any value
418 * If the module is a Python module (as opposed to a built-in module or a
419 dynamically loaded extension), the loader should execute the module's code
420 in the module's global name space (``module.__dict__``).
422 * If the loader cannot execute the module, it should raise an
430 Module loaders may opt in to creating the module object during loading
432 It takes one argument, the module spec, and returns the new module object
434 on the module object. If the method returns ``None``, the
435 import machinery will create the new module itself.
442 :meth:`~importlib.abc.Loader.exec_module` and the import
445 For compatibility with existing loaders, the import machinery will use
451 functionality described above in addition to executing the module. All
454 * If there is an existing module object with the given name in
455 :data:`sys.modules`, the loader must use that existing module.
457 named module does not exist in :data:`sys.modules`, the loader
458 must create a new module object and add it to :data:`sys.modules`.
460 * The module *must* exist in :data:`sys.modules` before the loader
461 executes the module code, to prevent unbounded recursion or multiple
466 module(s), and only if the loader itself has loaded the module(s)
481 ----------
484 ``import`` or ``import-from`` statements, or built-in ``__import__()``) a
485 binding is placed in the parent module's namespace to the submodule object.
496 from .foo import Foo
499 ``spam`` module::
501 >>> import spam
503 <module 'spam.foo' from '/tmp/imports/spam/foo.py'>
508 it's actually a fundamental feature of the import system. The invariant
510 ``sys.modules['spam.foo']`` (as you would after the above import), the latter
513 Module spec
514 -----------
516 The import machinery uses a variety of information about each module
517 during import, especially before loading. Most of the information is
518 common to all modules. The purpose of a module's spec is to encapsulate
519 this import-related information on a per-module basis.
521 Using a spec during import allows state to be transferred between import
522 system components, e.g. between the finder that creates the module spec
524 import machinery to perform the boilerplate operations of loading,
525 whereas without a module spec the loader had that responsibility.
527 The module's spec is exposed as the ``__spec__`` attribute on a module object.
529 the module spec.
533 .. _import-mod-attrs:
535 Import-related module attributes
536 --------------------------------
538 The import machinery fills in these attributes on each module object
539 during loading, based on the module's spec, before the loader executes
540 the module.
545 the module. This name is used to uniquely identify the module in
546 the import system.
551 the import machinery used when loading the module. This is mostly
552 for introspection, but can be used for additional loader-specific
557 The module's ``__package__`` attribute must be set. Its value must
559 the module is a package, its ``__package__`` value should be set to
560 its ``__name__``. When the module is not a package, ``__package__``
561 should be set to the empty string for top-level modules, or for
575 The ``__spec__`` attribute must be set to the module spec that was
576 used when importing the module. Setting ``__spec__``
592 If the module is a package (either regular or namespace), the module
597 :ref:`below <package-path-rules>`.
599 Non-package modules should not have a ``__path__`` attribute.
605 the pathname of the file from which the module was loaded (if
610 import system may opt to leave it unset if it has no semantic
611 meaning (e.g. a module loaded from a database).
615 the code (e.g. byte-compiled file). The file does not need to exist
621 loader is what makes use of the module spec provided by the finder
623 if a loader can load from a cached module but otherwise does not load
626 .. _package-path-rules:
628 module.__path__
629 ---------------
631 By definition, if a module has a ``__path__`` attribute, it is a package.
634 Within the import machinery, it functions much the same as :data:`sys.path`,
635 i.e. providing a list of locations to search for modules during import.
648 manipulation code; the import machinery automatically sets ``__path__``
651 Module reprs
652 ------------
655 attributes set above, and in the module's spec, you can more explicitly
656 control the repr of module objects.
658 If the module has a spec (``__spec__``), the import machinery will try
659 to generate a repr from it. If that fails or there is no spec, the import
661 on the module. It will try to use the ``module.__name__``,
662 ``module.__file__``, and ``module.__loader__`` as input into the repr,
667 * If the module has a ``__spec__`` attribute, the information in the spec
671 * If the module has a ``__file__`` attribute, this is used as part of the
672 module's repr.
674 * If the module has no ``__file__`` but does have a ``__loader__`` that is not
675 ``None``, then the loader's repr is used as part of the module's repr.
677 * Otherwise, just use the module's ``__name__`` in the repr.
681 has been deprecated and the module spec is now used by the import
682 machinery to generate a module repr.
684 For backward compatibility with Python 3.3, the module repr will be
692 use a module's ``__spec__`` attribute but before falling back on
696 .. _pyc-invalidation:
699 ----------------------------
702 cache is up-to-date with the source ``.py`` file. By default, Python does this
703 by storing the source's last-modified timestamp and size in the cache file when
704 writing it. At runtime, the import system then validates the cache file by
708 Python also supports "hash-based" cache files, which store a hash of the source
709 file's contents rather than its metadata. There are two variants of hash-based
710 ``.pyc`` files: checked and unchecked. For checked hash-based ``.pyc`` files,
712 resulting hash with the hash in the cache file. If a checked hash-based cache
714 hash-based cache file. For unchecked hash-based ``.pyc`` files, Python simply
715 assumes the cache file is valid if it exists. Hash-based ``.pyc`` files
716 validation behavior may be overridden with the :option:`--check-hash-based-pycs`
720 Added hash-based ``.pyc`` files. Previously, Python only supported
721 timestamp-based invalidation of bytecode caches.
732 (:class:`~importlib.machinery.PathFinder`), searches an :term:`import path`,
736 The path based finder itself doesn't know how to import anything. Instead, it
744 module in the standard library, the default path entry finders also handle
756 described below, which was then used to get a loader for the module from the
762 support similar protocols, and function in similar ways during the import
764 In particular, meta path finders operate at the beginning of the import
774 ------------------
790 customize how modules are found and loaded from the :term:`import path`.
795 that the import machinery can be customized.
799 environment variable and various other installation- and
800 implementation-specific defaults. Entries in :data:`sys.path` can name
802 (see the :mod:`site` module) that should be searched for modules, such as
806 The :term:`path based finder` is a :term:`meta path finder`, so the import
807 machinery begins the :term:`import path` search by calling the path
811 list of string paths to traverse - typically a package's ``__path__``
812 attribute for an import within that package. If the ``path`` argument is
813 ``None``, this indicates a top level import and :data:`sys.path` is used.
836 exception is ignored and :term:`import path` iteration continues. The hook
838 is up to the hook (e.g. it may be a file system encoding, UTF-8, or something
847 :term:`meta path finder` could not find the module.
851 to ask the finder for a module spec, which is then used when loading the
852 module.
854 The current working directory -- denoted by an empty string -- is handled
858 directory is looked up fresh for each module lookup. Third, the path used for
864 --------------------------
871 fully qualified name of the module being imported, and the (optional) target
872 module. ``find_spec()`` returns a fully populated spec for the module.
875 To indicate to the import machinery that the spec represents a namespace
891 fully qualified name of the module being imported. ``find_loader()``
892 returns a 2-tuple where the first item is the loader and the second item
895 For backwards compatibility with other implementations of the import
905 exist on a path entry finder, the import system will always call
910 :meth:`~importlib.abc.PathEntryFinder.find_loader` by the import
914 Replacing the standard import system
917 The most reliable mechanism for replacing the entire import system is to
921 If it is acceptable to only alter the behaviour of import statements
922 without affecting other APIs that access the import system, then replacing
924 may also be employed at the module level to only alter the behaviour of
925 import statements within that module.
927 To selectively prevent the import of some modules from a hook early on the
928 meta path (rather than disabling the standard import system entirely),
940 import, starting with the current package. Two or more leading dots indicate a
941 relative import to the parent(s) of the current package, one level per dot
958 from .moduleY import spam
959 from .moduleY import spam as ham
960 from . import moduleY
961 from ..subpackage1 import moduleY
962 from ..subpackage2.moduleZ import eggs
963 from ..moduleA import foo
965 Absolute imports may use either the ``import <>`` or ``from <> import <>``
969 import XXX.YYY.ZZZ
975 .. _import-dunder-main:
980 The :mod:`__main__` module is a special case relative to Python's import
981 system. As noted :ref:`elsewhere <programs>`, the ``__main__`` module
984 qualify as a built-in module. This is because the manner in which
991 -----------------
996 When Python is started with the :option:`-m` option, ``__spec__`` is set
997 to the module spec of the corresponding module or package. ``__spec__`` is
998 also populated when the ``__main__`` module is loaded as part of executing a
1001 In :ref:`the remaining cases <using-on-interface-options>`
1003 :mod:`__main__` does not correspond directly with an importable module:
1005 - interactive prompt
1006 - :option:`-c` option
1007 - running from stdin
1008 - running directly from a source or bytecode file
1011 *even if* the file could technically be imported directly as a module
1012 instead. Use the :option:`-m` switch if valid module metadata is desired
1015 Note also that even when ``__main__`` corresponds with an importable module
1018 ``if __name__ == "__main__":`` checks only execute when the module is used
1019 to populate the ``__main__`` namespace, and not during normal import.
1025 The import machinery has evolved considerably since Python's early days. The
1046 :pep:`451` adds the encapsulation of per-module import state in spec
1047 objects. It also off-loads most of the boilerplate responsibilities of
1048 loaders back onto the import machinery. These changes allow the
1049 deprecation of several APIs in the import system and also addition of new
1057 directly. Instead, it gets the module object by looking the module name up
1059 module may replace itself in :data:`sys.modules`. This is
1060 implementation-specific behavior that is not guaranteed to work in other