1======================================= 2Build System Support 3======================================= 4 5What is it? 6------------- 7 8Python packaging has come `a long way <https://bernat.tech/posts/pep-517-518/>`_. 9 10The traditional ``setuptools`` way of packaging Python modules 11uses a ``setup()`` function within the ``setup.py`` script. Commands such as 12``python setup.py bdist`` or ``python setup.py bdist_wheel`` generate a 13distribution bundle and ``python setup.py install`` installs the distribution. 14This interface makes it difficult to choose other packaging tools without an 15overhaul. Because ``setup.py`` scripts allowed for arbitrary execution, it 16proved difficult to provide a reliable user experience across environments 17and history. 18 19`PEP 517 <https://www.python.org/dev/peps/pep-0517/>`_ therefore came to 20rescue and specified a new standard to 21package and distribute Python modules. Under PEP 517: 22 23 a ``pyproject.toml`` file is used to specify what program to use 24 for generating distribution. 25 26 Then, two functions provided by the program, ``build_wheel(directory: str)`` 27 and ``build_sdist(directory: str)`` create the distribution bundle at the 28 specified ``directory``. The program is free to use its own configuration 29 script or extend the ``.toml`` file. 30 31 Lastly, ``pip install *.whl`` or ``pip install *.tar.gz`` does the actual 32 installation. If ``*.whl`` is available, ``pip`` will go ahead and copy 33 the files into ``site-packages`` directory. If not, ``pip`` will look at 34 ``pyproject.toml`` and decide what program to use to 'build from source' 35 (the default is ``setuptools``) 36 37With this standard, switching between packaging tools becomes a lot easier. ``build_meta`` 38implements ``setuptools``' build system support. 39 40How to use it? 41-------------- 42 43Starting with a package that you want to distribute. You will need your source 44scripts, a ``pyproject.toml`` file and a ``setup.cfg`` file:: 45 46 ~/meowpkg/ 47 pyproject.toml 48 setup.cfg 49 meowpkg/__init__.py 50 51The pyproject.toml file is required to specify the build system (i.e. what is 52being used to package your scripts and install from source). To use it with 53setuptools, the content would be:: 54 55 [build-system] 56 requires = ["setuptools"] 57 build-backend = "setuptools.build_meta" 58 59The ``setuptools`` package implements the ``build_sdist`` 60command and the ``wheel`` package implements the ``build_wheel`` 61command; the latter is a dependency of the former 62exposed via :pep:`517` hooks. 63 64Use ``setuptools``' :ref:`declarative config <declarative config>` to 65specify the package information:: 66 67 [metadata] 68 name = meowpkg 69 version = 0.0.1 70 description = a package that meows 71 72 [options] 73 packages = find: 74 75.. _building: 76 77Now generate the distribution. To build the package, use 78`PyPA build <https://pypa-build.readthedocs.io/en/latest/>`_:: 79 80 $ pip install -q build 81 $ python -m build 82 83And now it's done! The ``.whl`` file and ``.tar.gz`` can then be distributed 84and installed:: 85 86 dist/ 87 meowpkg-0.0.1.whl 88 meowpkg-0.0.1.tar.gz 89 90 $ pip install dist/meowpkg-0.0.1.whl 91 92or:: 93 94 $ pip install dist/meowpkg-0.0.1.tar.gz 95 96Dynamic build dependencies and other ``build_meta`` tweaks 97---------------------------------------------------------- 98 99With the changes introduced by :pep:`517` and :pep:`518`, the 100``setup_requires`` configuration field was made deprecated in ``setup.cfg`` and 101``setup.py``, in favour of directly listing build dependencies in the 102``requires`` field of the ``build-system`` table of ``pyproject.toml``. 103This approach has a series of advantages and gives package managers and 104installers the ability to inspect in advance the build requirements and 105perform a series of optimisations. 106 107However some package authors might still need to dynamically inspect the final 108users machine before deciding these requirements. One way of doing that, as 109specified by :pep:`517`, is to "tweak" ``setuptools.build_meta`` by using a 110:pep:`in-tree backend <517#in-tree-build-backends>`. 111 112.. tip:: Before implementing a *in-tree* backend, have a look on 113 :pep:`PEP 508 <508#environment-markers>`. Most of the times, dependencies 114 with **environment markers** are enough to differentiate operating systems 115 and platforms. 116 117If you add the following configuration to your ``pyprojec.toml``: 118 119 120.. code-block:: toml 121 122 [build-system] 123 requires = ["setuptools", "wheel"] 124 build-backend = "backend" 125 backend-path = ["_custom_build"] 126 127 128then you should be able to implement a thin wrapper around ``build_meta`` in 129the ``_custom_build/backend.py`` file, as shown in the following example: 130 131.. code-block:: python 132 133 from setuptools import build_meta as _orig 134 135 prepare_metadata_for_build_wheel = _orig.prepare_metadata_for_build_wheel 136 build_wheel = _orig.build_wheel 137 build_sdist = _orig.build_sdist 138 139 140 def get_requires_for_build_wheel(self, config_settings=None): 141 return _orig.get_requires_for_build_wheel(config_settings) + [...] 142 143 144 def get_requires_for_build_sdist(self, config_settings=None): 145 return _orig.get_requires_for_build_sdist(config_settings) + [...] 146 147 148Note that you can override any of the functions specified in :pep:`PEP 517 149<517#build-backend-interface>`, not only the ones responsible for gathering 150requirements. 151 152.. important:: Make sure your backend script is included in the :doc:`source 153 distribution </userguide/distribution>`, otherwise the build will fail. 154 This can be done by using a SCM_/VCS_ plugin (like :pypi:`setuptools-scm` 155 and :pypi:`setuptools-svn`), or by correctly setting up :ref:`MANIFEST.in 156 <manifest>`. 157 158 If this is the first time you are using a customised backend, please have a 159 look on the generated ``.tar.gz`` and ``.whl``. 160 On POSIX systems that can be done with ``tar -tf dist/*.tar.gz`` 161 and ``unzip -l dist/*.whl``. 162 On Windows systems you can rename the ``.whl`` to ``.zip`` to be able to 163 inspect it on the file explorer, and use the same ``tar`` command in a 164 command prompt (alternativelly there are GUI programs like `7-zip`_ that 165 handle ``.tar.gz``). 166 167 In general the backend script should be present in the ``.tar.gz`` (so the 168 project can be build from the source) but not in the ``.whl`` (otherwise the 169 backend script would end up being distributed alongside your package). 170 See ":doc:`/userguide/package_discovery`" for more details about package 171 files. 172 173 174.. _SCM: https://en.wikipedia.org/wiki/Software_configuration_management 175.. _VCS: https://en.wikipedia.org/wiki/Version_control 176.. _7-zip: https://www.7-zip.org 177