1.. _source-dist: 2 3****************************** 4Creating a Source Distribution 5****************************** 6 7.. include:: ./_setuptools_disclaimer.rst 8 9As shown in section :ref:`distutils-simple-example`, you use the :command:`sdist` command 10to create a source distribution. In the simplest case, :: 11 12 python setup.py sdist 13 14(assuming you haven't specified any :command:`sdist` options in the setup script 15or config file), :command:`sdist` creates the archive of the default format for 16the current platform. The default format is a gzip'ed tar file 17(:file:`.tar.gz`) on Unix, and ZIP file on Windows. 18 19You can specify as many formats as you like using the :option:`!--formats` 20option, for example:: 21 22 python setup.py sdist --formats=gztar,zip 23 24to create a gzipped tarball and a zip file. The available formats are: 25 26+-----------+-------------------------+-------------+ 27| Format | Description | Notes | 28+===========+=========================+=============+ 29| ``zip`` | zip file (:file:`.zip`) | (1),(3) | 30+-----------+-------------------------+-------------+ 31| ``gztar`` | gzip'ed tar file | \(2) | 32| | (:file:`.tar.gz`) | | 33+-----------+-------------------------+-------------+ 34| ``bztar`` | bzip2'ed tar file | \(5) | 35| | (:file:`.tar.bz2`) | | 36+-----------+-------------------------+-------------+ 37| ``xztar`` | xz'ed tar file | \(5) | 38| | (:file:`.tar.xz`) | | 39+-----------+-------------------------+-------------+ 40| ``ztar`` | compressed tar file | (4),(5) | 41| | (:file:`.tar.Z`) | | 42+-----------+-------------------------+-------------+ 43| ``tar`` | tar file (:file:`.tar`) | \(5) | 44+-----------+-------------------------+-------------+ 45 46.. versionchanged:: 3.5 47 Added support for the ``xztar`` format. 48 49Notes: 50 51(1) 52 default on Windows 53 54(2) 55 default on Unix 56 57(3) 58 requires either external :program:`zip` utility or :mod:`zipfile` module (part 59 of the standard Python library since Python 1.6) 60 61(4) 62 requires the :program:`compress` program. Notice that this format is now 63 pending for deprecation and will be removed in the future versions of Python. 64(5) 65 deprecated by `PEP 527 <https://www.python.org/dev/peps/pep-0527/>`_; 66 `PyPI <https://pypi.org>`_ only accepts ``.zip`` and ``.tar.gz`` files. 67 68When using any ``tar`` format (``gztar``, ``bztar``, ``xztar``, ``ztar`` or 69``tar``), under Unix you can specify the ``owner`` and ``group`` names 70that will be set for each member of the archive. 71 72For example, if you want all files of the archive to be owned by root:: 73 74 python setup.py sdist --owner=root --group=root 75 76 77.. _manifest: 78 79Specifying the files to distribute 80================================== 81 82If you don't supply an explicit list of files (or instructions on how to 83generate one), the :command:`sdist` command puts a minimal default set into the 84source distribution: 85 86* all Python source files implied by the ``py_modules`` and 87 ``packages`` options 88 89* all C source files mentioned in the ``ext_modules`` or 90 ``libraries`` options 91 92 .. XXX getting C library sources currently broken---no 93 :meth:`get_source_files` method in :file:`build_clib.py`! 94 95* scripts identified by the ``scripts`` option 96 See :ref:`distutils-installing-scripts`. 97 98* anything that looks like a test script: :file:`test/test\*.py` (currently, the 99 Distutils don't do anything with test scripts except include them in source 100 distributions, but in the future there will be a standard for testing Python 101 module distributions) 102 103* Any of the standard README files (:file:`README`, :file:`README.txt`, 104 or :file:`README.rst`), :file:`setup.py` (or whatever you called your setup 105 script), and :file:`setup.cfg`. 106 107* all files that matches the ``package_data`` metadata. 108 See :ref:`distutils-installing-package-data`. 109 110* all files that matches the ``data_files`` metadata. 111 See :ref:`distutils-additional-files`. 112 113Sometimes this is enough, but usually you will want to specify additional files 114to distribute. The typical way to do this is to write a *manifest template*, 115called :file:`MANIFEST.in` by default. The manifest template is just a list of 116instructions for how to generate your manifest file, :file:`MANIFEST`, which is 117the exact list of files to include in your source distribution. The 118:command:`sdist` command processes this template and generates a manifest based 119on its instructions and what it finds in the filesystem. 120 121If you prefer to roll your own manifest file, the format is simple: one filename 122per line, regular files (or symlinks to them) only. If you do supply your own 123:file:`MANIFEST`, you must specify everything: the default set of files 124described above does not apply in this case. 125 126.. versionchanged:: 3.1 127 An existing generated :file:`MANIFEST` will be regenerated without 128 :command:`sdist` comparing its modification time to the one of 129 :file:`MANIFEST.in` or :file:`setup.py`. 130 131.. versionchanged:: 3.1.3 132 :file:`MANIFEST` files start with a comment indicating they are generated. 133 Files without this comment are not overwritten or removed. 134 135.. versionchanged:: 3.2.2 136 :command:`sdist` will read a :file:`MANIFEST` file if no :file:`MANIFEST.in` 137 exists, like it used to do. 138 139.. versionchanged:: 3.7 140 :file:`README.rst` is now included in the list of distutils standard READMEs. 141 142 143The manifest template has one command per line, where each command specifies a 144set of files to include or exclude from the source distribution. For an 145example, again we turn to the Distutils' own manifest template: 146 147.. code-block:: none 148 149 include *.txt 150 recursive-include examples *.txt *.py 151 prune examples/sample?/build 152 153The meanings should be fairly clear: include all files in the distribution root 154matching :file:`\*.txt`, all files anywhere under the :file:`examples` directory 155matching :file:`\*.txt` or :file:`\*.py`, and exclude all directories matching 156:file:`examples/sample?/build`. All of this is done *after* the standard 157include set, so you can exclude files from the standard set with explicit 158instructions in the manifest template. (Or, you can use the 159:option:`!--no-defaults` option to disable the standard set entirely.) There are 160several other commands available in the manifest template mini-language; see 161section :ref:`sdist-cmd`. 162 163The order of commands in the manifest template matters: initially, we have the 164list of default files as described above, and each command in the template adds 165to or removes from that list of files. Once we have fully processed the 166manifest template, we remove files that should not be included in the source 167distribution: 168 169* all files in the Distutils "build" tree (default :file:`build/`) 170 171* all files in directories named :file:`RCS`, :file:`CVS`, :file:`.svn`, 172 :file:`.hg`, :file:`.git`, :file:`.bzr` or :file:`_darcs` 173 174Now we have our complete list of files, which is written to the manifest for 175future reference, and then used to build the source distribution archive(s). 176 177You can disable the default set of included files with the 178:option:`!--no-defaults` option, and you can disable the standard exclude set 179with :option:`!--no-prune`. 180 181Following the Distutils' own manifest template, let's trace how the 182:command:`sdist` command builds the list of files to include in the Distutils 183source distribution: 184 185#. include all Python source files in the :file:`distutils` and 186 :file:`distutils/command` subdirectories (because packages corresponding to 187 those two directories were mentioned in the ``packages`` option in the 188 setup script---see section :ref:`setup-script`) 189 190#. include :file:`README.txt`, :file:`setup.py`, and :file:`setup.cfg` (standard 191 files) 192 193#. include :file:`test/test\*.py` (standard files) 194 195#. include :file:`\*.txt` in the distribution root (this will find 196 :file:`README.txt` a second time, but such redundancies are weeded out later) 197 198#. include anything matching :file:`\*.txt` or :file:`\*.py` in the sub-tree 199 under :file:`examples`, 200 201#. exclude all files in the sub-trees starting at directories matching 202 :file:`examples/sample?/build`\ ---this may exclude files included by the 203 previous two steps, so it's important that the ``prune`` command in the manifest 204 template comes after the ``recursive-include`` command 205 206#. exclude the entire :file:`build` tree, and any :file:`RCS`, :file:`CVS`, 207 :file:`.svn`, :file:`.hg`, :file:`.git`, :file:`.bzr` and :file:`_darcs` 208 directories 209 210Just like in the setup script, file and directory names in the manifest template 211should always be slash-separated; the Distutils will take care of converting 212them to the standard representation on your platform. That way, the manifest 213template is portable across operating systems. 214 215 216.. _manifest-options: 217 218Manifest-related options 219======================== 220 221The normal course of operations for the :command:`sdist` command is as follows: 222 223* if the manifest file (:file:`MANIFEST` by default) exists and the first line 224 does not have a comment indicating it is generated from :file:`MANIFEST.in`, 225 then it is used as is, unaltered 226 227* if the manifest file doesn't exist or has been previously automatically 228 generated, read :file:`MANIFEST.in` and create the manifest 229 230* if neither :file:`MANIFEST` nor :file:`MANIFEST.in` exist, create a manifest 231 with just the default file set 232 233* use the list of files now in :file:`MANIFEST` (either just generated or read 234 in) to create the source distribution archive(s) 235 236There are a couple of options that modify this behaviour. First, use the 237:option:`!--no-defaults` and :option:`!--no-prune` to disable the standard 238"include" and "exclude" sets. 239 240Second, you might just want to (re)generate the manifest, but not create a source 241distribution:: 242 243 python setup.py sdist --manifest-only 244 245:option:`!-o` is a shortcut for :option:`!--manifest-only`. 246