1.. _source-dist: 2 3****************************** 4Creating a Source Distribution 5****************************** 6 7As shown in section :ref:`distutils-simple-example`, you use the :command:`sdist` command 8to create a source distribution. In the simplest case, :: 9 10 python setup.py sdist 11 12(assuming you haven't specified any :command:`sdist` options in the setup script 13or config file), :command:`sdist` creates the archive of the default format for 14the current platform. The default format is a gzip'ed tar file 15(:file:`.tar.gz`) on Unix, and ZIP file on Windows. 16 17You can specify as many formats as you like using the :option:`!--formats` 18option, for example:: 19 20 python setup.py sdist --formats=gztar,zip 21 22to create a gzipped tarball and a zip file. The available formats are: 23 24+-----------+-------------------------+---------+ 25| Format | Description | Notes | 26+===========+=========================+=========+ 27| ``zip`` | zip file (:file:`.zip`) | (1),(3) | 28+-----------+-------------------------+---------+ 29| ``gztar`` | gzip'ed tar file | \(2) | 30| | (:file:`.tar.gz`) | | 31+-----------+-------------------------+---------+ 32| ``bztar`` | bzip2'ed tar file | | 33| | (:file:`.tar.bz2`) | | 34+-----------+-------------------------+---------+ 35| ``xztar`` | xz'ed tar file | | 36| | (:file:`.tar.xz`) | | 37+-----------+-------------------------+---------+ 38| ``ztar`` | compressed tar file | \(4) | 39| | (:file:`.tar.Z`) | | 40+-----------+-------------------------+---------+ 41| ``tar`` | tar file (:file:`.tar`) | | 42+-----------+-------------------------+---------+ 43 44.. versionchanged:: 3.5 45 Added support for the ``xztar`` format. 46 47Notes: 48 49(1) 50 default on Windows 51 52(2) 53 default on Unix 54 55(3) 56 requires either external :program:`zip` utility or :mod:`zipfile` module (part 57 of the standard Python library since Python 1.6) 58 59(4) 60 requires the :program:`compress` program. Notice that this format is now 61 pending for deprecation and will be removed in the future versions of Python. 62 63When using any ``tar`` format (``gztar``, ``bztar``, ``xztar``, ``ztar`` or 64``tar``), under Unix you can specify the ``owner`` and ``group`` names 65that will be set for each member of the archive. 66 67For example, if you want all files of the archive to be owned by root:: 68 69 python setup.py sdist --owner=root --group=root 70 71 72.. _manifest: 73 74Specifying the files to distribute 75================================== 76 77If you don't supply an explicit list of files (or instructions on how to 78generate one), the :command:`sdist` command puts a minimal default set into the 79source distribution: 80 81* all Python source files implied by the ``py_modules`` and 82 ``packages`` options 83 84* all C source files mentioned in the ``ext_modules`` or 85 ``libraries`` options 86 87 .. XXX getting C library sources currently broken---no 88 :meth:`get_source_files` method in :file:`build_clib.py`! 89 90* scripts identified by the ``scripts`` option 91 See :ref:`distutils-installing-scripts`. 92 93* anything that looks like a test script: :file:`test/test\*.py` (currently, the 94 Distutils don't do anything with test scripts except include them in source 95 distributions, but in the future there will be a standard for testing Python 96 module distributions) 97 98* Any of the standard README files (:file:`README`, :file:`README.txt`, 99 or :file:`README.rst`), :file:`setup.py` (or whatever you called your setup 100 script), and :file:`setup.cfg`. 101 102* all files that matches the ``package_data`` metadata. 103 See :ref:`distutils-installing-package-data`. 104 105* all files that matches the ``data_files`` metadata. 106 See :ref:`distutils-additional-files`. 107 108Sometimes this is enough, but usually you will want to specify additional files 109to distribute. The typical way to do this is to write a *manifest template*, 110called :file:`MANIFEST.in` by default. The manifest template is just a list of 111instructions for how to generate your manifest file, :file:`MANIFEST`, which is 112the exact list of files to include in your source distribution. The 113:command:`sdist` command processes this template and generates a manifest based 114on its instructions and what it finds in the filesystem. 115 116If you prefer to roll your own manifest file, the format is simple: one filename 117per line, regular files (or symlinks to them) only. If you do supply your own 118:file:`MANIFEST`, you must specify everything: the default set of files 119described above does not apply in this case. 120 121.. versionchanged:: 3.1 122 An existing generated :file:`MANIFEST` will be regenerated without 123 :command:`sdist` comparing its modification time to the one of 124 :file:`MANIFEST.in` or :file:`setup.py`. 125 126.. versionchanged:: 3.1.3 127 :file:`MANIFEST` files start with a comment indicating they are generated. 128 Files without this comment are not overwritten or removed. 129 130.. versionchanged:: 3.2.2 131 :command:`sdist` will read a :file:`MANIFEST` file if no :file:`MANIFEST.in` 132 exists, like it used to do. 133 134.. versionchanged:: 3.7 135 :file:`README.rst` is now included in the list of distutils standard READMEs. 136 137 138The manifest template has one command per line, where each command specifies a 139set of files to include or exclude from the source distribution. For an 140example, again we turn to the Distutils' own manifest template: 141 142.. code-block:: none 143 144 include *.txt 145 recursive-include examples *.txt *.py 146 prune examples/sample?/build 147 148The meanings should be fairly clear: include all files in the distribution root 149matching :file:`\*.txt`, all files anywhere under the :file:`examples` directory 150matching :file:`\*.txt` or :file:`\*.py`, and exclude all directories matching 151:file:`examples/sample?/build`. All of this is done *after* the standard 152include set, so you can exclude files from the standard set with explicit 153instructions in the manifest template. (Or, you can use the 154:option:`!--no-defaults` option to disable the standard set entirely.) There are 155several other commands available in the manifest template mini-language; see 156section :ref:`sdist-cmd`. 157 158The order of commands in the manifest template matters: initially, we have the 159list of default files as described above, and each command in the template adds 160to or removes from that list of files. Once we have fully processed the 161manifest template, we remove files that should not be included in the source 162distribution: 163 164* all files in the Distutils "build" tree (default :file:`build/`) 165 166* all files in directories named :file:`RCS`, :file:`CVS`, :file:`.svn`, 167 :file:`.hg`, :file:`.git`, :file:`.bzr` or :file:`_darcs` 168 169Now we have our complete list of files, which is written to the manifest for 170future reference, and then used to build the source distribution archive(s). 171 172You can disable the default set of included files with the 173:option:`!--no-defaults` option, and you can disable the standard exclude set 174with :option:`!--no-prune`. 175 176Following the Distutils' own manifest template, let's trace how the 177:command:`sdist` command builds the list of files to include in the Distutils 178source distribution: 179 180#. include all Python source files in the :file:`distutils` and 181 :file:`distutils/command` subdirectories (because packages corresponding to 182 those two directories were mentioned in the ``packages`` option in the 183 setup script---see section :ref:`setup-script`) 184 185#. include :file:`README.txt`, :file:`setup.py`, and :file:`setup.cfg` (standard 186 files) 187 188#. include :file:`test/test\*.py` (standard files) 189 190#. include :file:`\*.txt` in the distribution root (this will find 191 :file:`README.txt` a second time, but such redundancies are weeded out later) 192 193#. include anything matching :file:`\*.txt` or :file:`\*.py` in the sub-tree 194 under :file:`examples`, 195 196#. exclude all files in the sub-trees starting at directories matching 197 :file:`examples/sample?/build`\ ---this may exclude files included by the 198 previous two steps, so it's important that the ``prune`` command in the manifest 199 template comes after the ``recursive-include`` command 200 201#. exclude the entire :file:`build` tree, and any :file:`RCS`, :file:`CVS`, 202 :file:`.svn`, :file:`.hg`, :file:`.git`, :file:`.bzr` and :file:`_darcs` 203 directories 204 205Just like in the setup script, file and directory names in the manifest template 206should always be slash-separated; the Distutils will take care of converting 207them to the standard representation on your platform. That way, the manifest 208template is portable across operating systems. 209 210 211.. _manifest-options: 212 213Manifest-related options 214======================== 215 216The normal course of operations for the :command:`sdist` command is as follows: 217 218* if the manifest file (:file:`MANIFEST` by default) exists and the first line 219 does not have a comment indicating it is generated from :file:`MANIFEST.in`, 220 then it is used as is, unaltered 221 222* if the manifest file doesn't exist or has been previously automatically 223 generated, read :file:`MANIFEST.in` and create the manifest 224 225* if neither :file:`MANIFEST` nor :file:`MANIFEST.in` exist, create a manifest 226 with just the default file set 227 228* use the list of files now in :file:`MANIFEST` (either just generated or read 229 in) to create the source distribution archive(s) 230 231There are a couple of options that modify this behaviour. First, use the 232:option:`!--no-defaults` and :option:`!--no-prune` to disable the standard 233"include" and "exclude" sets. 234 235Second, you might just want to (re)generate the manifest, but not create a source 236distribution:: 237 238 python setup.py sdist --manifest-only 239 240:option:`!-o` is a shortcut for :option:`!--manifest-only`. 241