1.. _source-dist: 2 3****************************** 4Creating a Source Distribution 5****************************** 6 7As shown in section :ref:`distutils-simple-example`, you use the :command:`sdist` command 8to create a source distribution. In the simplest case, :: 9 10 python setup.py sdist 11 12(assuming you haven't specified any :command:`sdist` options in the setup script 13or config file), :command:`sdist` creates the archive of the default format for 14the current platform. The default format is a gzip'ed tar file 15(:file:`.tar.gz`) on Unix, and ZIP file on Windows. 16 17You can specify as many formats as you like using the :option:`!--formats` 18option, for example:: 19 20 python setup.py sdist --formats=gztar,zip 21 22to create a gzipped tarball and a zip file. The available formats are: 23 24+-----------+-------------------------+---------+ 25| Format | Description | Notes | 26+===========+=========================+=========+ 27| ``zip`` | zip file (:file:`.zip`) | (1),(3) | 28+-----------+-------------------------+---------+ 29| ``gztar`` | gzip'ed tar file | \(2) | 30| | (:file:`.tar.gz`) | | 31+-----------+-------------------------+---------+ 32| ``bztar`` | bzip2'ed tar file | | 33| | (:file:`.tar.bz2`) | | 34+-----------+-------------------------+---------+ 35| ``ztar`` | compressed tar file | \(4) | 36| | (:file:`.tar.Z`) | | 37+-----------+-------------------------+---------+ 38| ``tar`` | tar file (:file:`.tar`) | | 39+-----------+-------------------------+---------+ 40 41Notes: 42 43(1) 44 default on Windows 45 46(2) 47 default on Unix 48 49(3) 50 requires either external :program:`zip` utility or :mod:`zipfile` module (part 51 of the standard Python library since Python 1.6) 52 53(4) 54 requires the :program:`compress` program. 55 56When using any ``tar`` format (``gztar``, ``bztar``, ``ztar`` or 57``tar``) under Unix, you can specify the ``owner`` and ``group`` names 58that will be set for each member of the archive. 59 60For example, if you want all files of the archive to be owned by root:: 61 62 python setup.py sdist --owner=root --group=root 63 64 65.. _manifest: 66 67Specifying the files to distribute 68================================== 69 70If you don't supply an explicit list of files (or instructions on how to 71generate one), the :command:`sdist` command puts a minimal default set into the 72source distribution: 73 74* all Python source files implied by the ``py_modules`` and 75 ``packages`` options 76 77* all C source files mentioned in the ``ext_modules`` or 78 ``libraries`` options 79 80 .. XXX Getting C library sources is currently broken -- no 81 :meth:`get_source_files` method in :file:`build_clib.py`! 82 83* scripts identified by the ``scripts`` option 84 See :ref:`distutils-installing-scripts`. 85 86* anything that looks like a test script: :file:`test/test\*.py` (currently, the 87 Distutils don't do anything with test scripts except include them in source 88 distributions, but in the future there will be a standard for testing Python 89 module distributions) 90 91* :file:`README.txt` (or :file:`README`), :file:`setup.py` (or whatever you 92 called your setup script), and :file:`setup.cfg` 93 94* all files that matches the ``package_data`` metadata. 95 See :ref:`distutils-installing-package-data`. 96 97* all files that matches the ``data_files`` metadata. 98 See :ref:`distutils-additional-files`. 99 100Sometimes this is enough, but usually you will want to specify additional files 101to distribute. The typical way to do this is to write a *manifest template*, 102called :file:`MANIFEST.in` by default. The manifest template is just a list of 103instructions for how to generate your manifest file, :file:`MANIFEST`, which is 104the exact list of files to include in your source distribution. The 105:command:`sdist` command processes this template and generates a manifest based 106on its instructions and what it finds in the filesystem. 107 108If you prefer to roll your own manifest file, the format is simple: one filename 109per line, regular files (or symlinks to them) only. If you do supply your own 110:file:`MANIFEST`, you must specify everything: the default set of files 111described above does not apply in this case. 112 113.. versionchanged:: 2.7 114 An existing generated :file:`MANIFEST` will be regenerated without 115 :command:`sdist` comparing its modification time to the one of 116 :file:`MANIFEST.in` or :file:`setup.py`. 117 118.. versionchanged:: 2.7.1 119 :file:`MANIFEST` files start with a comment indicating they are generated. 120 Files without this comment are not overwritten or removed. 121 122.. versionchanged:: 2.7.3 123 :command:`sdist` will read a :file:`MANIFEST` file if no :file:`MANIFEST.in` 124 exists, like it did before 2.7. 125 126See :ref:`manifest_template` section for a syntax reference. 127 128 129.. _manifest-options: 130 131Manifest-related options 132======================== 133 134The normal course of operations for the :command:`sdist` command is as follows: 135 136* if the manifest file (:file:`MANIFEST` by default) exists and the first line 137 does not have a comment indicating it is generated from :file:`MANIFEST.in`, 138 then it is used as is, unaltered 139 140* if the manifest file doesn't exist or has been previously automatically 141 generated, read :file:`MANIFEST.in` and create the manifest 142 143* if neither :file:`MANIFEST` nor :file:`MANIFEST.in` exist, create a manifest 144 with just the default file set 145 146* use the list of files now in :file:`MANIFEST` (either just generated or read 147 in) to create the source distribution archive(s) 148 149There are a couple of options that modify this behaviour. First, use the 150:option:`!--no-defaults` and :option:`!--no-prune` to disable the standard 151"include" and "exclude" sets. 152 153Second, you might just want to (re)generate the manifest, but not create a 154source distribution:: 155 156 python setup.py sdist --manifest-only 157 158:option:`!-o` is a shortcut for :option:`!--manifest-only`. 159 160.. _manifest_template: 161 162The MANIFEST.in template 163======================== 164 165A :file:`MANIFEST.in` file can be added in a project to define the list of 166files to include in the distribution built by the :command:`sdist` command. 167 168When :command:`sdist` is run, it will look for the :file:`MANIFEST.in` file 169and interpret it to generate the :file:`MANIFEST` file that contains the 170list of files that will be included in the package. 171 172This mechanism can be used when the default list of files is not enough. 173(See :ref:`manifest`). 174 175Principle 176--------- 177 178The manifest template has one command per line, where each command specifies a 179set of files to include or exclude from the source distribution. For an 180example, let's look at the Distutils' own manifest template:: 181 182 include *.txt 183 recursive-include examples *.txt *.py 184 prune examples/sample?/build 185 186The meanings should be fairly clear: include all files in the distribution root 187matching :file:`\*.txt`, all files anywhere under the :file:`examples` directory 188matching :file:`\*.txt` or :file:`\*.py`, and exclude all directories matching 189:file:`examples/sample?/build`. All of this is done *after* the standard 190include set, so you can exclude files from the standard set with explicit 191instructions in the manifest template. (Or, you can use the 192:option:`!--no-defaults` option to disable the standard set entirely.) 193 194The order of commands in the manifest template matters: initially, we have the 195list of default files as described above, and each command in the template adds 196to or removes from that list of files. Once we have fully processed the 197manifest template, we remove files that should not be included in the source 198distribution: 199 200* all files in the Distutils "build" tree (default :file:`build/`) 201 202* all files in directories named :file:`RCS`, :file:`CVS`, :file:`.svn`, 203 :file:`.hg`, :file:`.git`, :file:`.bzr` or :file:`_darcs` 204 205Now we have our complete list of files, which is written to the manifest for 206future reference, and then used to build the source distribution archive(s). 207 208You can disable the default set of included files with the 209:option:`!--no-defaults` option, and you can disable the standard exclude set 210with :option:`!--no-prune`. 211 212Following the Distutils' own manifest template, let's trace how the 213:command:`sdist` command builds the list of files to include in the Distutils 214source distribution: 215 216#. include all Python source files in the :file:`distutils` and 217 :file:`distutils/command` subdirectories (because packages corresponding to 218 those two directories were mentioned in the ``packages`` option in the 219 setup script---see section :ref:`setup-script`) 220 221#. include :file:`README.txt`, :file:`setup.py`, and :file:`setup.cfg` (standard 222 files) 223 224#. include :file:`test/test\*.py` (standard files) 225 226#. include :file:`\*.txt` in the distribution root (this will find 227 :file:`README.txt` a second time, but such redundancies are weeded out later) 228 229#. include anything matching :file:`\*.txt` or :file:`\*.py` in the sub-tree 230 under :file:`examples`, 231 232#. exclude all files in the sub-trees starting at directories matching 233 :file:`examples/sample?/build`\ ---this may exclude files included by the 234 previous two steps, so it's important that the ``prune`` command in the manifest 235 template comes after the ``recursive-include`` command 236 237#. exclude the entire :file:`build` tree, and any :file:`RCS`, :file:`CVS`, 238 :file:`.svn`, :file:`.hg`, :file:`.git`, :file:`.bzr` and :file:`_darcs` 239 directories 240 241Just like in the setup script, file and directory names in the manifest template 242should always be slash-separated; the Distutils will take care of converting 243them to the standard representation on your platform. That way, the manifest 244template is portable across operating systems. 245 246Commands 247-------- 248 249The manifest template commands are: 250 251+-------------------------------------------+-----------------------------------------------+ 252| Command | Description | 253+===========================================+===============================================+ 254| :command:`include pat1 pat2 ...` | include all files matching any of the listed | 255| | patterns | 256+-------------------------------------------+-----------------------------------------------+ 257| :command:`exclude pat1 pat2 ...` | exclude all files matching any of the listed | 258| | patterns | 259+-------------------------------------------+-----------------------------------------------+ 260| :command:`recursive-include dir pat1 pat2 | include all files under *dir* matching any of | 261| ...` | the listed patterns | 262+-------------------------------------------+-----------------------------------------------+ 263| :command:`recursive-exclude dir pat1 pat2 | exclude all files under *dir* matching any of | 264| ...` | the listed patterns | 265+-------------------------------------------+-----------------------------------------------+ 266| :command:`global-include pat1 pat2 ...` | include all files anywhere in the source tree | 267| | matching --- & any of the listed patterns | 268+-------------------------------------------+-----------------------------------------------+ 269| :command:`global-exclude pat1 pat2 ...` | exclude all files anywhere in the source tree | 270| | matching --- & any of the listed patterns | 271+-------------------------------------------+-----------------------------------------------+ 272| :command:`prune dir` | exclude all files under *dir* | 273+-------------------------------------------+-----------------------------------------------+ 274| :command:`graft dir` | include all files under *dir* | 275+-------------------------------------------+-----------------------------------------------+ 276 277The patterns here are Unix-style "glob" patterns: ``*`` matches any sequence of 278regular filename characters, ``?`` matches any single regular filename 279character, and ``[range]`` matches any of the characters in *range* (e.g., 280``a-z``, ``a-zA-Z``, ``a-f0-9_.``). The definition of "regular filename 281character" is platform-specific: on Unix it is anything except slash; on Windows 282anything except backslash or colon. 283