1Quick Build Info
2================
3
4For testing, the installer should be built with the Tools/msi/build.bat
5script:
6
7 build.bat [-x86] [-x64] [--doc]
8
9For an official release, the installer should be built with the
10Tools/msi/buildrelease.bat script and environment variables:
11
12 set PYTHON=<path to Python 2.7 or 3.4>
13 set SPHINXBUILD=<path to sphinx-build.exe>
14 set PATH=<path to Mercurial (hg.exe)>;
15 <path to HTML Help Compiler (hhc.exe)>;%PATH%
16
17 buildrelease.bat [-x86] [-x64] [-D] [-B]
18 [-o <output directory>] [-c <certificate name>]
19
20See the Building the Installer section for more information.
21
22Overview
23========
24
25Python is distributed on Windows as an installer that will configure the
26user's system. This allows users to have a functioning copy of Python
27without having to build it themselves.
28
29The main tasks of the installer are:
30
31* copy required files into the expected layout
32* configure system settings so the installation can be located by
33 other programs
34* add entry points for modifying, repairing and uninstalling Python
35* make it easy to launch Python, its documentation, and IDLE
36
37Each of these is discussed in a later section of this document.
38
39Structure of the Installer
40==========================
41
42The installer is structured as a 'layout', which consists of a number of
43CAB and MSI files and a single EXE.
44
45The EXE is the main entry point into the installer. It contains the UI
46and command-line logic, as well as the ability to locate and optionally
47download other parts of the layout.
48
49Each MSI contains the logic required to install a component or feature
50of Python. These MSIs should not be launched directly by users. MSIs can
51be embedded into the EXE or automatically downloaded as needed.
52
53Each CAB contains the files making up a Python installation. CABs are
54embedded into their associated MSI and are never seen by users.
55
56MSIs are only required when the related feature or component is being
57installed. When components are not selected for installation, the
58associated MSI is not downloaded. This allows the installer to offer
59options to install debugging symbols and binaries without increasing
60the initial download size by separating them into their own MSIs.
61
62Building the Installer
63======================
64
65Before building the installer, download extra build dependencies using
66Tools\msi\get_externals.bat. (Note that this is in addition to the
67similarly named file in PCbuild.)
68
69One of the dependencies used in builds is WiX, a toolset that lets developers
70create installers for Windows Installer, the Windows installation engine. WiX
71has a dependency on the Microsoft .NET Framework Version 3.5 (which may not be
72configured on recent versions of Windows, such as Windows 10). If you are
73building on a recent Windows version, use the Control Panel (Programs | Programs
74and Features | Turn Windows Features on or off) and ensure that the entry
75".NET Framework 3.5 (includes .NET 2.0 and 3.0)" is enabled.
76
77For testing, the installer should be built with the Tools/msi/build.bat
78script:
79
80 build.bat [-x86] [-x64] [--doc] [--test-marker] [--pack]
81
82This script will build the required configurations of Python and
83generate an installer layout in PCbuild/(win32|amd64)/en-us.
84
85Specify -x86 and/or -x64 to build for each platform. If neither is
86specified, both platforms will be built. Currently, both the debug and
87release versions of Python are required for the installer.
88
89Specify --doc to build the documentation (.chm) file. If the file is not
90available, it will simply be excluded from the installer. Ensure
91%PYTHON% and %SPHINXBUILD% are set when passing this option. You may
92also set %HTMLHELP% to the Html Help Compiler (hhc.exe), or put HHC on
93your PATH or in externals/.
94
95Specify --test-marker to build an installer that works side-by-side with
96an official Python release. All registry keys and install locations will
97include an extra marker to avoid overwriting files. This marker is
98currently an 'x' prefix, but may change at any time.
99
100Specify --pack to build an installer that does not require all MSIs to
101be available alongside. This takes longer, but is easier to share.
102
103
104For an official release, the installer should be built with the
105Tools/msi/buildrelease.bat script:
106
107 set PYTHON=<path to Python 2.7 or 3.4>
108 set SPHINXBUILD=<path to sphinx-build.exe>
109 set PATH=<path to Mercurial (hg.exe)>;
110 <path to HTML Help Compiler (hhc.exe)>;%PATH%
111
112 buildrelease.bat [-x86] [-x64] [-D] [-B]
113 [-o <output directory>] [-c <certificate name>]
114
115Specify -x86 and/or -x64 to build for each platform. If neither is
116specified, both platforms will be built. Currently, both the debug and
117release versions of Python are required for the installer.
118
119Specify -D to skip rebuilding the documentation. The documentation is
120required for a release and the build will fail if it is not available.
121
122Specify -B to skip rebuilding Python. This is useful to only rebuild the
123installer layout after a previous call to buildrelease.bat.
124
125Specify -o to set an output directory. The installer layouts will be
126copied to platform-specific subdirectories of this path.
127
128Specify -c to choose a code-signing certificate to be used for all the
129signable binaries in Python as well as each file making up the
130installer. Official releases of Python must be signed.
131
132Ensure %PYTHON% and %SPHINXBUILD% are set when passing this option. You
133may also set %HTMLHELP% to the Html Help Compiler (hhc.exe), or put HHC
134on your PATH or in externals/. You will also need Mercurial (hg.exe) on
135your PATH.
136
137If WiX is not found on your system, it will be automatically downloaded
138and extracted to the externals/ directory.
139
140To manually build layouts of the installer, build one of the projects in
141the bundle folder.
142
143 msbuild bundle\snapshot.wixproj
144 msbuild bundle\releaseweb.wixproj
145 msbuild bundle\releaselocal.wixproj
146 msbuild bundle\full.wixproj
147
148snapshot.wixproj produces a test installer versioned based on the date.
149
150releaseweb.wixproj produces a release installer that does not embed any
151of the layout.
152
153releaselocal.wixproj produces a release installer that embeds the files
154required for a default installation.
155
156full.wixproj produces a test installer that embeds the entire layout.
157
158The following properties may be passed when building these projects.
159
160 /p:BuildForRelease=(true|false)
161 When true, adds extra verification to ensure a complete installer is
162 produced. For example, binutils is required when building for a release
163 to generate MinGW-compatible libraries, and the build will be aborted if
164 this fails. Defaults to false.
165
166 /p:ReleaseUri=(any URI)
167 Used to generate unique IDs for the installers to allow side-by-side
168 installation. Forks of Python can use the same installer infrastructure
169 by providing a unique URI for this property. It does not need to be an
170 active internet address. Defaults to $(ComputerName).
171
172 Official releases use http://www.python.org/(architecture name)
173
174 /p:DownloadUrlBase=(any URI)
175 Specifies the base of a URL where missing parts of the installer layout
176 can be downloaded from. The build version and architecture will be
177 appended to create the full address. If omitted, missing components will
178 not be automatically downloaded.
179
180 /p:DownloadUrl=(any URI)
181 Specifies the full URL where missing parts of the installer layout can
182 be downloaded from. Should normally include '{2}', which will be
183 substituted for the filename. If omitted, missing components will not be
184 automatically downloaded. If specified, this value overrides
185 DownloadUrlBase.
186
187 /p:SigningCertificate=(certificate name)
188 Specifies the certificate to sign the installer layout with. If omitted,
189 the layout will not be signed.
190
191 /p:RebuildAll=(true|false)
192 When true, rebuilds all of the MSIs making up the layout. Defaults to
193 true.
194
195Uploading the Installer
196=======================
197
198For official releases, the uploadrelease.bat script should be used.
199
200You will require PuTTY so that plink.exe and pscp.exe can be used, and your
201SSH key can be activated in pageant.exe. PuTTY should be either on your path
202or in %ProgramFiles(x86)%\PuTTY.
203
204To include signatures for each uploaded file, you will need gpg2.exe on your
205path or have run get_externals.bat. You may also need to "gpg2.exe --import"
206your key before running the upload script.
207
208 uploadrelease.bat --host <host> --user <username> [--dry-run] [--no-gpg]
209
210The host is the URL to the server. This can be provided by the Release
211Manager. You should be able to SSH to this address.
212
213The username is your own username, which you have permission to SSH into
214the server containing downloads.
215
216Use --dry-run to display the generated upload commands without executing
217them. Signatures for each file will be generated but not uploaded unless
218--no-gpg is also passed.
219
220Use --no-gpg to suppress signature generation and upload.
221
222The default target directory (which appears in uploadrelease.proj) is
223correct for official Python releases, but may be overridden with
224--target <path> for other purposes. This path should generally not include
225any version specifier, as that will be added automatically.
226
227Modifying the Installer
228=======================
229
230The code for the installer is divided into three main groups: packages,
231the bundle and the bootstrap application.
232
233Packages
234--------
235
236Packages appear as subdirectories of Tools/msi (other than the bundle/
237directory). The project file is a .wixproj and the build output is a
238single MSI. Packages are built with the WiX Toolset. Some project files
239share source files and use preprocessor directives to enable particular
240features. These are typically used to keep the sources close when the
241files are related, but produce multiple independent packages.
242
243A package is the smallest element that may be independently installed or
244uninstalled (as used in this installer). For example, the test suite has
245its own package, as users can choose to add or remove it after the
246initial installation.
247
248All the files installed by a single package should be related, though
249some packages may not install any files. For example, the pip package
250executes the ensurepip package, but does not add or remove any of its
251own files. (It is represented as a package because of its
252installed/uninstalled nature, as opposed to the "precompile standard
253library" option, for example.) Dependencies between packages are handled
254by the bundle, but packages should detect when dependencies are missing
255and raise an error.
256
257Packages that include a lot of files may use an InstallFiles element in
258the .wixproj file to generate sources. See lib/lib.wixproj for an
259example, and msi.targets and csv_to_wxs.py for the implementation. This
260element is also responsible for generating the code for cleaning up and
261removing __pycache__ folders in any directory containing .py files.
262
263All packages are built with the Tools/msi/common.wxs file, and so any
264directory or property in this file may be referenced. Of particular
265interest:
266
267 REGISTRYKEY (property)
268 The registry key for the current installation.
269
270 InstallDirectory (directory)
271 The root install directory for the current installation. Subdirectories
272 are also specified in this file (DLLs, Lib, etc.)
273
274 MenuDir (directory)
275 The Start Menu folder for the current installation.
276
277 UpgradeTable (property)
278 Every package should reference this property to include upgrade
279 information.
280
281 OptionalFeature (Component)
282 Packages that may be enabled or disabled should reference this component
283 and have an OPTIONAL_FEATURES entry in the bootstrap application to
284 properly handle Modify and Upgrade.
285
286The .wxl_template file is specially handled by the build system for this
287project to perform {{substitutions}} as defined in msi.targets. They
288should be included in projects as <WxlTemplate> items, where .wxl files
289are normally included as <EmbeddedResource> items.
290
291Bundle
292------
293
294The bundle is compiled to the main EXE entry point that for most users
295will represent the Python installer. It is built from Tools/msi/bundle
296with packages references in Tools/msi/bundle/packagegroups.
297
298Build logic for the bundle is in bundle.targets, but should be invoked
299through one of the .wixproj files as described in Building the
300Installer.
301
302The UI is separated between Default.thm (UI layout), Default.wxl
303(strings), bundle.wxs (properties) and the bootstrap application.
304Bundle.wxs also contains the chain, which is the list of packages to
305install and the order they should be installed in. These refer to named
306package groups in bundle/packagegroups.
307
308Each package group specifies one or more packages to install. Most
309packages require two separate entries to support both per-user and
310all-users installations. Because these reuse the same package, it does
311not increase the overall size of the package.
312
313Package groups refer to payload groups, which allow better control over
314embedding and downloading files than the default settings. Whether files
315are embedded and where they are downloaded from depends on settings
316created by the project files.
317
318Package references can include install conditions that determine when to
319install the package. When a package is a dependency for others, the
320condition should be crafted to ensure it is installed.
321
322MSI packages are installed or uninstalled based on their current state
323and the install condition. This makes them most suitable for features
324that are clearly present or absent from the user's machine.
325
326EXE packages are executed based on a customisable condition that can be
327omitted. This makes them suitable for pre- or post-install tasks that
328need to run regardless of whether features have been added or removed.
329
330Bootstrap Application
331---------------------
332
333The bootstrap application is a C++ application that controls the UI and
334installation. While it does not directly compile into the main EXE of
335the installer, it forms the main active component. Most of the
336installation functionality is provided by WiX, and so the bootstrap
337application is predominantly responsible for the code behind the UI that
338is defined in the Default.thm file. The bootstrap application code is in
339bundle/bootstrap and is built automatically when building the bundle.
340
341Installation Layout
342===================
343
344There are two installation layouts for Python on Windows, with the only
345differences being supporting files. A layout is selected implicitly
346based on whether the install is for all users of the machine or just for
347the user performing the installation.
348
349The default installation location when installing for all users is
350"%ProgramFiles%\Python3X" for the 64-bit interpreter and
351"%ProgramFiles(x86)%\Python3X-32" for the 32-bit interpreter. (Note that
352the latter path is equivalent to "%ProgramFiles%\Python3X-32" when
353running a 32-bit version of Windows.) This location requires
354administrative privileges to install or later modify the installation.
355
356The default installation location when installing for the current user
357is "%LocalAppData%\Programs\Python\Python3X" for the 64-bit interpreter
358and "%LocalAppData%\Programs\Python\Python3X-32" for the 32-bit
359interpreter. Only the current user can access this location. This
360provides a suitable level of protection against malicious modification
361of Python's files.
362
363(Default installation locations are set in Tools\msi\bundle\bundle.wxs.)
364
365Within this install directory is the following approximate layout:
366
367.\python[w].exe The core executable files
368.\DLLs Stdlib extensions (*.pyd) and dependencies
369.\Doc Documentation (*.chm)
370.\include Development headers (*.h)
371.\Lib Standard library
372.\Lib\test Test suite
373.\libs Development libraries (*.lib)
374.\Scripts Launcher scripts (*.exe, *.py)
375.\tcl Tcl dependencies (*.dll, *.tcl and others)
376.\Tools Tool scripts (*.py)
377
378When installed for all users, the following files are installed to
379either "%SystemRoot%\System32" or "%SystemRoot%\SysWOW64" as
380appropriate. For the current user, they are installed in the Python
381install directory.
382
383.\python3x.dll The core interpreter
384.\python3.dll The stable ABI reference
385
386When installed for all users, the following files are installed to
387"%SystemRoot%" (typically "C:\Windows") to ensure they are always
388available on PATH. (See Launching Python below.) For the current user,
389they are installed in "%LocalAppData%\Programs\Python\PyLauncher".
390
391.\py[w].exe PEP 397 launcher
392
393System Settings
394===============
395
396On installation, registry keys are created so that other applications
397can locate and identify installations of Python. The locations of these
398keys vary based on the install type.
399
400For 64-bit interpreters installed for all users, the root key is:
401 HKEY_LOCAL_MACHINE\Software\Python\PythonCore\3.X
402
403For 32-bit interpreters installed for all users on a 64-bit operating
404system, the root key is:
405 HKEY_LOCAL_MACHINE\Software\Wow6432Node\Python\PythonCore\3.X-32
406
407For 32-bit interpreters installed for all users on a 32-bit operating
408system, the root key is:
409 HKEY_LOCAL_MACHINE\Software\Python\PythonCore\3.X-32
410
411For 64-bit interpreters installed for the current user:
412 HKEY_CURRENT_USER\Software\Python\PythonCore\3.X
413
414For 32-bit interpreters installed for the current user:
415 HKEY_CURRENT_USER\Software\Python\PythonCore\3.X-32
416
417When the core Python executables are installed, a key "InstallPath" is
418created within the root key with its default value set to the
419executable's install directory. A value named "ExecutablePath" is added
420with the full path to the main Python interpreter, and a key
421"InstallGroup" is created with its default value set to the product
422name "Python 3.X".
423
424When the Python standard library is installed, a key "PythonPath" is
425created within the root key with its default value set to the full path
426to the Lib folder followed by the path to the DLLs folder, separated by
427a semicolon.
428
429When the documentation is installed, a key "Help" is created within the
430root key, with a subkey "Main Python Documentation" with its default
431value set to the full path to the installed CHM file.
432
433
434The py.exe launcher is installed as part of a regular Python install,
435but using a separate mechanism that allows it to more easily span
436versions of Python. As a result, it has different root keys for its
437registry entries:
438
439When installed for all users on a 64-bit operating system, the
440launcher's root key is:
441 HKEY_LOCAL_MACHINE\Software\Wow6432Node\Python\Launcher
442
443When installed for all users on a 32-bit operating system, the
444launcher's root key is:
445 HKEY_LOCAL_MACHINE\Software\Python\Launcher
446
447When installed for the current user:
448 HKEY_CURRENT_USER\Software\Python\Launcher
449
450When the launcher is installed, a key "InstallPath" is created within
451its root key with its default value set to the launcher's install
452directory. File associations are also created for .py, .pyw, .pyc and
453.pyo files.
454
455Launching Python
456================
457
458When a feature offering user entry points in the Start Menu is
459installed, a folder "Python 3.X" is created. Every shortcut should be
460created within this folder, and each shortcut should include the version
461and platform to allow users to identify the shortcut in a search results
462page.
463
464The core Python executables creates a shortcut "Python 3.X (32-bit)" or
465"Python 3.X (64-bit)" depending on the interpreter.
466
467The documentation creates a shortcut "Python 3.X 32-bit Manuals" or
468"Python 3.X 64-bit Manuals". The documentation is identical for all
469platforms, but the shortcuts need to be separate to avoid uninstallation
470conflicts.
471
472Installing IDLE creates a shortcut "IDLE (Python 3.X 32-bit)" or "IDLE
473(Python 3.X 64-bit)" depending on the interpreter.
474
475
476For users who often launch Python from a Command Prompt, an option is
477provided to add the directory containing python.exe to the user or
478system PATH variable. If the option is selected, the install directory
479and the Scripts directory will be added at the start of the system PATH
480for an all users install and the user PATH for a per-user install.
481
482When the user only has one version of Python installed, this will behave
483as expected. However, because Windows searches the system PATH before
484the user PATH, users cannot override a system-wide installation of
485Python on their PATH. Further, because the installer can only prepend to
486the path, later installations of Python will take precedence over
487earlier installations, regardless of interpreter version.
488
489Because it is not possible to automatically create a sensible PATH
490configuration, users are recommended to use the py.exe launcher and
491manually modify their PATH variable to add Scripts directories in their
492preferred order. System-wide installations of Python should consider not
493modifying PATH, or using an alternative technology to modify their
494users' PATH variables.
495
496
497The py.exe launcher is recommended because it uses a consistent and
498sensible search order for Python installations. User installations are
499preferred over system-wide installs, and later versions are preferred
500regardless of installation order (with the exception that py.exe
501currently prefers 2.x versions over 3.x versions without the -3 command
502line argument).
503
504For both 32-bit and 64-bit interpreters, the 32-bit version of the
505launcher is installed. This ensures that the search order is always
506consistent (as the 64-bit launcher is subtly different from the 32-bit
507launcher) and also avoids the need to install it multiple times. Future
508versions of Python will upgrade the launcher in-place, using Windows
509Installer's upgrade functionality to avoid conflicts with earlier
510installed versions.
511
512When installed, file associations are created for .py, .pyc and .pyo
513files to launch with py.exe and .pyw files to launch with pyw.exe. This
514makes Python files respect shebang lines by default and also avoids
515conflicts between multiple Python installations.
516
517
518Repair, Modify and Uninstall
519============================
520
521After installation, Python may be modified, repaired or uninstalled by
522running the original EXE again or via the Programs and Features applet
523(formerly known as Add or Remove Programs).
524
525Modifications allow features to be added or removed. The install
526directory and kind (all users/single user) cannot be modified. Because
527Windows Installer caches installation packages, removing features will
528not require internet access unless the package cache has been corrupted
529or deleted. Adding features that were not previously installed and are
530not embedded or otherwise available will require internet access.
531
532Repairing will rerun the installation for all currently installed
533features, restoring files and registry keys that have been modified or
534removed. This operation generally will not redownload any files unless
535the cached packages have been corrupted or deleted.
536
537Removing Python will clean up all the files and registry keys that were
538created by the installer, as well as __pycache__ folders that are
539explicitly handled by the installer. Python packages installed later
540using a tool like pip will not be removed. Some components may be
541installed by other installers and these will not be removed if another
542product has a dependency on them.
543
544