1:mod:`venv` --- Creation of virtual environments 2================================================ 3 4.. module:: venv 5 :synopsis: Creation of virtual environments. 6 7.. moduleauthor:: Vinay Sajip <vinay_sajip@yahoo.co.uk> 8.. sectionauthor:: Vinay Sajip <vinay_sajip@yahoo.co.uk> 9 10.. versionadded:: 3.3 11 12**Source code:** :source:`Lib/venv/` 13 14.. index:: pair: Environments; virtual 15 16-------------- 17 18The :mod:`venv` module provides support for creating lightweight "virtual 19environments" with their own site directories, optionally isolated from system 20site directories. Each virtual environment has its own Python binary (which 21matches the version of the binary that was used to create this environment) and 22can have its own independent set of installed Python packages in its site 23directories. 24 25See :pep:`405` for more information about Python virtual environments. 26 27.. seealso:: 28 29 `Python Packaging User Guide: Creating and using virtual environments 30 <https://packaging.python.org/installing/#creating-virtual-environments>`__ 31 32 33Creating virtual environments 34----------------------------- 35 36.. include:: /using/venv-create.inc 37 38 39.. _venv-def: 40 41.. note:: A virtual environment is a Python environment such that the Python 42 interpreter, libraries and scripts installed into it are isolated from those 43 installed in other virtual environments, and (by default) any libraries 44 installed in a "system" Python, i.e., one which is installed as part of your 45 operating system. 46 47 A virtual environment is a directory tree which contains Python executable 48 files and other files which indicate that it is a virtual environment. 49 50 Common installation tools such as setuptools_ and pip_ work as 51 expected with virtual environments. In other words, when a virtual 52 environment is active, they install Python packages into the virtual 53 environment without needing to be told to do so explicitly. 54 55 When a virtual environment is active (i.e., the virtual environment's Python 56 interpreter is running), the attributes :attr:`sys.prefix` and 57 :attr:`sys.exec_prefix` point to the base directory of the virtual 58 environment, whereas :attr:`sys.base_prefix` and 59 :attr:`sys.base_exec_prefix` point to the non-virtual environment Python 60 installation which was used to create the virtual environment. If a virtual 61 environment is not active, then :attr:`sys.prefix` is the same as 62 :attr:`sys.base_prefix` and :attr:`sys.exec_prefix` is the same as 63 :attr:`sys.base_exec_prefix` (they all point to a non-virtual environment 64 Python installation). 65 66 When a virtual environment is active, any options that change the 67 installation path will be ignored from all :mod:`distutils` configuration 68 files to prevent projects being inadvertently installed outside of the 69 virtual environment. 70 71 When working in a command shell, users can make a virtual environment active 72 by running an ``activate`` script in the virtual environment's executables 73 directory (the precise filename and command to use the file is 74 shell-dependent), which prepends the virtual environment's directory for 75 executables to the ``PATH`` environment variable for the running shell. There 76 should be no need in other circumstances to activate a virtual 77 environment; scripts installed into virtual environments have a "shebang" 78 line which points to the virtual environment's Python interpreter. This means 79 that the script will run with that interpreter regardless of the value of 80 ``PATH``. On Windows, "shebang" line processing is supported if you have the 81 Python Launcher for Windows installed (this was added to Python in 3.3 - see 82 :pep:`397` for more details). Thus, double-clicking an installed script in a 83 Windows Explorer window should run the script with the correct interpreter 84 without there needing to be any reference to its virtual environment in 85 ``PATH``. 86 87 88.. _venv-api: 89 90API 91--- 92 93.. highlight:: python 94 95The high-level method described above makes use of a simple API which provides 96mechanisms for third-party virtual environment creators to customize environment 97creation according to their needs, the :class:`EnvBuilder` class. 98 99.. class:: EnvBuilder(system_site_packages=False, clear=False, \ 100 symlinks=False, upgrade=False, with_pip=False, \ 101 prompt=None) 102 103 The :class:`EnvBuilder` class accepts the following keyword arguments on 104 instantiation: 105 106 * ``system_site_packages`` -- a Boolean value indicating that the system Python 107 site-packages should be available to the environment (defaults to ``False``). 108 109 * ``clear`` -- a Boolean value which, if true, will delete the contents of 110 any existing target directory, before creating the environment. 111 112 * ``symlinks`` -- a Boolean value indicating whether to attempt to symlink the 113 Python binary rather than copying. 114 115 * ``upgrade`` -- a Boolean value which, if true, will upgrade an existing 116 environment with the running Python - for use when that Python has been 117 upgraded in-place (defaults to ``False``). 118 119 * ``with_pip`` -- a Boolean value which, if true, ensures pip is 120 installed in the virtual environment. This uses :mod:`ensurepip` with 121 the ``--default-pip`` option. 122 123 * ``prompt`` -- a String to be used after virtual environment is activated 124 (defaults to ``None`` which means directory name of the environment would 125 be used). 126 127 .. versionchanged:: 3.4 128 Added the ``with_pip`` parameter 129 130 .. versionadded:: 3.6 131 Added the ``prompt`` parameter 132 133 Creators of third-party virtual environment tools will be free to use the 134 provided :class:`EnvBuilder` class as a base class. 135 136 The returned env-builder is an object which has a method, ``create``: 137 138 .. method:: create(env_dir) 139 140 Create a virtual environment by specifying the target directory 141 (absolute or relative to the current directory) which is to contain the 142 virtual environment. The ``create`` method will either create the 143 environment in the specified directory, or raise an appropriate 144 exception. 145 146 The ``create`` method of the :class:`EnvBuilder` class illustrates the 147 hooks available for subclass customization:: 148 149 def create(self, env_dir): 150 """ 151 Create a virtualized Python environment in a directory. 152 env_dir is the target directory to create an environment in. 153 """ 154 env_dir = os.path.abspath(env_dir) 155 context = self.ensure_directories(env_dir) 156 self.create_configuration(context) 157 self.setup_python(context) 158 self.setup_scripts(context) 159 self.post_setup(context) 160 161 Each of the methods :meth:`ensure_directories`, 162 :meth:`create_configuration`, :meth:`setup_python`, 163 :meth:`setup_scripts` and :meth:`post_setup` can be overridden. 164 165 .. method:: ensure_directories(env_dir) 166 167 Creates the environment directory and all necessary directories, and 168 returns a context object. This is just a holder for attributes (such as 169 paths), for use by the other methods. The directories are allowed to 170 exist already, as long as either ``clear`` or ``upgrade`` were 171 specified to allow operating on an existing environment directory. 172 173 .. method:: create_configuration(context) 174 175 Creates the ``pyvenv.cfg`` configuration file in the environment. 176 177 .. method:: setup_python(context) 178 179 Creates a copy or symlink to the Python executable in the environment. 180 On POSIX systems, if a specific executable ``python3.x`` was used, 181 symlinks to ``python`` and ``python3`` will be created pointing to that 182 executable, unless files with those names already exist. 183 184 .. method:: setup_scripts(context) 185 186 Installs activation scripts appropriate to the platform into the virtual 187 environment. 188 189 .. method:: post_setup(context) 190 191 A placeholder method which can be overridden in third party 192 implementations to pre-install packages in the virtual environment or 193 perform other post-creation steps. 194 195 .. versionchanged:: 3.7.2 196 Windows now uses redirector scripts for ``python[w].exe`` instead of 197 copying the actual binaries. In 3.7.2 only :meth:`setup_python` does 198 nothing unless running from a build in the source tree. 199 200 .. versionchanged:: 3.7.3 201 Windows copies the redirector scripts as part of :meth:`setup_python` 202 instead of :meth:`setup_scripts`. This was not the case in 3.7.2. 203 When using symlinks, the original executables will be linked. 204 205 In addition, :class:`EnvBuilder` provides this utility method that can be 206 called from :meth:`setup_scripts` or :meth:`post_setup` in subclasses to 207 assist in installing custom scripts into the virtual environment. 208 209 .. method:: install_scripts(context, path) 210 211 *path* is the path to a directory that should contain subdirectories 212 "common", "posix", "nt", each containing scripts destined for the bin 213 directory in the environment. The contents of "common" and the 214 directory corresponding to :data:`os.name` are copied after some text 215 replacement of placeholders: 216 217 * ``__VENV_DIR__`` is replaced with the absolute path of the environment 218 directory. 219 220 * ``__VENV_NAME__`` is replaced with the environment name (final path 221 segment of environment directory). 222 223 * ``__VENV_PROMPT__`` is replaced with the prompt (the environment 224 name surrounded by parentheses and with a following space) 225 226 * ``__VENV_BIN_NAME__`` is replaced with the name of the bin directory 227 (either ``bin`` or ``Scripts``). 228 229 * ``__VENV_PYTHON__`` is replaced with the absolute path of the 230 environment's executable. 231 232 The directories are allowed to exist (for when an existing environment 233 is being upgraded). 234 235There is also a module-level convenience function: 236 237.. function:: create(env_dir, system_site_packages=False, clear=False, \ 238 symlinks=False, with_pip=False, prompt=None) 239 240 Create an :class:`EnvBuilder` with the given keyword arguments, and call its 241 :meth:`~EnvBuilder.create` method with the *env_dir* argument. 242 243 .. versionadded:: 3.3 244 245 .. versionchanged:: 3.4 246 Added the ``with_pip`` parameter 247 248 .. versionchanged:: 3.6 249 Added the ``prompt`` parameter 250 251An example of extending ``EnvBuilder`` 252-------------------------------------- 253 254The following script shows how to extend :class:`EnvBuilder` by implementing a 255subclass which installs setuptools and pip into a created virtual environment:: 256 257 import os 258 import os.path 259 from subprocess import Popen, PIPE 260 import sys 261 from threading import Thread 262 from urllib.parse import urlparse 263 from urllib.request import urlretrieve 264 import venv 265 266 class ExtendedEnvBuilder(venv.EnvBuilder): 267 """ 268 This builder installs setuptools and pip so that you can pip or 269 easy_install other packages into the created virtual environment. 270 271 :param nodist: If true, setuptools and pip are not installed into the 272 created virtual environment. 273 :param nopip: If true, pip is not installed into the created 274 virtual environment. 275 :param progress: If setuptools or pip are installed, the progress of the 276 installation can be monitored by passing a progress 277 callable. If specified, it is called with two 278 arguments: a string indicating some progress, and a 279 context indicating where the string is coming from. 280 The context argument can have one of three values: 281 'main', indicating that it is called from virtualize() 282 itself, and 'stdout' and 'stderr', which are obtained 283 by reading lines from the output streams of a subprocess 284 which is used to install the app. 285 286 If a callable is not specified, default progress 287 information is output to sys.stderr. 288 """ 289 290 def __init__(self, *args, **kwargs): 291 self.nodist = kwargs.pop('nodist', False) 292 self.nopip = kwargs.pop('nopip', False) 293 self.progress = kwargs.pop('progress', None) 294 self.verbose = kwargs.pop('verbose', False) 295 super().__init__(*args, **kwargs) 296 297 def post_setup(self, context): 298 """ 299 Set up any packages which need to be pre-installed into the 300 virtual environment being created. 301 302 :param context: The information for the virtual environment 303 creation request being processed. 304 """ 305 os.environ['VIRTUAL_ENV'] = context.env_dir 306 if not self.nodist: 307 self.install_setuptools(context) 308 # Can't install pip without setuptools 309 if not self.nopip and not self.nodist: 310 self.install_pip(context) 311 312 def reader(self, stream, context): 313 """ 314 Read lines from a subprocess' output stream and either pass to a progress 315 callable (if specified) or write progress information to sys.stderr. 316 """ 317 progress = self.progress 318 while True: 319 s = stream.readline() 320 if not s: 321 break 322 if progress is not None: 323 progress(s, context) 324 else: 325 if not self.verbose: 326 sys.stderr.write('.') 327 else: 328 sys.stderr.write(s.decode('utf-8')) 329 sys.stderr.flush() 330 stream.close() 331 332 def install_script(self, context, name, url): 333 _, _, path, _, _, _ = urlparse(url) 334 fn = os.path.split(path)[-1] 335 binpath = context.bin_path 336 distpath = os.path.join(binpath, fn) 337 # Download script into the virtual environment's binaries folder 338 urlretrieve(url, distpath) 339 progress = self.progress 340 if self.verbose: 341 term = '\n' 342 else: 343 term = '' 344 if progress is not None: 345 progress('Installing %s ...%s' % (name, term), 'main') 346 else: 347 sys.stderr.write('Installing %s ...%s' % (name, term)) 348 sys.stderr.flush() 349 # Install in the virtual environment 350 args = [context.env_exe, fn] 351 p = Popen(args, stdout=PIPE, stderr=PIPE, cwd=binpath) 352 t1 = Thread(target=self.reader, args=(p.stdout, 'stdout')) 353 t1.start() 354 t2 = Thread(target=self.reader, args=(p.stderr, 'stderr')) 355 t2.start() 356 p.wait() 357 t1.join() 358 t2.join() 359 if progress is not None: 360 progress('done.', 'main') 361 else: 362 sys.stderr.write('done.\n') 363 # Clean up - no longer needed 364 os.unlink(distpath) 365 366 def install_setuptools(self, context): 367 """ 368 Install setuptools in the virtual environment. 369 370 :param context: The information for the virtual environment 371 creation request being processed. 372 """ 373 url = 'https://bitbucket.org/pypa/setuptools/downloads/ez_setup.py' 374 self.install_script(context, 'setuptools', url) 375 # clear up the setuptools archive which gets downloaded 376 pred = lambda o: o.startswith('setuptools-') and o.endswith('.tar.gz') 377 files = filter(pred, os.listdir(context.bin_path)) 378 for f in files: 379 f = os.path.join(context.bin_path, f) 380 os.unlink(f) 381 382 def install_pip(self, context): 383 """ 384 Install pip in the virtual environment. 385 386 :param context: The information for the virtual environment 387 creation request being processed. 388 """ 389 url = 'https://raw.github.com/pypa/pip/master/contrib/get-pip.py' 390 self.install_script(context, 'pip', url) 391 392 def main(args=None): 393 compatible = True 394 if sys.version_info < (3, 3): 395 compatible = False 396 elif not hasattr(sys, 'base_prefix'): 397 compatible = False 398 if not compatible: 399 raise ValueError('This script is only for use with ' 400 'Python 3.3 or later') 401 else: 402 import argparse 403 404 parser = argparse.ArgumentParser(prog=__name__, 405 description='Creates virtual Python ' 406 'environments in one or ' 407 'more target ' 408 'directories.') 409 parser.add_argument('dirs', metavar='ENV_DIR', nargs='+', 410 help='A directory in which to create the 411 'virtual environment.') 412 parser.add_argument('--no-setuptools', default=False, 413 action='store_true', dest='nodist', 414 help="Don't install setuptools or pip in the " 415 "virtual environment.") 416 parser.add_argument('--no-pip', default=False, 417 action='store_true', dest='nopip', 418 help="Don't install pip in the virtual " 419 "environment.") 420 parser.add_argument('--system-site-packages', default=False, 421 action='store_true', dest='system_site', 422 help='Give the virtual environment access to the ' 423 'system site-packages dir.') 424 if os.name == 'nt': 425 use_symlinks = False 426 else: 427 use_symlinks = True 428 parser.add_argument('--symlinks', default=use_symlinks, 429 action='store_true', dest='symlinks', 430 help='Try to use symlinks rather than copies, ' 431 'when symlinks are not the default for ' 432 'the platform.') 433 parser.add_argument('--clear', default=False, action='store_true', 434 dest='clear', help='Delete the contents of the ' 435 'virtual environment ' 436 'directory if it already ' 437 'exists, before virtual ' 438 'environment creation.') 439 parser.add_argument('--upgrade', default=False, action='store_true', 440 dest='upgrade', help='Upgrade the virtual ' 441 'environment directory to ' 442 'use this version of ' 443 'Python, assuming Python ' 444 'has been upgraded ' 445 'in-place.') 446 parser.add_argument('--verbose', default=False, action='store_true', 447 dest='verbose', help='Display the output ' 448 'from the scripts which ' 449 'install setuptools and pip.') 450 options = parser.parse_args(args) 451 if options.upgrade and options.clear: 452 raise ValueError('you cannot supply --upgrade and --clear together.') 453 builder = ExtendedEnvBuilder(system_site_packages=options.system_site, 454 clear=options.clear, 455 symlinks=options.symlinks, 456 upgrade=options.upgrade, 457 nodist=options.nodist, 458 nopip=options.nopip, 459 verbose=options.verbose) 460 for d in options.dirs: 461 builder.create(d) 462 463 if __name__ == '__main__': 464 rc = 1 465 try: 466 main() 467 rc = 0 468 except Exception as e: 469 print('Error: %s' % e, file=sys.stderr) 470 sys.exit(rc) 471 472 473This script is also available for download `online 474<https://gist.github.com/vsajip/4673395>`_. 475 476 477.. _setuptools: https://pypi.org/project/setuptools/ 478.. _pip: https://pypi.org/project/pip/ 479