1.. _module-pw_env_setup: 2 3============ 4pw_env_setup 5============ 6.. pigweed-module:: 7 :name: pw_env_setup 8 9A classic problem in the embedded space is reducing the time from git clone 10to having a binary executing on a device. The issue is that an entire suite 11of tools is needed for non-trivial production embedded projects. For example: 12 13- A C++ compiler for your target device, and also for your host 14- A build system or three; for example, GN, Ninja, CMake, Bazel 15- A code formatting program like clang-format 16- A debugger like OpenOCD to flash and debug your embedded device (OpenOCD 17 support removed for Windows) 18- A known Python version with known modules installed for scripting 19- A Go compiler for the Go-based command line tools 20 21...and so on 22 23In the server space, container solutions like Docker or Podman solve this; 24however, in our experience container solutions are a mixed bag for embedded 25systems development where one frequently needs access to native system 26resources like USB devices, or must operate on Windows. 27 28``pw_env_setup`` is our compromise solution for this problem that works on Mac, 29Windows, and Linux. It leverages the Chrome packaging system `CIPD`_ to 30bootstrap a Python installation, which in turn inflates a virtual 31environment. The tooling is installed into your workspace, and makes no 32changes to your system. This tooling is designed to be reused by any 33project. 34 35.. _CIPD: https://github.com/luci/luci-go/tree/HEAD/cipd 36 37Users interact with ``pw_env_setup`` with two commands: ``. bootstrap.sh`` and 38``. activate.sh``. The bootstrap command always pulls down the current versions 39of CIPD packages and sets up the Python virtual environment. The activate 40command reinitializes a previously configured environment, and if none is found, 41runs bootstrap. 42 43.. note:: 44 45 On Windows the scripts used to set up the environment are ``bootstrap.bat`` 46 and ``activate.bat``. 47 48 ``bootstrap.fish`` and ``activate.fish`` are also available for `Fish shell 49 <https://fishshell.com/>`_ users. 50 51 For simplicity they will be referred to with the ``.sh`` endings unless the 52 distinction is relevant. 53 54On POSIX systems, the environment can be deactivated by running ``deactivate``. 55 56---------------------------------- 57Using pw_env_setup in your project 58---------------------------------- 59 60Downstream Projects Using Pigweed's Packages 61============================================ 62 63Projects using Pigweed can leverage ``pw_env_setup`` to install Pigweed's 64dependencies or their own dependencies. Projects that only want to use Pigweed's 65dependencies without modifying them can just source Pigweed's ``bootstrap.sh`` 66and ``activate.sh`` scripts. 67 68An example of what your project's `bootstrap.sh` could look like is below. This 69assumes `bootstrap.sh` is at the top level of your repository. 70 71.. code-block:: bash 72 73 # Do not include a "#!" line, this must be sourced and not executed. 74 75 # This assumes the user is sourcing this file from it's parent directory. See 76 # below for a more flexible way to handle this. 77 PROJ_SETUP_SCRIPT_PATH="$(pwd)/bootstrap.sh" 78 79 export PW_PROJECT_ROOT="$(_python_abspath "$(dirname "$PROJ_SETUP_SCRIPT_PATH")")" 80 81 # You may wish to check if the user is attempting to execute this script 82 # instead of sourcing it. See below for an example of how to handle that 83 # situation. 84 85 # Source Pigweed's bootstrap utility script. 86 # Using '.' instead of 'source' for POSIX compatibility. Since users don't use 87 # dash directly, using 'source' in most documentation so users don't get 88 # confused and try to `./bootstrap.sh`. 89 . "$PW_PROJECT_ROOT/third_party/pigweed/pw_env_setup/util.sh" 90 91 pw_check_root "$PW_ROOT" 92 _PW_ACTUAL_ENVIRONMENT_ROOT="$(pw_get_env_root)" 93 export _PW_ACTUAL_ENVIRONMENT_ROOT 94 SETUP_SH="$_PW_ACTUAL_ENVIRONMENT_ROOT/activate.sh" 95 pw_bootstrap --args... # See below for details about args. 96 pw_finalize bootstrap "$SETUP_SH" 97 98 99Bazel Usage 100----------- 101It is possible to pull in a CIPD dependency into Bazel using WORKSPACE rules 102rather than using `bootstrap.sh`. e.g. 103 104.. code-block:: python 105 106 # WORKSPACE 107 108 load("//pw_env_setup/bazel/cipd_setup:cipd_rules.bzl", "pigweed_deps") 109 110 # Setup CIPD client and packages. 111 # Required by: pigweed. 112 # Used by modules: all. 113 pigweed_deps() 114 115 load("@cipd_deps//:cipd_init.bzl", "cipd_init") 116 117 cipd_init() 118 119 120This will make the entire set of Pigweeds remote repositories available to your 121project. Though these repositories will only be donwloaded if you use them. To 122get a full list of the remote repositories that this configures, run: 123 124.. code-block:: console 125 126 $ bazel query //external:all | grep cipd_ 127 128All files and executables in each CIPD remote repository is exported and visible 129either directely (`@cipd_<dep>//:<file>`) or from 'all' filegroup 130(`@cipd_<dep>//:all`). 131 132From here it is possible to get access to the Bloaty binaries using the 133following command. For example; 134 135.. code-block:: console 136 137 $ bazel run @cipd_pigweed_third_party_bloaty_embedded_linux_amd64//:bloaty \ 138 > -- --help 139 140User-Friendliness 141----------------- 142 143You may wish to allow sourcing `bootstrap.sh` from a different directory. In 144that case you'll need the following at the top of `bootstrap.sh`. 145 146.. code-block:: bash 147 148 _python_abspath () { 149 python -c "import os.path; print(os.path.abspath('$@'))" 150 } 151 152 # Use this code from Pigweed's bootstrap to find the path to this script when 153 # sourced. This should work with common shells. PW_CHECKOUT_ROOT is only used in 154 # presubmit tests with strange setups, and can be omitted if you're not using 155 # Pigweed's automated testing infrastructure. 156 if test -n "$PW_CHECKOUT_ROOT"; then 157 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$PW_CHECKOUT_ROOT/bootstrap.sh")" 158 unset PW_CHECKOUT_ROOT 159 # Shell: bash. 160 elif test -n "$BASH"; then 161 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$BASH_SOURCE")" 162 # Shell: zsh. 163 elif test -n "$ZSH_NAME"; then 164 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "${(%):-%N}")" 165 # Shell: dash. 166 elif test ${0##*/} = dash; then 167 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath \ 168 "$(lsof -p $$ -Fn0 | tail -1 | sed 's#^[^/]*##;')")" 169 # If everything else fails, try $0. It could work. 170 else 171 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$0")" 172 fi 173 174You may also wish to check if the user is attempting to execute `bootstrap.sh` 175instead of sourcing it. Executing `bootstrap.sh` would download everything 176required for the environment, but cannot modify the environment of the parent 177process. To check for this add the following. 178 179.. code-block:: bash 180 181 # Check if this file is being executed or sourced. 182 _pw_sourced=0 183 # If not running in Pigweed's automated testing infrastructure the 184 # SWARMING_BOT_ID check is unnecessary. 185 if [ -n "$SWARMING_BOT_ID" ]; then 186 # If set we're running on swarming and don't need this check. 187 _pw_sourced=1 188 elif [ -n "$ZSH_EVAL_CONTEXT" ]; then 189 case $ZSH_EVAL_CONTEXT in *:file) _pw_sourced=1;; esac 190 elif [ -n "$KSH_VERSION" ]; then 191 [ "$(cd $(dirname -- $0) && pwd -P)/$(basename -- $0)" != \ 192 "$(cd $(dirname -- ${.sh.file}) && pwd -P)/$(basename -- ${.sh.file})" ] \ 193 && _pw_sourced=1 194 elif [ -n "$BASH_VERSION" ]; then 195 (return 0 2>/dev/null) && _pw_sourced=1 196 else # All other shells: examine $0 for known shell binary filenames 197 # Detects `sh` and `dash`; add additional shell filenames as needed. 198 case ${0##*/} in sh|dash) _pw_sourced=1;; esac 199 fi 200 201 _pw_eval_sourced "$_pw_sourced" 202 203Downstream Projects Using Different Packages 204============================================ 205Projects depending on Pigweed but using additional or different packages should 206copy the Pigweed `sample project`'s ``bootstrap.sh`` and ``pigweed.json`` and 207update the call to ``pw_bootstrap``. Search for "downstream" for other places 208that may require changes, like setting the ``PW_ROOT`` and ``PW_PROJECT_ROOT`` 209environment variables. Explanations of parts of ``pigweed.json`` are described 210here. 211 212.. _sample project: https://pigweed.googlesource.com/pigweed/sample_project/+/HEAD 213 214``pw.pw_env_setup.root_variable`` 215 Variable used to point to the root of the source tree. Optional, can always 216 use ``PW_PROJECT_ROOT`` instead. (That variable will be set regardless of 217 whether this is provided.) 218 219``pw.pw_env_setup.relative_pigweed_root`` 220 Location of the Pigweed submodule within the source tree. Optional—environment 221 setup will work correctly without this. If present, will confirm that it's 222 correct. May be used by other tooling. 223 224``pw.pw_env_setup.cipd_package_files`` 225 CIPD package file. JSON file consisting of a list of additional CIPD package 226 files to import and a list of dictionaries with "path", "platforms", "subdir", 227 "tags", and "version_file" keys. Both top-level lists are optional. An 228 example is below. Only "path", "platforms", and "tags" are required. If 229 "version_file" is specified then ``pw doctor`` will fail if that version file 230 is not present. If "subdir" is specified then this packages will be installed 231 in a subdirectory of the directory created for packages in this file. 232 233.. code-block:: json 234 235 { 236 "included_files": [ 237 "foo.json" 238 ], 239 "packages": [ 240 { 241 "path": "infra/3pp/tools/go/${platform}", 242 "platforms": [ 243 "linux-amd64", 244 "linux-arm64", 245 "mac-amd64", 246 "windows-amd64" 247 ], 248 "subdir": "pa/th", 249 "tags": [ 250 "version:2@1.16.3" 251 ], 252 "version_file": ".versions/go.cipd_version" 253 } 254 ] 255 } 256 257``pw.pw_env_setup.project_actions`` 258 A list of plugins to load and run after CIPD setup, but prior to virtualenv 259 setup, for e.g. downloading project-specific tools or artifacts needed by 260 later steps. Particularly useful for downstream projects with limited CIPD 261 access. 262 263 A plugin is specified as a dictionary with two keys: "import_path" and 264 "module_name". The "import_path" is relative to the root of the checkout. 265 266 The specified module must provide a "run_actions" method which takes a single 267 argument, "env_vars", which is a pw_env_setup.Environment instance. 268 269 Sample plugin and pigweed.json blob: 270 271.. code-block:: python 272 273 """Sample pw_env_setup project action plugin. 274 275 A sample/starter project action plugin template for pw_env_setup. 276 """ 277 def run_action(**kwargs): 278 """Sample project action.""" 279 if "env" not in kwargs: 280 raise ValueError(f"Missing required kwarg 'env', got %{kwargs}") 281 282 kwargs["env"].prepend("PATH", "PATH_TO_NEW_TOOLS") 283 raise NotImplementedError("Sample project action running!") 284 285.. code-block:: json 286 287 "project_actions" : [ 288 { 289 "import_path": "pw_env_setup", 290 "module_name": "sample_project_action" 291 } 292 ], 293 294``pw.pw_env_setup.virtualenv.gn_args`` 295 Any necessary GN args to be used when installing Python packages. 296 297``pw.pw_env_setup.virtualenv.gn_targets`` 298 Target for installing Python packages. Downstream projects will need to 299 create targets to install their packages or only use Pigweed Python packages. 300 301``pw.pw_env_setup.virtualenv.gn_root`` 302 The root directory of your GN build tree, relative to ``PW_PROJECT_ROOT``. 303 This is the directory your project's ``.gn`` file is located in. If you're 304 only installing Pigweed Python packages, use the location of the Pigweed 305 submodule. 306 307``pw.pw_env_setup.virtualenv.requirements`` 308 A list of Python Pip requirements files for installing into the Pigweed 309 virtualenv. Each file will be passed as additional ``--requirement`` argument 310 to a single ```pip install`` at the beginning of bootstrap's ``Python 311 environment`` setup stage. See the `Requirements Files documentation`_ for 312 details on what can be specified using requirements files. 313 314``pw.pw_env_setup.virtualenv.constraints`` 315 A list of Python Pip constraints files. These constraints will be passed to 316 every ``pip`` invocation as an additional ``--constraint`` argument during 317 bootstrap. virtualenv. See the `Constraints Files documentation`_ for details 318 on formatting. 319 320``pw.pw_env_setup.virtualenv.system_packages`` 321 A boolean value that can be used the give the Python virtual environment 322 access to the system site packages. Defaults to ``false``. 323 324``pw.pw_env_setup.virtualenv.pip_install_offline`` 325 A boolean value that adds ``--no-index`` to all ``pip install`` commands that 326 are part of bootstrap. This forces pip to not reach out to the internet 327 (usually `pypi.org <https://pypi.org/>`_) to download packages. Using this 328 option requires setting 329 ``pw.pw_env_setup.virtualenv.pip_install_find_links``. Defaults to 330 ``false``. 331 332 .. seealso:: 333 The Python GN guide for offline pip installation: 334 :ref:`docs-python-build-installing-offline` 335 336``pw.pw_env_setup.virtualenv.pip_install_find_links`` 337 List of paths to folders containing Python wheels (``*.whl``) or source tar 338 files (``*.tar.gz``). Pip will check each of these directories when looking 339 for potential install candidates. Each path will be passed to all ``pip 340 install`` commands as ``--find-links PATH``. 341 342 .. tip:: 343 Environment variables may be used in these paths. For example: 344 345 .. code-block:: json 346 347 "virtualenv": { 348 "pip_install_find_links": [ 349 "${PW_PROJECT_ROOT}/pip_cache" 350 ] 351 } 352 353``pw.pw_env_setup.virtualenv.pip_install_require_hashes`` 354 Adds ``--require-hashes`` This option enforces hash checking on Python 355 package files. Defaults to ``false``. 356 357``pw.pw_env_setup.virtualenv.pip_install_disable_cache`` 358 A boolean value that adds ``--no-cache-dir`` to all ``pip install`` commands 359 that are part of bootstrap. This forces pip to ignore any previously cached 360 Python packages. On most systems this is located in 361 ``~/.cache/pip/``. Defaults to ``false``. 362 363``pw.pw_env_setup.optional_submodules`` 364 By default environment setup will check that all submodules are present in 365 the checkout. Any submodules in this list are excluded from that check. 366 367``pw.pw_env_setup.required_submodules`` 368 If this is specified instead of ``optional_submodules`` bootstrap will only 369 complain if one of the required submodules is not present. Combining this 370 with ``optional_submodules`` is not supported. 371 372``pw.pw_env_setup.pw_packages`` 373 A list of packages to install using :ref:`pw_package <module-pw_package>` 374 after the rest of bootstrap completes. 375 376``pw.pw_env_setup.gni_file`` 377 Location to write a ``.gni`` file containing paths to many things within the 378 environment directory. Defaults to 379 ``build_overrides/pigweed_environment.gni``. 380 381``pw.pw_env_setup.json_file`` 382 Location to write a ``.json`` file containing step-by-step modifications to 383 the environment, for reading by tools that don't inherit an environment from 384 a sourced ``bootstrap.sh``. 385 386``pw.pw_env_setup.rosetta`` 387 Whether to use Rosetta to use amd64 packages on arm64 Macs. Accepted values 388 are ``never``, ``allow``, and ``force``. For now, ``allow`` means ``force``. 389 At some point in the future ``allow`` will be changed to mean ``never``. 390 391An example of a config file is below. 392 393.. code-block:: json 394 395 { 396 "pw": { 397 "pw_env_setup": { 398 "root_variable": "EXAMPLE_ROOT", 399 "cipd_package_files": [ 400 "pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/pigweed.json", 401 "pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/luci.json" 402 "tools/myprojectname.json" 403 ], 404 "virtualenv": { 405 "gn_root": ".", 406 "gn_targets": [ 407 ":python.install", 408 ], 409 "system_packages": false 410 }, 411 "pw_packages": [], 412 "optional_submodules": [ 413 "optional/submodule/one", 414 "optional/submodule/two" 415 ], 416 "gni_file": "tools/environment.gni", 417 "json_file": "tools/environment.json", 418 "rosetta": "allow" 419 } 420 } 421 } 422 423Only the packages necessary for almost all projects based on Pigweed are 424included in the ``cipd_setup/pigweed.json`` file. A number of other files are 425present in that directory for projects that need more than the minimum. 426Internal-Google projects using LUCI should at least include ``luci.json``. 427 428In case the CIPD packages need to be referenced from other scripts, variables 429like ``PW_${BASENAME}_CIPD_INSTALL_DIR`` point to the CIPD install directories, 430where ``${BASENAME}`` is ``"PIGWEED"`` for 431``"pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/pigweed.json"`` and 432``"LUCI"`` for 433``"pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/luci.json"``. This example 434would set the following environment variables. 435 436- ``PW_LUCI_CIPD_INSTALL_DIR`` 437- ``PW_MYPROJECTNAME_CIPD_INSTALL_DIR`` 438- ``PW_PIGWEED_CIPD_INSTALL_DIR`` 439 440These directories are also referenced in the gni_file specified by the 441environment config file as ``dir_cipd_${BASENAME}``. This allows the GN build to 442reliably reference these directories without using GN ``getenv()`` calls or 443hardcoding paths. 444 445In addition, ``PW_${BASENAME}_CIPD_INSTALL_DIR`` and 446``PW_${BASENAME}_CIPD_INSTALL_DIR/bin`` are both added to ``PATH`` for each 447package directory. 448 449If multiple packages install executables with the same name, the file mentioned 450last topologically takes priority. For example, with the file contents below, 451``d.json``'s entries will appear in ``PATH`` before ``c.json``'s, which will 452appear before ``b.json``'s, which will appear before ``a.json``'s. 453 454.. code-block:: json 455 :caption: :octicon:`file;1em` pigweed.json 456 457 { 458 "pw": { 459 "pw_env_setup": { 460 "cipd_package_files": [ 461 "a.json", 462 "b.json", 463 "d.json" 464 ] 465 } 466 } 467 } 468 469.. code-block:: json 470 :caption: :octicon:`file;1em` a.json 471 472 { 473 "package_files": [ 474 // ... 475 ] 476 } 477 478.. code-block:: json 479 :caption: :octicon:`file;1em` b.json 480 481 { 482 "included_files": ["c.json"], 483 "package_files": [ 484 // ... 485 ] 486 } 487 488.. code-block:: json 489 :caption: :octicon:`file;1em` c.json 490 491 { 492 "package_files": [ 493 // ... 494 ] 495 } 496 497.. code-block:: json 498 :caption: :octicon:`file;1em` d.json 499 500 { 501 "package_files": [ 502 // ... 503 ] 504 } 505 506.. code-block:: 507 :caption: Effective File Loading Order 508 509 pigweed.json 510 a.json 511 b.json 512 c.json 513 d.json 514 515Pinning Python Packages 516======================= 517Python modules usually express dependencies as ranges, which makes it easier to 518install many Python packages that might otherwise have conflicting dependencies. 519However, this means version of packages can often change underneath us and 520builds will not be hermetic. 521 522To ensure versions don't change without approval, Pigweed by default pins the 523versions of packages it depends on using a `pip constraints file`_. To pin the 524versions of additional packages your project depends on, run 525``pw python-packages list <path/to/constraints/file>`` and then add 526``pw_build_PIP_CONSTRAINTS = ["//path/to/constraints/file"]`` to your project's 527``.gn`` file (see `Pigweed's .gn file`_ for an example). 528 529.. _pip constraints file: https://pip.pypa.io/en/stable/user_guide/#constraints-files 530.. _default constraints: https://cs.pigweed.dev/pigweed/+/main:pw_env_setup/py/pw_env_setup/virtualenv_setup/constraint.list 531.. _Pigweed's .gn file: https://cs.pigweed.dev/pigweed/+/main:.gn 532 533To update packages, set ``pw_build_PIP_CONSTRAINTS = []``, delete the 534environment, and bootstrap again. Then run the ``list`` command from above 535again, and run ``pw presubmit``. 536 537Environment Variables 538===================== 539Input Variables 540--------------- 541The following environment variables affect env setup behavior. Most users will 542never need to set these. 543 544``CIPD_CACHE_DIR`` 545 Location of CIPD cache dir. Read by CIPD, but if unset will be defaulted to 546 ``$HOME/.cipd-cache-dir``. 547 548``PW_NO_CIPD_CACHE_DIR`` 549 Disables the CIPD cache. 550 551``PW_ACTIVATE_SKIP_CHECKS`` 552 If set, skip running ``pw doctor`` at end of bootstrap/activate. Intended to 553 be used by automated tools but not interactively. 554 555``PW_BANNER_FUNC`` 556 Command to print a banner at the beginning of bootstrap. 557 558``PW_BOOTSTRAP_PYTHON`` 559 Python executable to be used, for example "python3". Defaults to 560 "python3" if that's in ``PATH``, then tries "python". 561 562``PW_CIPD_SERVICE_ACCOUNT_JSON`` 563 Value to pass as ``-service-account-json`` to CIPD invocations. This should 564 point either to a service account JSON key file, or be the magical value 565 ``:gce`` to tell the tool to fetch tokens from GCE metadata server. 566 567``PW_ENVIRONMENT_ROOT`` 568 Location to which packages are installed. Defaults to ``environment`` folder 569 within the checkout root. This variable is cleared after environment setup is 570 complete. 571 572``PW_ENVSETUP_DISABLE_SPINNER`` 573 Disable the spinner during env setup. Intended to be used when the output is 574 being redirected to a log. 575 576``PW_ENVSETUP_DISABLE_SPINNER`` 577 Disable the console spinner that runs when waiting for env setup steps to 578 complete. 579 580``PW_ENVSETUP_NO_BANNER`` 581 Skip printing the banner. 582 583``PW_ENVSETUP_QUIET`` 584 Disables all non-error output. 585 586``PW_PROJECT_ROOT`` 587 The absolute path of the project using Pigweed's env setup. For Pigweed this 588 is the same as ``PW_ROOT``. This should be set by the project's bootstrap 589 script. 590 591``PW_ROOT`` 592 The absolute path to the Pigweed repository within ``PW_PROJECT_ROOT``. This 593 should be set by the project's bootstrap script. 594 595Output Variables 596---------------- 597The following environment variables are set by env setup. 598 599``PATH`` 600 System executable search path. Many of the environment variables below are 601 also added to this variable. 602 603``_PW_ACTUAL_ENVIRONMENT_ROOT`` 604 Location the environment was installed into. Separate from 605 ``PW_ENVIRONMENT_ROOT`` because setting that implicitly and switching to 606 another project directory causes unexpected behavior. 607 608``PW_CIPD_INSTALL_DIR`` 609 Top-level CIPD install directory. This is where the ``cipd`` executable is. 610 611``PW_*_CIPD_INSTALL_DIR`` 612 Each CIPD package file is installed into its own directory. This allows other 613 tools to determine what those directories are. The ``*`` is replaced with an 614 all-caps version of the basename of the package file, without the extension. 615 (E.g., "path/foo.json" becomes ``PW_FOO_CIPD_INSTALL_DIR``.) 616 617``PW_PACKAGE_ROOT`` 618 Location that packages installed by ``pw package`` will be installed to. 619 620``VIRTUAL_ENV`` 621 Path to Pigweed's virtualenv. 622 623Non-Shell Environments 624====================== 625If using this outside of bash—for example directly from an IDE or CI 626system—users can process the ``actions.json`` file that's generated in the 627location specified by the environment config. It lists variables to set, clear, 628and modify. An example ``actions.json`` is shown below. The "append" and 629"prepend" actions are listed in the order they should be applied, so the 630``<pigweed-root>/out/host/host_tools`` entry should be at the beginning of 631``PATH`` and not in the middle somewhere. 632 633.. code-block:: json 634 635 { 636 "modify": { 637 "PATH": { 638 "append": [], 639 "prepend": [ 640 "<pigweed-root>/environment/cipd", 641 "<pigweed-root>/environment/cipd/pigweed", 642 "<pigweed-root>/environment/cipd/pigweed/bin", 643 "<pigweed-root>/environment/cipd/luci", 644 "<pigweed-root>/environment/cipd/luci/bin", 645 "<pigweed-root>/environment/pigweed-venv/bin", 646 "<pigweed-root>/out/host/host_tools" 647 ], 648 "remove": [] 649 } 650 }, 651 "set": { 652 "PW_PROJECT_ROOT": "<pigweed-root>", 653 "PW_ROOT": "<pigweed-root>", 654 "_PW_ACTUAL_ENVIRONMENT_ROOT": "<pigweed-root>/environment", 655 "PW_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd", 656 "CIPD_CACHE_DIR": "<home>/.cipd-cache-dir", 657 "PW_PIGWEED_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/pigweed", 658 "PW_LUCI_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/luci", 659 "VIRTUAL_ENV": "<pigweed-root>/environment/pigweed-venv", 660 "PYTHONHOME": null, 661 "__PYVENV_LAUNCHER__": null 662 } 663 } 664 665Many of these variables are directly exposed to the GN build as well, through 666the GNI file specified in the environment config file. 667 668.. code-block:: 669 670 declare_args() { 671 pw_env_setup_CIPD_LUCI = "<environment-root>/cipd/packages/luci" 672 pw_env_setup_CIPD_PIGWEED = "<environment-root>/cipd/packages/pigweed" 673 pw_env_setup_PACKAGE_ROOT = "<environment-root>/packages" 674 pw_env_setup_VIRTUAL_ENV = "<environment-root>/pigweed-venv" 675 } 676 677It's straightforward to use these variables. 678 679.. code-block:: cpp 680 681 import("//build_overrides/pigweed_environment.gni") 682 683 deps = [ "$pw_env_setup_CIPD_PIGWEED/..." ] 684 685Implementation 686============== 687The environment is set up by installing CIPD and Python packages in 688``PW_ENVIRONMENT_ROOT`` or ``<checkout>/environment``, and saving modifications 689to environment variables in setup scripts in those directories. To support 690multiple operating systems this is done in an operating system-agnostic manner 691and then written into operating system-specific files to be sourced now and in 692the future when running ``activate.sh`` instead of ``bootstrap.sh``. In the 693future these could be extended to C shell and PowerShell. A logical mapping of 694high-level commands to system-specific initialization files is shown below. 695 696.. grid:: 1 697 :padding: 0 698 699 .. grid-item-card:: 700 :columns: 12 701 :class-header: font-monospace 702 703 SET $PW_ROOT /home/$USER/pigweed 704 ^^^ 705 706 .. grid:: 2 707 :margin: 0 708 :padding: 0 709 710 .. grid-item:: **Windows** 711 712 .. grid-item:: **Linux & Mac (sh-compatible shells)** 713 714 .. grid:: 2 715 :margin: 0 716 :padding: 0 717 718 .. grid-item:: 719 720 .. code-block:: dosbatch 721 722 set PW_ROOT /home/%USER%/pigweed 723 724 .. grid-item:: 725 726 .. code-block:: shell 727 728 PW_ROOT="/home/$USER/pigweed" 729 export PW_ROOT 730 731.. grid:: 1 732 :padding: 0 733 734 .. grid-item-card:: 735 :columns: 12 736 :class-header: font-monospace 737 738 PREPEND $PATH $PW_ROOT/.env/bin 739 ^^^ 740 .. grid:: 2 741 :margin: 0 742 :padding: 0 743 744 .. grid-item:: **Windows** 745 746 .. grid-item:: **Linux & Mac (sh-compatible shells)** 747 748 .. grid:: 2 749 :margin: 0 750 :padding: 0 751 752 .. grid-item:: 753 754 .. code-block:: dosbatch 755 756 set PATH=%PW_ROOT%/.env/bin;%PATH% 757 758 .. grid-item:: 759 760 .. code-block:: shell 761 762 PATH="$(\ 763 echo "$PATH" | \ 764 sed "s|:$PW_ROOT/.env/bin:|:|g;" | \ 765 sed "s|^$PW_ROOT/.env/bin:||g;" | \ 766 sed "s|:$PW_ROOT/.env/bin$||g;")" 767 PATH="$PW_ROOT/.env/bin;$PATH" 768 export PATH 769 770.. grid:: 1 771 :padding: 0 772 773 .. grid-item-card:: 774 :columns: 12 775 :class-header: font-monospace 776 777 ECHO "Setup Complete!" 778 ^^^ 779 780 .. grid:: 2 781 :margin: 0 782 :padding: 0 783 784 .. grid-item:: **Windows** 785 786 .. grid-item:: **Linux & Mac (sh-compatible shells)** 787 788 789 .. grid:: 2 790 :margin: 0 791 :padding: 0 792 793 .. grid-item:: 794 795 .. code-block:: doscon 796 797 > echo Setup Complete! 798 799 .. grid-item:: 800 801 .. code-block:: console 802 803 $ echo "Setup Complete!" 804 805 806.. _Requirements Files documentation: https://pip.pypa.io/en/stable/user_guide/#requirements-files 807.. _Constraints Files documentation: https://pip.pypa.io/en/stable/user_guide/#constraints-files 808