1.. _module-pw_env_setup: 2 3------------ 4pw_env_setup 5------------ 6A classic problem in the embedded space is reducing the time from git clone 7to having a binary executing on a device. The issue is that an entire suite 8of tools is needed for non-trivial production embedded projects. For example: 9 10- A C++ compiler for your target device, and also for your host 11- A build system or three; for example, GN, Ninja, CMake, Bazel 12- A code formatting program like clang-format 13- A debugger like OpenOCD to flash and debug your embedded device (OpenOCD 14 support removed for Windows) 15- A known Python version with known modules installed for scripting 16- A Go compiler for the Go-based command line tools 17 18...and so on 19 20In the server space, container solutions like Docker or Podman solve this; 21however, in our experience container solutions are a mixed bag for embedded 22systems development where one frequently needs access to native system 23resources like USB devices, or must operate on Windows. 24 25``pw_env_setup`` is our compromise solution for this problem that works on Mac, 26Windows, and Linux. It leverages the Chrome packaging system `CIPD`_ to 27bootstrap a Python installation, which in turn inflates a virtual 28environment. The tooling is installed into your workspace, and makes no 29changes to your system. This tooling is designed to be reused by any 30project. 31 32.. _CIPD: https://github.com/luci/luci-go/tree/HEAD/cipd 33 34Users interact with ``pw_env_setup`` with two commands: ``. bootstrap.sh`` and 35``. activate.sh``. The bootstrap command always pulls down the current versions 36of CIPD packages and sets up the Python virtual environment. The activate 37command reinitializes a previously configured environment, and if none is found, 38runs bootstrap. 39 40.. note:: 41 42 On Windows the scripts used to set up the environment are ``bootstrap.bat`` 43 and ``activate.bat``. 44 45 ``bootstrap.fish`` and ``activate.fish`` are also available for `Fish shell 46 <https://fishshell.com/>`_ users. 47 48 For simplicity they will be referred to with the ``.sh`` endings unless the 49 distinction is relevant. 50 51On POSIX systems, the environment can be deactivated by running ``deactivate``. 52 53================================== 54Using pw_env_setup in your project 55================================== 56 57Downstream Projects Using Pigweed's Packages 58******************************************** 59 60Projects using Pigweed can leverage ``pw_env_setup`` to install Pigweed's 61dependencies or their own dependencies. Projects that only want to use Pigweed's 62dependencies without modifying them can just source Pigweed's ``bootstrap.sh`` 63and ``activate.sh`` scripts. 64 65An example of what your project's `bootstrap.sh` could look like is below. This 66assumes `bootstrap.sh` is at the top level of your repository. 67 68.. code-block:: bash 69 70 # Do not include a "#!" line, this must be sourced and not executed. 71 72 # This assumes the user is sourcing this file from it's parent directory. See 73 # below for a more flexible way to handle this. 74 PROJ_SETUP_SCRIPT_PATH="$(pwd)/bootstrap.sh" 75 76 export PW_PROJECT_ROOT="$(_python_abspath "$(dirname "$PROJ_SETUP_SCRIPT_PATH")")" 77 78 # You may wish to check if the user is attempting to execute this script 79 # instead of sourcing it. See below for an example of how to handle that 80 # situation. 81 82 # Source Pigweed's bootstrap utility script. 83 # Using '.' instead of 'source' for POSIX compatibility. Since users don't use 84 # dash directly, using 'source' in most documentation so users don't get 85 # confused and try to `./bootstrap.sh`. 86 . "$PW_PROJECT_ROOT/third_party/pigweed/pw_env_setup/util.sh" 87 88 pw_check_root "$PW_ROOT" 89 _PW_ACTUAL_ENVIRONMENT_ROOT="$(pw_get_env_root)" 90 export _PW_ACTUAL_ENVIRONMENT_ROOT 91 SETUP_SH="$_PW_ACTUAL_ENVIRONMENT_ROOT/activate.sh" 92 pw_bootstrap --args... # See below for details about args. 93 pw_finalize bootstrap "$SETUP_SH" 94 95 96Bazel Usage 97----------- 98It is possible to pull in a CIPD dependency into Bazel using WORKSPACE rules 99rather than using `bootstrap.sh`. e.g. 100 101.. code-block:: python 102 103 # WORKSPACE 104 105 load("//pw_env_setup/bazel/cipd_setup:cipd_rules.bzl", "pigweed_deps") 106 107 # Setup CIPD client and packages. 108 # Required by: pigweed. 109 # Used by modules: all. 110 pigweed_deps() 111 112 load("@cipd_deps//:cipd_init.bzl", "cipd_init") 113 114 cipd_init() 115 116 117This will make the entire set of Pigweeds remote repositories available to your 118project. Though these repositories will only be donwloaded if you use them. To 119get a full list of the remote repositories that this configures, run: 120 121.. code-block:: sh 122 123 bazel query //external:all | grep cipd_ 124 125All files and executables in each CIPD remote repository is exported and visible 126either directely (`@cipd_<dep>//:<file>`) or from 'all' filegroup 127(`@cipd_<dep>//:all`). 128 129From here it is possible to get access to the Bloaty binaries using the 130following command. For example; 131 132.. code-block:: sh 133 134 bazel run @cipd_pigweed_third_party_bloaty_embedded_linux_amd64//:bloaty \ 135 -- --help 136 137User-Friendliness 138----------------- 139 140You may wish to allow sourcing `bootstrap.sh` from a different directory. In 141that case you'll need the following at the top of `bootstrap.sh`. 142 143.. code-block:: bash 144 145 _python_abspath () { 146 python -c "import os.path; print(os.path.abspath('$@'))" 147 } 148 149 # Use this code from Pigweed's bootstrap to find the path to this script when 150 # sourced. This should work with common shells. PW_CHECKOUT_ROOT is only used in 151 # presubmit tests with strange setups, and can be omitted if you're not using 152 # Pigweed's automated testing infrastructure. 153 if test -n "$PW_CHECKOUT_ROOT"; then 154 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$PW_CHECKOUT_ROOT/bootstrap.sh")" 155 unset PW_CHECKOUT_ROOT 156 # Shell: bash. 157 elif test -n "$BASH"; then 158 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$BASH_SOURCE")" 159 # Shell: zsh. 160 elif test -n "$ZSH_NAME"; then 161 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "${(%):-%N}")" 162 # Shell: dash. 163 elif test ${0##*/} = dash; then 164 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath \ 165 "$(lsof -p $$ -Fn0 | tail -1 | sed 's#^[^/]*##;')")" 166 # If everything else fails, try $0. It could work. 167 else 168 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$0")" 169 fi 170 171You may also wish to check if the user is attempting to execute `bootstrap.sh` 172instead of sourcing it. Executing `bootstrap.sh` would download everything 173required for the environment, but cannot modify the environment of the parent 174process. To check for this add the following. 175 176.. code-block:: bash 177 178 # Check if this file is being executed or sourced. 179 _pw_sourced=0 180 # If not running in Pigweed's automated testing infrastructure the 181 # SWARMING_BOT_ID check is unnecessary. 182 if [ -n "$SWARMING_BOT_ID" ]; then 183 # If set we're running on swarming and don't need this check. 184 _pw_sourced=1 185 elif [ -n "$ZSH_EVAL_CONTEXT" ]; then 186 case $ZSH_EVAL_CONTEXT in *:file) _pw_sourced=1;; esac 187 elif [ -n "$KSH_VERSION" ]; then 188 [ "$(cd $(dirname -- $0) && pwd -P)/$(basename -- $0)" != \ 189 "$(cd $(dirname -- ${.sh.file}) && pwd -P)/$(basename -- ${.sh.file})" ] \ 190 && _pw_sourced=1 191 elif [ -n "$BASH_VERSION" ]; then 192 (return 0 2>/dev/null) && _pw_sourced=1 193 else # All other shells: examine $0 for known shell binary filenames 194 # Detects `sh` and `dash`; add additional shell filenames as needed. 195 case ${0##*/} in sh|dash) _pw_sourced=1;; esac 196 fi 197 198 _pw_eval_sourced "$_pw_sourced" 199 200Downstream Projects Using Different Packages 201******************************************** 202Projects depending on Pigweed but using additional or different packages should 203copy the Pigweed `sample project`'s ``bootstrap.sh`` and ``pigweed.json`` and 204update the call to ``pw_bootstrap``. Search for "downstream" for other places 205that may require changes, like setting the ``PW_ROOT`` and ``PW_PROJECT_ROOT`` 206environment variables. Explanations of parts of ``pigweed.json`` are described 207here. 208 209.. _sample project: https://pigweed.googlesource.com/pigweed/sample_project/+/HEAD 210 211``pw.pw_env_setup.root_variable`` 212 Variable used to point to the root of the source tree. Optional, can always 213 use ``PW_PROJECT_ROOT`` instead. (That variable will be set regardless of 214 whether this is provided.) 215 216``pw.pw_env_setup.relative_pigweed_root`` 217 Location of the Pigweed submodule within the source tree. Optional—environment 218 setup will work correctly without this. If present, will confirm that it's 219 correct. May be used by other tooling. 220 221``pw.pw_env_setup.cipd_package_files`` 222 CIPD package file. JSON file consisting of a list of additional CIPD package 223 files to import and a list of dictionaries with "path", "platforms", "subdir", 224 "tags", and "version_file" keys. Both top-level lists are optional. An 225 example is below. Only "path", "platforms", and "tags" are required. If 226 "version_file" is specified then ``pw doctor`` will fail if that version file 227 is not present. If "subdir" is specified then this packages will be installed 228 in a subdirectory of the directory created for packages in this file. 229 230.. code-block:: json 231 232 { 233 "included_files": [ 234 "foo.json" 235 ], 236 "packages": [ 237 { 238 "path": "infra/3pp/tools/go/${platform}", 239 "platforms": [ 240 "linux-amd64", 241 "linux-arm64", 242 "mac-amd64", 243 "windows-amd64" 244 ], 245 "subdir": "pa/th", 246 "tags": [ 247 "version:2@1.16.3" 248 ], 249 "version_file": ".versions/go.cipd_version" 250 } 251 ] 252 } 253 254``pw.pw_env_setup.project_actions`` 255 A list of plugins to load and run after CIPD setup, but prior to virtualenv 256 setup, for e.g. downloading project-specific tools or artifacts needed by 257 later steps. Particularly useful for downstream projects with limited CIPD 258 access. 259 260 A plugin is specified as a dictionary with two keys: "import_path" and 261 "module_name" 262 263 The specified module must provide a "run_actions" method which takes a single 264 argument, "env_vars", which is a pw_env_setup.Environment instance. 265 266 NB: This feature is not supported when using a python2.7 system python. 267 268 Sample plugin and pigweed.json blob: 269 270.. code-block:: python 271 272 """Sample pw_env_setup project action plugin. 273 274 A sample/starter project action plugin template for pw_env_setup. 275 """ 276 def run_action(**kwargs): 277 """Sample project action.""" 278 if "env" not in kwargs: 279 raise ValueError(f"Missing required kwarg 'env', got %{kwargs}") 280 281 kwargs["env"].prepend("PATH", "PATH_TO_NEW_TOOLS") 282 raise NotImplementedError("Sample project action running!") 283 284.. code-block:: json 285 286 "project_actions" : [ 287 { 288 "import_path": "pw_env_setup", 289 "module_name": "sample_project_action" 290 } 291 ], 292 293``pw.pw_env_setup.virtualenv.gn_args`` 294 Any necessary GN args to be used when installing Python packages. 295 296``pw.pw_env_setup.virtualenv.gn_targets`` 297 Target for installing Python packages. Downstream projects will need to 298 create targets to install their packages or only use Pigweed Python packages. 299 300``pw.pw_env_setup.virtualenv.gn_root`` 301 The root directory of your GN build tree, relative to ``PW_PROJECT_ROOT``. 302 This is the directory your project's ``.gn`` file is located in. If you're 303 only installing Pigweed Python packages, use the location of the Pigweed 304 submodule. 305 306``pw.pw_env_setup.virtualenv.requirements`` 307 A list of Python Pip requirements files for installing into the Pigweed 308 virtualenv. Each file will be passed as additional ``--requirement`` argument 309 to a single ```pip install`` at the beginning of bootstrap's ``Python 310 environment`` setup stage. See the `Requirements Files documentation`_ for 311 details on what can be specified using requirements files. 312 313``pw.pw_env_setup.virtualenv.constraints`` 314 A list of Python Pip constraints files. These constraints will be passed to 315 every ``pip`` invocation as an additional ``--constraint`` argument during 316 bootstrap. virtualenv. See the `Constraints Files documentation`_ for details 317 on formatting. 318 319``pw.pw_env_setup.virtualenv.system_packages`` 320 A boolean value that can be used the give the Python virtual environment 321 access to the system site packages. Defaults to ``false``. 322 323``pw.pw_env_setup.virtualenv.pip_install_offline`` 324 A boolean value that adds ``--no-index`` to all ``pip install`` commands that 325 are part of bootstrap. This forces pip to not reach out to the internet 326 (usually `pypi.org <https://pypi.org/>`_) to download packages. Using this 327 option requires setting 328 ``pw.pw_env_setup.virtualenv.pip_install_find_links``. Defaults to 329 ``false``. 330 331 .. seealso:: 332 The Python GN guide for offline pip installation: 333 :ref:`docs-python-build-installing-offline` 334 335``pw.pw_env_setup.virtualenv.pip_install_find_links`` 336 List of paths to folders containing Python wheels (``*.whl``) or source tar 337 files (``*.tar.gz``). Pip will check each of these directories when looking 338 for potential install candidates. Each path will be passed to all ``pip 339 install`` commands as ``--find-links PATH``. 340 341 .. tip:: 342 Environment variables may be used in these paths. For example: 343 344 .. code-block:: json 345 346 "virtualenv": { 347 "pip_install_find_links": [ 348 "${PW_PROJECT_ROOT}/pip_cache" 349 ] 350 } 351 352``pw.pw_env_setup.virtualenv.pip_install_require_hashes`` 353 Adds ``--require-hashes`` This option enforces hash checking on Python 354 package files. Defaults to ``false``. 355 356``pw.pw_env_setup.virtualenv.pip_install_disable_cache`` 357 A boolean value that adds ``--no-cache-dir`` to all ``pip install`` commands 358 that are part of bootstrap. This forces pip to ignore any previously cached 359 Python packages. On most systems this is located in 360 ``~/.cache/pip/``. Defaults to ``false``. 361 362``pw.pw_env_setup.optional_submodules`` 363 By default environment setup will check that all submodules are present in 364 the checkout. Any submodules in this list are excluded from that check. 365 366``pw.pw_env_setup.required_submodules`` 367 If this is specified instead of ``optional_submodules`` bootstrap will only 368 complain if one of the required submodules is not present. Combining this 369 with ``optional_submodules`` is not supported. 370 371``pw.pw_env_setup.pw_packages`` 372 A list of packages to install using :ref:`pw_package <module-pw_package>` 373 after the rest of bootstrap completes. 374 375``pw.pw_env_setup.gni_file`` 376 Location to write a ``.gni`` file containing paths to many things within the 377 environment directory. Defaults to 378 ``build_overrides/pigweed_environment.gni``. 379 380``pw.pw_env_setup.json_file`` 381 Location to write a ``.json`` file containing step-by-step modifications to 382 the environment, for reading by tools that don't inherit an environment from 383 a sourced ``bootstrap.sh``. 384 385``pw.pw_env_setup.rosetta`` 386 Whether to use Rosetta to use amd64 packages on arm64 Macs. Accepted values 387 are ``never``, ``allow``, and ``force``. For now, ``allow`` means ``force``. 388 At some point in the future ``allow`` will be changed to mean ``never``. 389 390An example of a config file is below. 391 392.. code-block:: json 393 394 { 395 "pw": { 396 "pw_env_setup": { 397 "root_variable": "EXAMPLE_ROOT", 398 "cipd_package_files": [ 399 "pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/pigweed.json", 400 "pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/luci.json" 401 "tools/myprojectname.json" 402 ], 403 "virtualenv": { 404 "gn_root": ".", 405 "gn_targets": [ 406 ":python.install", 407 ], 408 "system_packages": false 409 }, 410 "pw_packages": [], 411 "optional_submodules": [ 412 "optional/submodule/one", 413 "optional/submodule/two" 414 ], 415 "gni_file": "tools/environment.gni", 416 "json_file": "tools/environment.json", 417 "rosetta": "allow" 418 } 419 } 420 } 421 422Only the packages necessary for almost all projects based on Pigweed are 423included in the ``cipd_setup/pigweed.json`` file. A number of other files are 424present in that directory for projects that need more than the minimum. 425Internal-Google projects using LUCI should at least include ``luci.json``. 426 427In case the CIPD packages need to be referenced from other scripts, variables 428like ``PW_${BASENAME}_CIPD_INSTALL_DIR`` point to the CIPD install directories, 429where ``${BASENAME}`` is ``"PIGWEED"`` for 430``"pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/pigweed.json"`` and 431``"LUCI"`` for 432``"pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/luci.json"``. This example 433would set the following environment variables. 434 435- ``PW_LUCI_CIPD_INSTALL_DIR`` 436- ``PW_MYPROJECTNAME_CIPD_INSTALL_DIR`` 437- ``PW_PIGWEED_CIPD_INSTALL_DIR`` 438 439These directories are also referenced in the gni_file specified by the 440environment config file as ``dir_cipd_${BASENAME}``. This allows the GN build to 441reliably reference these directories without using GN ``getenv()`` calls or 442hardcoding paths. 443 444In addition, ``PW_${BASENAME}_CIPD_INSTALL_DIR`` and 445``PW_${BASENAME}_CIPD_INSTALL_DIR/bin`` are both added to ``PATH`` for each 446package directory. 447 448If multiple packages install executables with the same name, the file mentioned 449last topologically takes priority. For example, with the file contents below, 450``d.json``'s entries will appear in ``PATH`` before ``c.json``'s, which will 451appear before ``b.json``'s, which will appear before ``a.json``'s. 452 453.. code-block:: json 454 :caption: :octicon:`file;1em` pigweed.json 455 456 { 457 "pw": { 458 "pw_env_setup": { 459 "cipd_package_files": [ 460 "a.json", 461 "b.json", 462 "d.json" 463 ] 464 } 465 } 466 } 467 468.. code-block:: json 469 :caption: :octicon:`file;1em` a.json 470 471 { 472 "package_files": [ 473 // ... 474 ] 475 } 476 477.. code-block:: json 478 :caption: :octicon:`file;1em` b.json 479 480 { 481 "included_files": ["c.json"], 482 "package_files": [ 483 // ... 484 ] 485 } 486 487.. code-block:: json 488 :caption: :octicon:`file;1em` c.json 489 490 { 491 "package_files": [ 492 // ... 493 ] 494 } 495 496.. code-block:: json 497 :caption: :octicon:`file;1em` d.json 498 499 { 500 "package_files": [ 501 // ... 502 ] 503 } 504 505.. code-block:: 506 :caption: Effective File Loading Order 507 508 pigweed.json 509 a.json 510 b.json 511 c.json 512 d.json 513 514Pinning Python Packages 515*********************** 516Python modules usually express dependencies as ranges, which makes it easier to 517install many Python packages that might otherwise have conflicting dependencies. 518However, this means version of packages can often change underneath us and 519builds will not be hermetic. 520 521To ensure versions don't change without approval, Pigweed by default pins the 522versions of packages it depends on using a `pip constraints file`_. To pin the 523versions of additional packages your project depends on, run 524``pw python-packages list <path/to/constraints/file>`` and then add 525``pw_build_PIP_CONSTRAINTS = ["//path/to/constraints/file"]`` to your project's 526``.gn`` file (see `Pigweed's .gn file`_ for an example). 527 528.. _pip constraints file: https://pip.pypa.io/en/stable/user_guide/#constraints-files 529.. _default constraints: https://cs.pigweed.dev/pigweed/+/main:pw_env_setup/py/pw_env_setup/virtualenv_setup/constraint.list 530.. _Pigweed's .gn file: https://cs.pigweed.dev/pigweed/+/main:.gn 531 532To update packages, set ``pw_build_PIP_CONSTRAINTS = []``, delete the 533environment, and bootstrap again. Then run the ``list`` command from above 534again, and run ``pw presubmit``. 535 536Environment Variables 537********************* 538Input Variables 539--------------- 540The following environment variables affect env setup behavior. Most users will 541never need to set these. 542 543``CIPD_CACHE_DIR`` 544 Location of CIPD cache dir. Read by CIPD, but if unset will be defaulted to 545 ``$HOME/.cipd-cache-dir``. 546 547``PW_NO_CIPD_CACHE_DIR`` 548 Disables the CIPD cache. 549 550``PW_ACTIVATE_SKIP_CHECKS`` 551 If set, skip running ``pw doctor`` at end of bootstrap/activate. Intended to 552 be used by automated tools but not interactively. 553 554``PW_BANNER_FUNC`` 555 Command to print a banner at the beginning of bootstrap. 556 557``PW_BOOTSTRAP_PYTHON`` 558 Python executable to be used, for example "python2" or "python3". Defaults to 559 "python". 560 561``PW_CIPD_SERVICE_ACCOUNT_JSON`` 562 Value to pass as ``-service-account-json`` to CIPD invocations. This should 563 point either to a service account JSON key file, or be the magical value 564 ``:gce`` to tell the tool to fetch tokens from GCE metadata server. 565 566``PW_ENVIRONMENT_ROOT`` 567 Location to which packages are installed. Defaults to ``environment`` folder 568 within the checkout root. This variable is cleared after environment setup is 569 complete. 570 571``PW_ENVSETUP_DISABLE_SPINNER`` 572 Disable the spinner during env setup. Intended to be used when the output is 573 being redirected to a log. 574 575``PW_ENVSETUP_DISABLE_SPINNER`` 576 Disable the console spinner that runs when waiting for env setup steps to 577 complete. 578 579``PW_ENVSETUP_NO_BANNER`` 580 Skip printing the banner. 581 582``PW_ENVSETUP_QUIET`` 583 Disables all non-error output. 584 585``PW_PROJECT_ROOT`` 586 The absolute path of the project using Pigweed's env setup. For Pigweed this 587 is the same as ``PW_ROOT``. This should be set by the project's bootstrap 588 script. 589 590``PW_ROOT`` 591 The absolute path to the Pigweed repository within ``PW_PROJECT_ROOT``. This 592 should be set by the project's bootstrap script. 593 594Output Variables 595---------------- 596The following environment variables are set by env setup. 597 598``PATH`` 599 System executable search path. Many of the environment variables below are 600 also added to this variable. 601 602``_PW_ACTUAL_ENVIRONMENT_ROOT`` 603 Location the environment was installed into. Separate from 604 ``PW_ENVIRONMENT_ROOT`` because setting that implicitly and switching to 605 another project directory causes unexpected behavior. 606 607``PW_CIPD_INSTALL_DIR`` 608 Top-level CIPD install directory. This is where the ``cipd`` executable is. 609 610``PW_*_CIPD_INSTALL_DIR`` 611 Each CIPD package file is installed into its own directory. This allows other 612 tools to determine what those directories are. The ``*`` is replaced with an 613 all-caps version of the basename of the package file, without the extension. 614 (E.g., "path/foo.json" becomes ``PW_FOO_CIPD_INSTALL_DIR``.) 615 616``PW_PACKAGE_ROOT`` 617 Location that packages installed by ``pw package`` will be installed to. 618 619``VIRTUAL_ENV`` 620 Path to Pigweed's virtualenv. 621 622Non-Shell Environments 623********************** 624If using this outside of bash—for example directly from an IDE or CI 625system—users can process the ``actions.json`` file that's generated in the 626location specified by the environment config. It lists variables to set, clear, 627and modify. An example ``actions.json`` is shown below. The "append" and 628"prepend" actions are listed in the order they should be applied, so the 629``<pigweed-root>/out/host/host_tools`` entry should be at the beginning of 630``PATH`` and not in the middle somewhere. 631 632.. code-block:: json 633 634 { 635 "modify": { 636 "PATH": { 637 "append": [], 638 "prepend": [ 639 "<pigweed-root>/environment/cipd", 640 "<pigweed-root>/environment/cipd/pigweed", 641 "<pigweed-root>/environment/cipd/pigweed/bin", 642 "<pigweed-root>/environment/cipd/luci", 643 "<pigweed-root>/environment/cipd/luci/bin", 644 "<pigweed-root>/environment/pigweed-venv/bin", 645 "<pigweed-root>/out/host/host_tools" 646 ], 647 "remove": [] 648 } 649 }, 650 "set": { 651 "PW_PROJECT_ROOT": "<pigweed-root>", 652 "PW_ROOT": "<pigweed-root>", 653 "_PW_ACTUAL_ENVIRONMENT_ROOT": "<pigweed-root>/environment", 654 "PW_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd", 655 "CIPD_CACHE_DIR": "<home>/.cipd-cache-dir", 656 "PW_PIGWEED_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/pigweed", 657 "PW_LUCI_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/luci", 658 "VIRTUAL_ENV": "<pigweed-root>/environment/pigweed-venv", 659 "PYTHONHOME": null, 660 "__PYVENV_LAUNCHER__": null 661 } 662 } 663 664Many of these variables are directly exposed to the GN build as well, through 665the GNI file specified in the environment config file. 666 667.. code-block:: 668 669 declare_args() { 670 pw_env_setup_CIPD_LUCI = "<environment-root>/cipd/packages/luci" 671 pw_env_setup_CIPD_PIGWEED = "<environment-root>/cipd/packages/pigweed" 672 pw_env_setup_PACKAGE_ROOT = "<environment-root>/packages" 673 pw_env_setup_VIRTUAL_ENV = "<environment-root>/pigweed-venv" 674 } 675 676It's straightforward to use these variables. 677 678.. code-block:: cpp 679 680 import("//build_overrides/pigweed_environment.gni") 681 682 deps = [ "$pw_env_setup_CIPD_PIGWEED/..." ] 683 684Implementation 685************** 686The environment is set up by installing CIPD and Python packages in 687``PW_ENVIRONMENT_ROOT`` or ``<checkout>/environment``, and saving modifications 688to environment variables in setup scripts in those directories. To support 689multiple operating systems this is done in an operating system-agnostic manner 690and then written into operating system-specific files to be sourced now and in 691the future when running ``activate.sh`` instead of ``bootstrap.sh``. In the 692future these could be extended to C shell and PowerShell. A logical mapping of 693high-level commands to system-specific initialization files is shown below. 694 695.. grid:: 1 696 :padding: 0 697 698 .. grid-item-card:: 699 :columns: 12 700 :class-header: font-monospace 701 702 SET $PW_ROOT /home/$USER/pigweed 703 ^^^ 704 705 .. grid:: 2 706 :margin: 0 707 :padding: 0 708 709 .. grid-item:: **Windows** 710 711 .. grid-item:: **Linux & Mac (sh-compatible shells)** 712 713 .. grid:: 2 714 :margin: 0 715 :padding: 0 716 717 .. grid-item:: 718 719 .. code-block:: dosbatch 720 721 set PW_ROOT /home/%USER%/pigweed 722 723 .. grid-item:: 724 725 .. code-block:: shell 726 727 PW_ROOT="/home/$USER/pigweed" 728 export PW_ROOT 729 730.. grid:: 1 731 :padding: 0 732 733 .. grid-item-card:: 734 :columns: 12 735 :class-header: font-monospace 736 737 PREPEND $PATH $PW_ROOT/.env/bin 738 ^^^ 739 .. grid:: 2 740 :margin: 0 741 :padding: 0 742 743 .. grid-item:: **Windows** 744 745 .. grid-item:: **Linux & Mac (sh-compatible shells)** 746 747 .. grid:: 2 748 :margin: 0 749 :padding: 0 750 751 .. grid-item:: 752 753 .. code-block:: dosbatch 754 755 set PATH=%PW_ROOT%/.env/bin;%PATH% 756 757 .. grid-item:: 758 759 .. code-block:: shell 760 761 PATH="$(\ 762 echo "$PATH" | \ 763 sed "s|:$PW_ROOT/.env/bin:|:|g;" | \ 764 sed "s|^$PW_ROOT/.env/bin:||g;" | \ 765 sed "s|:$PW_ROOT/.env/bin$||g;")" 766 PATH="$PW_ROOT/.env/bin;$PATH" 767 export PATH 768 769.. grid:: 1 770 :padding: 0 771 772 .. grid-item-card:: 773 :columns: 12 774 :class-header: font-monospace 775 776 ECHO "Setup Complete!" 777 ^^^ 778 779 .. grid:: 2 780 :margin: 0 781 :padding: 0 782 783 .. grid-item:: **Windows** 784 785 .. grid-item:: **Linux & Mac (sh-compatible shells)** 786 787 788 .. grid:: 2 789 :margin: 0 790 :padding: 0 791 792 .. grid-item:: 793 794 .. code-block:: dosbatch 795 796 echo Setup Complete! 797 798 .. grid-item:: 799 800 .. code-block:: shell 801 802 echo "Setup Complete!" 803 804 805.. _Requirements Files documentation: https://pip.pypa.io/en/stable/user_guide/#requirements-files 806.. _Constraints Files documentation: https://pip.pypa.io/en/stable/user_guide/#constraints-files 807