Lines Matching +full:environment +full:- +full:file
1 .. _module-pw_env_setup:
6 .. pigweed-module::
11 of tools is needed for non-trivial production embedded projects. For example:
13 - A C++ compiler for your target device, and also for your host
14 - A build system or three; for example, GN, Ninja, CMake, Bazel
15 - A code formatting program like clang-format
16 - A debugger like OpenOCD to flash and debug your embedded device (OpenOCD
18 - A known Python version with known modules installed for scripting
19 - A Go compiler for the Go-based command line tools
31 environment. The tooling is installed into your workspace, and makes no
35 .. _CIPD: https://github.com/luci/luci-go/tree/HEAD/cipd
39 of CIPD packages and sets up the Python virtual environment. The activate
40 command reinitializes a previously configured environment, and if none is found,
45 On Windows the scripts used to set up the environment are ``bootstrap.bat``
54 On POSIX systems, the environment can be deactivated by running ``deactivate``.
56 ----------------------------------
58 ----------------------------------
71 .. code-block:: bash
75 # This assumes the user is sourcing this file from it's parent directory. See
95 pw_bootstrap --args... # See below for details about args.
100 -----------
104 .. code-block:: python
124 .. code-block:: console
129 either directely (`@cipd_<dep>//:<file>`) or from 'all' filegroup
135 .. code-block:: console
138 > -- --help
140 User-Friendliness
141 -----------------
146 .. code-block:: bash
149 python -c "import os.path; print(os.path.abspath('$@'))"
156 if test -n "$PW_CHECKOUT_ROOT"; then
160 elif test -n "$BASH"; then
163 elif test -n "$ZSH_NAME"; then
164 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "${(%):-%N}")"
168 "$(lsof -p $$ -Fn0 | tail -1 | sed 's#^[^/]*##;')")"
176 required for the environment, but cannot modify the environment of the parent
179 .. code-block:: bash
181 # Check if this file is being executed or sourced.
185 if [ -n "$SWARMING_BOT_ID" ]; then
188 elif [ -n "$ZSH_EVAL_CONTEXT" ]; then
189 case $ZSH_EVAL_CONTEXT in *:file) _pw_sourced=1;; esac
190 elif [ -n "$KSH_VERSION" ]; then
191 [ "$(cd $(dirname -- $0) && pwd -P)/$(basename -- $0)" != \
192 "$(cd $(dirname -- ${.sh.file}) && pwd -P)/$(basename -- ${.sh.file})" ] \
194 elif [ -n "$BASH_VERSION" ]; then
209 environment variables. Explanations of parts of ``pigweed.json`` are described
220 Location of the Pigweed submodule within the source tree. Optional—environment
225 CIPD package file. JSON file consisting of a list of additional CIPD package
227 "tags", and "version_file" keys. Both top-level lists are optional. An
229 "version_file" is specified then ``pw doctor`` will fail if that version file
231 in a subdirectory of the directory created for packages in this file.
233 .. code-block:: json
243 "linux-amd64",
244 "linux-arm64",
245 "mac-amd64",
246 "windows-amd64"
259 setup, for e.g. downloading project-specific tools or artifacts needed by
267 argument, "env_vars", which is a pw_env_setup.Environment instance.
271 .. code-block:: python
285 .. code-block:: json
303 This is the directory your project's ``.gn`` file is located in. If you're
309 virtualenv. Each file will be passed as additional ``--requirement`` argument
311 environment`` setup stage. See the `Requirements Files documentation`_ for
316 every ``pip`` invocation as an additional ``--constraint`` argument during
321 A boolean value that can be used the give the Python virtual environment
325 A boolean value that adds ``--no-index`` to all ``pip install`` commands that
334 :ref:`docs-python-build-installing-offline`
340 install`` commands as ``--find-links PATH``.
343 Environment variables may be used in these paths. For example:
345 .. code-block:: json
354 Adds ``--require-hashes`` This option enforces hash checking on Python
358 A boolean value that adds ``--no-cache-dir`` to all ``pip install`` commands
364 By default environment setup will check that all submodules are present in
373 A list of packages to install using :ref:`pw_package <module-pw_package>`
377 Location to write a ``.gni`` file containing paths to many things within the
378 environment directory. Defaults to
382 Location to write a ``.json`` file containing step-by-step modifications to
383 the environment, for reading by tools that don't inherit an environment from
391 An example of a config file is below.
393 .. code-block:: json
416 "gni_file": "tools/environment.gni",
417 "json_file": "tools/environment.json",
424 included in the ``cipd_setup/pigweed.json`` file. A number of other files are
426 Internal-Google projects using LUCI should at least include ``luci.json``.
434 would set the following environment variables.
436 - ``PW_LUCI_CIPD_INSTALL_DIR``
437 - ``PW_MYPROJECTNAME_CIPD_INSTALL_DIR``
438 - ``PW_PIGWEED_CIPD_INSTALL_DIR``
441 environment config file as ``dir_cipd_${BASENAME}``. This allows the GN build to
449 If multiple packages install executables with the same name, the file mentioned
450 last topologically takes priority. For example, with the file contents below,
454 .. code-block:: json
455 :caption: :octicon:`file;1em` pigweed.json
469 .. code-block:: json
470 :caption: :octicon:`file;1em` a.json
478 .. code-block:: json
479 :caption: :octicon:`file;1em` b.json
488 .. code-block:: json
489 :caption: :octicon:`file;1em` c.json
497 .. code-block:: json
498 :caption: :octicon:`file;1em` d.json
506 .. code-block::
507 :caption: Effective File Loading Order
523 versions of packages it depends on using a `pip constraints file`_. To pin the
525 ``pw python-packages list <path/to/constraints/file>`` and then add
526 ``pw_build_PIP_CONSTRAINTS = ["//path/to/constraints/file"]`` to your project's
527 ``.gn`` file (see `Pigweed's .gn file`_ for an example).
529 .. _pip constraints file: https://pip.pypa.io/en/stable/user_guide/#constraints-files
531 .. _Pigweed's .gn file: https://cs.pigweed.dev/pigweed/+/main:.gn
534 environment, and bootstrap again. Then run the ``list`` command from above
537 Environment Variables
540 ---------------
541 The following environment variables affect env setup behavior. Most users will
546 ``$HOME/.cipd-cache-dir``.
563 Value to pass as ``-service-account-json`` to CIPD invocations. This should
564 point either to a service account JSON key file, or be the magical value
568 Location to which packages are installed. Defaults to ``environment`` folder
569 within the checkout root. This variable is cleared after environment setup is
584 Disables all non-error output.
596 ----------------
597 The following environment variables are set by env setup.
600 System executable search path. Many of the environment variables below are
604 Location the environment was installed into. Separate from
609 Top-level CIPD install directory. This is where the ``cipd`` executable is.
612 Each CIPD package file is installed into its own directory. This allows other
614 all-caps version of the basename of the package file, without the extension.
623 Non-Shell Environments
626 system—users can process the ``actions.json`` file that's generated in the
627 location specified by the environment config. It lists variables to set, clear,
630 ``<pigweed-root>/out/host/host_tools`` entry should be at the beginning of
633 .. code-block:: json
640 "<pigweed-root>/environment/cipd",
641 "<pigweed-root>/environment/cipd/pigweed",
642 "<pigweed-root>/environment/cipd/pigweed/bin",
643 "<pigweed-root>/environment/cipd/luci",
644 "<pigweed-root>/environment/cipd/luci/bin",
645 "<pigweed-root>/environment/pigweed-venv/bin",
646 "<pigweed-root>/out/host/host_tools"
652 "PW_PROJECT_ROOT": "<pigweed-root>",
653 "PW_ROOT": "<pigweed-root>",
654 "_PW_ACTUAL_ENVIRONMENT_ROOT": "<pigweed-root>/environment",
655 "PW_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd",
656 "CIPD_CACHE_DIR": "<home>/.cipd-cache-dir",
657 "PW_PIGWEED_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/pigweed",
658 "PW_LUCI_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/luci",
659 "VIRTUAL_ENV": "<pigweed-root>/environment/pigweed-venv",
666 the GNI file specified in the environment config file.
668 .. code-block::
671 pw_env_setup_CIPD_LUCI = "<environment-root>/cipd/packages/luci"
672 pw_env_setup_CIPD_PIGWEED = "<environment-root>/cipd/packages/pigweed"
673 pw_env_setup_PACKAGE_ROOT = "<environment-root>/packages"
674 pw_env_setup_VIRTUAL_ENV = "<environment-root>/pigweed-venv"
679 .. code-block:: cpp
687 The environment is set up by installing CIPD and Python packages in
688 ``PW_ENVIRONMENT_ROOT`` or ``<checkout>/environment``, and saving modifications
689 to environment variables in setup scripts in those directories. To support
690 multiple operating systems this is done in an operating system-agnostic manner
691 and then written into operating system-specific files to be sourced now and in
694 high-level commands to system-specific initialization files is shown below.
699 .. grid-item-card::
701 :class-header: font-monospace
710 .. grid-item:: **Windows**
712 .. grid-item:: **Linux & Mac (sh-compatible shells)**
718 .. grid-item::
720 .. code-block:: dosbatch
724 .. grid-item::
726 .. code-block:: shell
734 .. grid-item-card::
736 :class-header: font-monospace
744 .. grid-item:: **Windows**
746 .. grid-item:: **Linux & Mac (sh-compatible shells)**
752 .. grid-item::
754 .. code-block:: dosbatch
758 .. grid-item::
760 .. code-block:: shell
773 .. grid-item-card::
775 :class-header: font-monospace
784 .. grid-item:: **Windows**
786 .. grid-item:: **Linux & Mac (sh-compatible shells)**
793 .. grid-item::
795 .. code-block:: doscon
799 .. grid-item::
801 .. code-block:: console
806 .. _Requirements Files documentation: https://pip.pypa.io/en/stable/user_guide/#requirements-files
807 .. _Constraints Files documentation: https://pip.pypa.io/en/stable/user_guide/#constraints-files