Lines Matching refs:Python
4 Pigweed's GN Python Build
15 Pigweed uses a custom GN-based build system to manage its Python code. The
16 Pigweed Python build supports packaging, installation and distribution of
17 interdependent local Python packages. It also provides for fast, incremental
21 Pigweed's Python code is exclusively managed by GN, but the GN-based build may
23 setup uses GN to set up the initial Python environment, regardless of the final
24 build system. As needed, non-GN projects can declare just their Python packages
29 In addition to compiler commands a Pigweed GN build will execute Python scripts
33 ``python``. Running Python on it's own by default will make any Python packages
36 workstation. To get around this the Python community uses `virtual environments
38 set of Python packages separate from the host system.
42 graph. Once created, all required third-party Python packages needed for the
65 subgraph pyactions1[Python venv 1]
74 subgraph pyactions2[Python venv 2]
93 dependencies you may be wondering how you can import your own in-tree Python
94 packages. Python code run in the build may still import any in-tree Python
97 provided. Having that Python dependency defined in GN allows the
102 dependency for any Python action or package is missing.
104 .. admonition:: Benefits of Python ``venvs`` in GN
107 - Using venvs to execute Python in GN provides reproducible builds with fixed
109 - Using ``PYTHONPATH`` coupled with ``python_deps`` to import in-tree Python
113 Managing Python Requirements
118 Build Time Python Virtualenv
120 Pigweed's GN Python build infrastructure relies on a single build-only venv for
121 executing Python code. This provides an isolated environment with a reproducible
122 set of third party Python constraints where all Python tests and linting can
135 Third-party Python Requirements and Constraints
137 Your project may have third party Python dependencies you wish to install into
139 to add Python package dependencies:
145 if your in-tree Python package requires an external Python package.
147 2. Create a standard Python ``requirements.txt`` file in your project and add it
170 Every project should ideally inherit Pigweed's third party Python package
171 version. This is accomplished via `Python constraints files
174 installed. To inherit Pigweed's Python constraints include ``constraint.list``
187 GN Structure for Python Code
189 Here is a full example of what is required to build Python packages using
192 on the structure of Python packages.
238 # Inherit Pigweed Python constraints
295 # This group is built during bootstrap to setup the interactive Python
304 # In-tree Python packages
309 # Pigweed Python packages to include
319 # The default venv for Python actions in GN
330 # Ensure all third party Python dependencies are installed into this venv.
337 # single super Python package for installation into the bootstrapped virtual
349 # Install the project-tools super Python package into the bootstrapped
350 # Python venv.
357 Pigweed Module Structure for Python Code
359 Pigweed Python code is structured into standard Python packages. This makes it
360 simple to package and distribute Pigweed Python packages with common Python
363 Like all Pigweed source code, Python packages are organized into Pigweed
364 modules. A module's Python package is nested under a ``py/`` directory (see
368 :caption: :octicon:`file-directory;1em` Example layout of a Pigweed Python package.
389 check in ensures that all Python files are listed in a ``BUILD.gn``.
391 Pigweed prefers to define Python packages using ``setup.cfg`` files. In the
413 used for the given Python package. In Pigweed's case it always specifies using
426 The key abstraction in the Python build is the ``pw_python_package``.
427 A ``pw_python_package`` represents a Python package as a GN target. It is
439 subtarget represents different functionality in the Python build.
441 - ``<name>`` - Represents the Python files in the build, but does not take any
447 - ``<name>.lint`` - Runs static analysis tools on the Python code. This is a
450 - ``<name>.lint.mypy`` - Runs Mypy on all Python files, if enabled.
451 - ``<name>.lint.pylint`` - Runs Pylint on all Python files, if enabled.
453 - ``<name>.install`` - Installs the package in a Python virtual environment.
454 - ``<name>.wheel`` - Builds a Python wheel for this package.
456 To avoid unnecessary duplication, all Python actions are executed in the default
461 Tests for a Python package are listed in its ``pw_python_package`` target.
463 Python package. The build will run it when the test, the package, or one of its
485 Building Python wheels
488 distributing Python packages. The Pigweed Python build supports creating wheels
497 Wheels for a Python package and its transitive dependencies can be collected
504 target (see :ref:`module-pw_protobuf_compiler`). Python protobuf modules are
505 generated as standalone Python packages by default. Protocol buffers may also be
506 nested within existing Python packages. In this case, the Python package in the
507 source tree is incomplete; the final Python package, including protobufs, is
534 Python is an interpreted language, but it shares most build automation concerns
535 with other languages. Pigweed uses Python extensively and must address these
540 The Python programming langauge does not have an official build automation
541 system. However, there are numerous Python-focused build automation tools with
542 varying degrees of adoption. See the `Python Wiki
545 A few Python tools have become defacto standards, including `setuptools
548 These essential tools address key aspects of Python packaging and distribution,
551 provide more general build automation for Python.
554 Python and other languages used by Pigweed, including protocol buffers.
558 Pigweed's use of Python is different from many other projects. Pigweed is a
562 This section describes Python build automation challenges encountered by
567 Pigweed is organized into distinct modules. In Python, each module is a separate
571 The basic Python packaging tools lack dependency tracking for local packages.
590 There are lots of great Python libraries for testing, such as
593 write and execute individual Python tests, but are not well suited for managing
606 Various static analysis tools exist for Python. Two widely used, powerful tools
631 The protobuf compiler ``protoc`` generates Python modules from ``.proto`` files.
642 Python build.
647 - Correctly manage interdependent local Python packages.
656 Existing Python tools may be effective for Python codebases, but their utility
662 natively support Python, adding support is straightforward with GN templates.
666 example, C++, Go, and Python targets can depend on the same protobuf
672 clean syntax. This makes it feasible to use GN only for Python while building
675 Given these considerations, GN is an ideal choice for Pigweed's Python build.