Lines Matching +full:- +full:python +full:- +full:pip
1 :::{default-domain} bzl
6 Using PyPI packages (aka "pip install") involves two main steps.
8 1. [Installing third party packages](#installing-third-party-packages)
9 2. [Using third party packages as dependencies](#using-third-party-packages)
11 {#installing-third-party-packages}
16 To add pip dependencies to your `MODULE.bazel` file, use the `pip.parse`
22 pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
23 pip.parse(
28 use_repo(pip, "my_deps")
31 file, see the bzlmod examples under the {gh-path}`examples` folder or the documentation
32 for the {obj}`@rules_python//python/extensions:pip.bzl` extension.
35 We are using a host-platform compatible toolchain by default to setup pip dependencies.
40 startup --windows_enable_symlinks
43 hermetic host python interpreter on this platform. Linux and OSX users should see no
49 To add pip dependencies to your `WORKSPACE`, load the `pip_parse` function and
53 load("@rules_python//python:pip.bzl", "pip_parse")
67 (vendoring-requirements)=
77 [`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos)
80 https://blog.aspect.dev/bazel-can-write-to-the-source-folder
85 (per-os-arch-requirements)=
88 …inations. This is enabled via the `requirements_by_platform` attribute in `pip.parse` extension an…
107 An alternative way is to use per-OS requirement attributes.
117 ### pip rules
119 Note that since `pip_parse` and `pip.parse` are executed at evaluation time,
120 Bazel has no information about the Python toolchain and cannot enforce that the
121 interpreter used to invoke `pip` matches the interpreter used to run
124 `python_interpreter_target` attribute to `pip_parse`. The `pip.parse` `bzlmod` extension
125 by default uses the hermetic python toolchain for the host platform.
127 You can have multiple `pip_parse`s in the same workspace, or use the pip
133 re-executed to pick up a non-hermetic change to your environment (e.g., updating
134 your system `python` interpreter), you can force it to re-execute by running
135 `bazel sync --only [pip_parse name]`.
137 {#using-third-party-packages}
143 repo's `//:requirements.bzl` file. This function maps a pip package
165 [this issue][requirements-drawbacks] for an enumeration. If you don't
174 `package` is the pip package name with characters that are illegal in
175 Bazel label names (e.g. `-`, `.`) replaced with `_`. If you need to
183 [requirements-drawbacks]: https://github.com/bazelbuild/rules_python/issues/414
190 [whl_ep]: https://packaging.python.org/specifications/entry-points/
223 (advance-topics)=
226 (circular-deps)=
229 Sometimes PyPi packages contain dependency cycles -- for instance a particular
231 2024-06-02) depends on `sphinxcontrib-serializinghtml`. When using them as
244 Bazel will protest because it doesn't support cycles in the build graph --
250 .-> @pypi_sphinxcontrib_serializinghtml//:pkg (...)
254 `-- @pypi_sphinxcontrib_serializinghtml//:pkg (...)
268 "sphinxcontrib-serializinghtml",
275 be distinct. `apache-airflow` for instance has dependency cycles with a number
277 be a part of the `airflow` cycle. For instance --
284 "apache-airflow",
285 "apache-airflow-providers-common-sql",
286 "apache-airflow-providers-postgres",
287 "apache-airflow-providers-sqlite",
295 For example while `apache-airflow-providers-sqlite` is "baked into" the Airflow
296 package, `apache-airflow-providers-postgres` is not and is an optional feature.
297 Rather than listing `apache-airflow[postgres]` in your `requirements.txt` which
299 `apache-airflow` and `apache-airflow-providers-postgres` separately as
300 requirements. Bazel rules which need only `apache-airflow` can take it as a
302 `apache-airflow-providers-postgres` now can.
306 `apache-airflow-providers-postgres` not explicitly depend on `apache-airflow` or
307 perhaps `apache-airflow-providers-common-sql`.
310 (bazel-downloader)=
311 ### Multi-platform support
313 Multi-platform support of cross-building the wheels can be done in two ways - either
314 using {bzl:attr}`experimental_index_url` for the {bzl:obj}`pip.parse` bzlmod tag class
315 or by using the {bzl:attr}`pip.parse.download_only` setting. In this section we
321 --platform=manylinux_2_17_x86_64
322 --python-version=39
323 --implementation=cp
324 --abi=cp39
326 foo==0.0.1 --hash=sha256:deadbeef
327 bar==0.0.1 --hash=sha256:deadb00f
332 --platform=macosx_10_9_arm64
333 --python-version=39
334 --implementation=cp
335 --abi=cp39
337 foo==0.0.3 --hash=sha256:deadbaaf
340 With these 2 files your {bzl:obj}`pip.parse` could look like:
342 pip.parse(
343 hub_name = "pip",
345 # Tell `pip` to ignore sdists
354 With this, the `pip.parse` will create a hub repository that is going to
355 support only two platforms - `cp39_osx_aarch64` and `cp39_linux_x86_64` and it
363 (bazel-downloader)=
364 ### Bazel downloader and multi-platform wheel hub repository.
366 The `bzlmod` `pip.parse` call supports pulling information from `PyPI` (or a
369 the users to use the [credential helper](#credential-helper) to authenticate
371 It also avoids using `pip` altogether and results in much faster dependency
375 the {gh-path}`examples/bzlmod/MODULE.bazel` example.
377 When using this feature during the `pip` extension evaluation you will see the accessed indexes sim…
381 Fetching module extension pip in @@//python/extensions:pip.bzl; starting
400 * {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant.
401 * {obj}`--@rules_python//python/config_settings:pip_whl` for selecting `whl` distribution preferenc…
402 * {obj}`--@rules_python//python/config_settings:pip_whl_osx_arch` for selecting MacOS wheel prefere…
403 * {obj}`--@rules_python//python/config_settings:pip_whl_glibc_version` for selecting the GLIBC vers…
404 * {obj}`--@rules_python//python/config_settings:pip_whl_muslc_version` for selecting the musl versi…
405 * {obj}`--@rules_python//python/config_settings:pip_whl_osx_version` for selecting MacOS version co…
408 [pep600]: https://peps.python.org/pep-0600/
409 [pep656]: https://peps.python.org/pep-0656/
411 (credential-helper)=
414 The "use Bazel downloader for python wheels" experimental feature includes support for the Bazel
415 [Credential Helper][cred-helper-design].
417 Your python artifact registry may provide a credential helper for you. Refer to your index's docs
420 See the [Credential Helper Spec][cred-helper-spec] for details.
422 [cred-helper-design]: https://github.com/bazelbuild/proposals/blob/main/designs/2022-06-07-bazel-cr…
423 [cred-helper-spec]: https://github.com/EngFlow/credential-helper-spec/blob/main/spec.md
430 not provide a credential helper that conforms to the [spec][cred-helper-spec], the script might
446 Configure Bazel to use this credential helper for your python index `example.com`:
450 build --credential_helper=example.com=/full/path/to/cred_helper.sh