Lines Matching +full:build +full:- +full:with +full:- +full:python
1 # Python WebAssembly (WASM) build
3 **WARNING: WASM support is work-in-progress! Lots of features are not working yet.**
6 compilation of CPython to WebAssembly (WASM). Python supports Emscripten
7 (*wasm32-emscripten*) and WASI (*wasm32-wasi*) targets. Emscripten builds
12 `Tools/wasm/wasm_build.py`. The tool automates the build process and provides
13 assistance with installation of SDKs.
15 ## wasm32-emscripten build
17 For now the build system has two target flavors. The ``Emscripten/browser``
18 target (``--with-emscripten-target=browser``) is optimized for browsers.
19 It comes with a reduced and preloaded stdlib without tests and threading
23 Cross compiling to the wasm32-emscripten platform needs the
24 [Emscripten](https://emscripten.org/) SDK and a build Python interpreter.
28 Christian Heimes maintains a container image with Emscripten SDK, Python
29 build dependencies, WASI-SDK, wasmtime, and several additional tools.
35 podman run --rm -ti -v $(pwd):/python-wasm/cpython:Z -w /python-wasm/cpython quay.io/tiran/cpythonb…
38 docker run --rm -ti -v $(pwd):/python-wasm/cpython -w /python-wasm/cpython quay.io/tiran/cpythonbui…
41 ### Compile a build Python interpreter argument
46 ./Tools/wasm/wasm_build.py build
52 mkdir -p builddir/build
53 pushd builddir/build
54 ../../configure -C
55 make -j$(nproc)
59 ### Cross-compile to wasm32-emscripten for browser
62 ./Tools/wasm/wasm_build.py emscripten-browser
68 mkdir -p builddir/emscripten-browser
69 pushd builddir/emscripten-browser
71 CONFIG_SITE=../../Tools/wasm/config.site-wasm32-emscripten \
72 emconfigure ../../configure -C \
73 --host=wasm32-unknown-emscripten \
74 --build=$(../../config.guess) \
75 --with-emscripten-target=browser \
76 --with-build-python=$(pwd)/../build/python
78 emmake make -j$(nproc)
82 Serve `python.html` with a local webserver and open the file in a browser.
83 Python comes with a minimal web server script that sets necessary HTTP
91 and open http://localhost:8000/builddir/emscripten-browser/python.html . This
93 and header files with debug builds.
96 ### Cross compile to wasm32-emscripten for node
99 ./Tools/wasm/wasm_build.py emscripten-browser-dl
105 mkdir -p builddir/emscripten-node-dl
106 pushd builddir/emscripten-node-dl
108 CONFIG_SITE=../../Tools/wasm/config.site-wasm32-emscripten \
109 emconfigure ../../configure -C \
110 --host=wasm32-unknown-emscripten \
111 --build=$(../../config.guess) \
112 --with-emscripten-target=node \
113 --enable-wasm-dynamic-linking \
114 --with-build-python=$(pwd)/../build/python
116 emmake make -j$(nproc)
121 node --experimental-wasm-threads --experimental-wasm-bulk-memory --experimental-wasm-bigint builddi…
124 (``--experimental-wasm-bigint`` is not needed with recent NodeJS versions)
126 # wasm32-emscripten limitations and issues
134 - Python's socket module does not work with Emscripten's emulated POSIX
137 - Only ``AF_INET`` and ``AF_INET6`` with ``SOCK_STREAM`` (TCP) or
139 - ``socketpair`` does not work.
140 - Blocking sockets are not available and non-blocking sockets don't work
143 - The ``select`` module is limited. ``select.select()`` crashes the runtime
148 - Processes are not supported. System calls like fork, popen, and subprocess
149 fail with ``ENOSYS`` or ``ENOSUP``.
150 - Signal support is limited. ``signal.alarm``, ``itimer``, ``sigaction``
152 - Keyboard interrupt (CTRL+C) handling is not implemented yet.
153 - Resource-related functions like ``os.nice`` and most functions of the
158 - Threading is disabled by default. The ``configure`` option
159 ``--enable-wasm-pthreads`` adds compiler flag ``-pthread`` and
160 linker flags ``-sUSE_PTHREADS -sPROXY_TO_PTHREAD``.
161 - pthread support requires WASM threads and SharedArrayBuffer (bulk memory).
164 - It's not advised to enable threading when building for browsers or with
169 - Most user, group, and permission related function and modules are not
173 - ``umask`` is a no-op.
174 - hard links (``os.link``) are not supported.
175 - Offset and iovec I/O functions (e.g. ``os.pread``, ``os.preadv``) are not
177 - ``os.mknod`` and ``os.mkfifo``
178 [don't work](https://github.com/emscripten-core/emscripten/issues/16158)
180 - Large file support crashes the runtime and is disabled.
181 - ``mmap`` module is unstable. flush (``msync``) can crash the runtime.
185 - Heap memory and stack size are limited. Recursion or extensive memory
186 consumption can crash Python.
187 - Most stdlib modules with a dependency on external libraries are missing,
189 - Shared extension modules are not implemented yet. All extension modules
191 option ``--enable-wasm-dynamic-linking`` enables dynamic extensions
192 supports. It's currently known to crash in combination with threading.
193 - glibc extensions for date and time formatting are not available.
194 - ``locales`` module is affected by musl libc issues,
195 [bpo-46390](https://bugs.python.org/issue46390).
196 - Python's object allocator ``obmalloc`` is disabled by default.
197 - ``ensurepip`` is not available.
198 - Some ``ctypes`` features like ``c_longlong`` and ``c_longdouble`` may need
199 NodeJS option ``--experimental-wasm-bigint``.
201 ## wasm32-emscripten in browsers
203 - The interactive shell does not handle copy 'n paste and unicode support
205 - The bundled stdlib is limited. Network-related modules,
207 are not shipped. All other modules are bundled as pre-compiled
209 - In-memory file system (MEMFS) is not persistent and limited.
210 - Test modules are disabled by default. Use ``--enable-test-modules`` build
213 ## wasm32-emscripten in node
217 - Node RawFS allows direct access to the host file system without need to
220 ## wasm64-emscripten
222 - wasm64 requires recent NodeJS and ``--experimental-wasm-memory64``.
223 - ``EM_JS`` functions must return ``BigInt()``.
224 - ``Py_BuildValue()`` format strings must match size of types. Confusing 32
226 [gh-95876](https://github.com/python/cpython/issues/95876) and
227 [gh-95878](https://github.com/python/cpython/issues/95878).
229 # Hosting Python WASM builds
232 browsers only provide the feature in secure environents with cross-origin
233 isolation. The webserver must send cross-origin headers and correct MIME types
235 with an error message like ``Browsers disable shared array buffer``.
239 Place a ``.htaccess`` file in the same directory as ``python.wasm``.
243 Header set Cross-Origin-Opener-Policy same-origin
244 Header set Cross-Origin-Embedder-Policy require-corp
254 # WASI (wasm32-wasi)
256 WASI builds require [WASI SDK](https://github.com/WebAssembly/wasi-sdk) 16.0+.
258 ## Cross-compile to wasm32-wasi
260 The script ``wasi-env`` sets necessary compiler and linker flags as well as
261 ``pkg-config`` overrides. The script assumes that WASI-SDK is installed in
262 ``/opt/wasi-sdk`` or ``$WASI_SDK_PATH``.
271 mkdir -p builddir/wasi
274 CONFIG_SITE=../../Tools/wasm/config.site-wasm32-wasi \
275 ../../Tools/wasm/wasi-env ../../configure -C \
276 --host=wasm32-unknown-wasi \
277 --build=$(../../config.guess) \
278 --with-build-python=$(pwd)/../build/python
280 make -j$(nproc)
289 - Call stack size is limited. Default recursion limit and parser stack size
290 are smaller than in regular Python builds.
291 - ``socket(2)`` cannot create new socket file descriptors. WASI programs can
293 - ``socket.gethostname()`` and host name resolution APIs like
295 - ``open(2)`` checks flags more strictly. Caller must pass either
297 descriptors must be created with flags ``O_RDONLY | O_DIRECTORY``.
298 - ``chmod(2)`` is not available. It's not possible to modify file permissions,
300 - User/group related features like ``os.chown()``, ``os.getuid``, etc. are
301 stubs or fail with ``ENOTSUP``.
302 - File locking (``fcntl``) is not available.
303 - ``os.pipe()``, ``os.mkfifo()``, and ``os.mknod()`` are not supported.
304 - ``process_time`` does not work as expected because it's implemented using
306 - ``os.umask()`` is a stub.
307 - ``sys.executable`` is empty.
308 - ``/dev/null`` / ``os.devnull`` may not be available.
309 - ``os.utime*()`` is buggy in WASM SDK 15.0, see
310 …[utimensat() with timespec=NULL sets wrong time](https://github.com/bytecodealliance/wasmtime/issu…
311 - ``os.symlink()`` fails with ``PermissionError`` when attempting to create a
312 symlink with an absolute path with wasmtime 0.36.0. The wasmtime runtime
313 uses ``openat2(2)`` syscall with flag ``RESOLVE_BENEATH`` to open files.
314 The flag causes the syscall to reject symlinks with absolute paths.
315 - ``os.curdir`` (aka ``.``) seems to behave differently, which breaks some
318 - WASI runtime environments may not provide a dedicated temp directory.
323 ## Python code
325 ```python
329 # Python on Emscripten
331 # Python on WASI
334 # WASM platforms identify as POSIX-like.
341 ```python
364 ```python
374 ```python
392 Emscripten SDK and WASI SDK define several built-in macros. You can dump a
393 full list of built-ins with ``emcc -dM -E - < /dev/null`` and
394 ``/path/to/wasi-sdk/bin/clang -dM -E - < /dev/null``.
398 // Python on Emscripten
421 In some cases (e.g. build bots) you may prefer to install build dependencies
430 apt install -y git make xz-utils bzip2 curl python3-minimal ccache
435 dnf install -y git make xz bzip2 which ccache
440 **NOTE**: Follow the on-screen instructions how to add the SDK to ``PATH``.
443 git clone https://github.com/emscripten-core/emsdk.git /opt/emsdk
458 ### Optionally: pre-build and cache static libraries
461 (position-independent code). Python builds with ``dlopen`` support require
462 PIC. To populate the build cache, run:
466 embuilder build zlib bzip2 MINIMAL_PIC
467 embuilder build --pic zlib bzip2 MINIMAL_PIC
470 ### Install [WASI-SDK](https://github.com/WebAssembly/wasi-sdk)
472 **NOTE**: WASI-SDK's clang may show a warning on Fedora:
480 curl -sSf -L -O https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-${WASI_VERSION}/…
481 mkdir -p /opt/wasi-sdk
482 tar --strip-components=1 -C /opt/wasi-sdk -xvf wasi-sdk-${WASI_VERSION_FULL}-linux.tar.gz
483 rm -f wasi-sdk-${WASI_VERSION_FULL}-linux.tar.gz
491 curl -sSf -L -o ~/install-wasmtime.sh https://wasmtime.dev/install.sh
492 chmod +x ~/install-wasmtime.sh
493 ~/install-wasmtime.sh --version v0.38.0
494 ln -srf -t /usr/local/bin/ ~/.wasmtime/bin/wasmtime
500 * ``wasmtime run -g`` generates debugging symbols for gdb and lldb. The