1==================== 2Python on iOS README 3==================== 4 5:Authors: 6 Russell Keith-Magee (2023-11) 7 8This document provides a quick overview of some iOS specific features in the 9Python distribution. 10 11These instructions are only needed if you're planning to compile Python for iOS 12yourself. Most users should *not* need to do this. If you're looking to 13experiment with writing an iOS app in Python, tools such as `BeeWare's Briefcase 14<https://briefcase.readthedocs.io>`__ and `Kivy's Buildozer 15<https://buildozer.readthedocs.io>`__ will provide a much more approachable 16user experience. 17 18Compilers for building on iOS 19============================= 20 21Building for iOS requires the use of Apple's Xcode tooling. It is strongly 22recommended that you use the most recent stable release of Xcode. This will 23require the use of the most (or second-most) recently released macOS version, 24as Apple does not maintain Xcode for older macOS versions. The Xcode Command 25Line Tools are not sufficient for iOS development; you need a *full* Xcode 26install. 27 28If you want to run your code on the iOS simulator, you'll also need to install 29an iOS Simulator Platform. You should be prompted to select an iOS Simulator 30Platform when you first run Xcode. Alternatively, you can add an iOS Simulator 31Platform by selecting an open the Platforms tab of the Xcode Settings panel. 32 33iOS specific arguments to configure 34=================================== 35 36* ``--enable-framework[=DIR]`` 37 38 This argument specifies the location where the Python.framework will be 39 installed. If ``DIR`` is not specified, the framework will be installed into 40 a subdirectory of the ``iOS/Frameworks`` folder. 41 42 This argument *must* be provided when configuring iOS builds. iOS does not 43 support non-framework builds. 44 45* ``--with-framework-name=NAME`` 46 47 Specify the name for the Python framework; defaults to ``Python``. 48 49 .. admonition:: Use this option with care! 50 51 Unless you know what you're doing, changing the name of the Python 52 framework on iOS is not advised. If you use this option, you won't be able 53 to run the ``make testios`` target without making significant manual 54 alterations, and you won't be able to use any binary packages unless you 55 compile them yourself using your own framework name. 56 57Building Python on iOS 58====================== 59 60ABIs and Architectures 61---------------------- 62 63iOS apps can be deployed on physical devices, and on the iOS simulator. Although 64the API used on these devices is identical, the ABI is different - you need to 65link against different libraries for an iOS device build (``iphoneos``) or an 66iOS simulator build (``iphonesimulator``). 67 68Apple uses the ``XCframework`` format to allow specifying a single dependency 69that supports multiple ABIs. An ``XCframework`` is a wrapper around multiple 70ABI-specific frameworks that share a common API. 71 72iOS can also support different CPU architectures within each ABI. At present, 73there is only a single supported architecture on physical devices - ARM64. 74However, the *simulator* supports 2 architectures - ARM64 (for running on Apple 75Silicon machines), and x86_64 (for running on older Intel-based machines). 76 77To support multiple CPU architectures on a single platform, Apple uses a "fat 78binary" format - a single physical file that contains support for multiple 79architectures. It is possible to compile and use a "thin" single architecture 80version of a binary for testing purposes; however, the "thin" binary will not be 81portable to machines using other architectures. 82 83Building a single-architecture framework 84---------------------------------------- 85 86The Python build system will create a ``Python.framework`` that supports a 87*single* ABI with a *single* architecture. Unlike macOS, iOS does not allow a 88framework to contain non-library content, so the iOS build will produce a 89``bin`` and ``lib`` folder in the same output folder as ``Python.framework``. 90The ``lib`` folder will be needed at runtime to support the Python library. 91 92If you want to use Python in a real iOS project, you need to produce multiple 93``Python.framework`` builds, one for each ABI and architecture. iOS builds of 94Python *must* be constructed as framework builds. To support this, you must 95provide the ``--enable-framework`` flag when configuring the build. The build 96also requires the use of cross-compilation. The minimal commands for building 97Python for the ARM64 iOS simulator will look something like:: 98 99 $ export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" 100 $ ./configure \ 101 --enable-framework \ 102 --host=arm64-apple-ios-simulator \ 103 --build=arm64-apple-darwin \ 104 --with-build-python=/path/to/python.exe 105 $ make 106 $ make install 107 108In this invocation: 109 110* ``iOS/Resources/bin`` has been added to the path, providing some shims for the 111 compilers and linkers needed by the build. Xcode requires the use of ``xcrun`` 112 to invoke compiler tooling. However, if ``xcrun`` is pre-evaluated and the 113 result passed to ``configure``, these results can embed user- and 114 version-specific paths into the sysconfig data, which limits the portability 115 of the compiled Python. Alternatively, if ``xcrun`` is used *as* the compiler, 116 it requires that compiler variables like ``CC`` include spaces, which can 117 cause significant problems with many C configuration systems which assume that 118 ``CC`` will be a single executable. 119 120 To work around this problem, the ``iOS/Resources/bin`` folder contains some 121 wrapper scripts that present as simple compilers and linkers, but wrap 122 underlying calls to ``xcrun``. This allows configure to use a ``CC`` 123 definition without spaces, and without user- or version-specific paths, while 124 retaining the ability to adapt to the local Xcode install. These scripts are 125 included in the ``bin`` directory of an iOS install. 126 127 These scripts will, by default, use the currently active Xcode installation. 128 If you want to use a different Xcode installation, you can use 129 ``xcode-select`` to set a new default Xcode globally, or you can use the 130 ``DEVELOPER_DIR`` environment variable to specify an Xcode install. The 131 scripts will use the default ``iphoneos``/``iphonesimulator`` SDK version for 132 the select Xcode install; if you want to use a different SDK, you can set the 133 ``IOS_SDK_VERSION`` environment variable. (e.g, setting 134 ``IOS_SDK_VERSION=17.1`` would cause the scripts to use the ``iphoneos17.1`` 135 and ``iphonesimulator17.1`` SDKs, regardless of the Xcode default.) 136 137 The path has also been cleared of any user customizations. A common source of 138 bugs is for tools like Homebrew to accidentally leak macOS binaries into an iOS 139 build. Resetting the path to a known "bare bones" value is the easiest way to 140 avoid these problems. 141 142* ``--host`` is the architecture and ABI that you want to build, in GNU compiler 143 triple format. This will be one of: 144 145 - ``arm64-apple-ios`` for ARM64 iOS devices. 146 - ``arm64-apple-ios-simulator`` for the iOS simulator running on Apple 147 Silicon devices. 148 - ``x86_64-apple-ios-simulator`` for the iOS simulator running on Intel 149 devices. 150 151* ``--build`` is the GNU compiler triple for the machine that will be running 152 the compiler. This is one of: 153 154 - ``arm64-apple-darwin`` for Apple Silicon devices. 155 - ``x86_64-apple-darwin`` for Intel devices. 156 157* ``/path/to/python.exe`` is the path to a Python binary on the machine that 158 will be running the compiler. This is needed because the Python compilation 159 process involves running some Python code. On a normal desktop build of 160 Python, you can compile a python interpreter and then use that interpreter to 161 run Python code. However, the binaries produced for iOS won't run on macOS, so 162 you need to provide an external Python interpreter. This interpreter must be 163 the same version as the Python that is being compiled. To be completely safe, 164 this should be the *exact* same commit hash. However, the longer a Python 165 release has been stable, the more likely it is that this constraint can be 166 relaxed - the same micro version will often be sufficient. 167 168* The ``install`` target for iOS builds is slightly different to other 169 platforms. On most platforms, ``make install`` will install the build into 170 the final runtime location. This won't be the case for iOS, as the final 171 runtime location will be on a physical device. 172 173 However, you still need to run the ``install`` target for iOS builds, as it 174 performs some final framework assembly steps. The location specified with 175 ``--enable-framework`` will be the location where ``make install`` will 176 assemble the complete iOS framework. This completed framework can then 177 be copied and relocated as required. 178 179For a full CPython build, you also need to specify the paths to iOS builds of 180the binary libraries that CPython depends on (XZ, BZip2, LibFFI and OpenSSL). 181This can be done by defining the ``LIBLZMA_CFLAGS``, ``LIBLZMA_LIBS``, 182``BZIP2_CFLAGS``, ``BZIP2_LIBS``, ``LIBFFI_CFLAGS``, and ``LIBFFI_LIBS`` 183environment variables, and the ``--with-openssl`` configure option. Versions of 184these libraries pre-compiled for iOS can be found in `this repository 185<https://github.com/beeware/cpython-apple-source-deps/releases>`__. LibFFI is 186especially important, as many parts of the standard library (including the 187``platform``, ``sysconfig`` and ``webbrowser`` modules) require the use of the 188``ctypes`` module at runtime. 189 190By default, Python will be compiled with an iOS deployment target (i.e., the 191minimum supported iOS version) of 13.0. To specify a different deployment 192target, provide the version number as part of the ``--host`` argument - for 193example, ``--host=arm64-apple-ios15.4-simulator`` would compile an ARM64 194simulator build with a deployment target of 15.4. 195 196Merge thin frameworks into fat frameworks 197----------------------------------------- 198 199Once you've built a ``Python.framework`` for each ABI and and architecture, you 200must produce a "fat" framework for each ABI that contains all the architectures 201for that ABI. 202 203The ``iphoneos`` build only needs to support a single architecture, so it can be 204used without modification. 205 206If you only want to support a single simulator architecture, (e.g., only support 207ARM64 simulators), you can use a single architecture ``Python.framework`` build. 208However, if you want to create ``Python.xcframework`` that supports *all* 209architectures, you'll need to merge the ``iphonesimulator`` builds for ARM64 and 210x86_64 into a single "fat" framework. 211 212The "fat" framework can be constructed by performing a directory merge of the 213content of the two "thin" ``Python.framework`` directories, plus the ``bin`` and 214``lib`` folders for each thin framework. When performing this merge: 215 216* The pure Python standard library content is identical for each architecture, 217 except for a handful of platform-specific files (such as the ``sysconfig`` 218 module). Ensure that the "fat" framework has the union of all standard library 219 files. 220 221* Any binary files in the standard library, plus the main 222 ``libPython3.X.dylib``, can be merged using the ``lipo`` tool, provide by 223 Xcode:: 224 225 $ lipo -create -output module.dylib path/to/x86_64/module.dylib path/to/arm64/module.dylib 226 227* The header files will be identical on both architectures, except for 228 ``pyconfig.h``. Copy all the headers from one platform (say, arm64), rename 229 ``pyconfig.h`` to ``pyconfig-arm64.h``, and copy the ``pyconfig.h`` for the 230 other architecture into the merged header folder as ``pyconfig-x86_64.h``. 231 Then copy the ``iOS/Resources/pyconfig.h`` file from the CPython sources into 232 the merged headers folder. This will allow the two Python architectures to 233 share a common ``pyconfig.h`` header file. 234 235At this point, you should have 2 Python.framework folders - one for ``iphoneos``, 236and one for ``iphonesimulator`` that is a merge of x86+64 and ARM64 content. 237 238Merge frameworks into an XCframework 239------------------------------------ 240 241Now that we have 2 (potentially fat) ABI-specific frameworks, we can merge those 242frameworks into a single ``XCframework``. 243 244The initial skeleton of an ``XCframework`` is built using:: 245 246 xcodebuild -create-xcframework -output Python.xcframework -framework path/to/iphoneos/Python.framework -framework path/to/iphonesimulator/Python.framework 247 248Then, copy the ``bin`` and ``lib`` folders into the architecture-specific slices of 249the XCframework:: 250 251 cp path/to/iphoneos/bin Python.xcframework/ios-arm64 252 cp path/to/iphoneos/lib Python.xcframework/ios-arm64 253 254 cp path/to/iphonesimulator/bin Python.xcframework/ios-arm64_x86_64-simulator 255 cp path/to/iphonesimulator/lib Python.xcframework/ios-arm64_x86_64-simulator 256 257Note that the name of the architecture-specific slice for the simulator will 258depend on the CPU architecture(s) that you build. 259 260You now have a Python.xcframework that can be used in a project. 261 262Testing Python on iOS 263===================== 264 265The ``iOS/testbed`` folder that contains an Xcode project that is able to run 266the iOS test suite. This project converts the Python test suite into a single 267test case in Xcode's XCTest framework. The single XCTest passes if the test 268suite passes. 269 270To run the test suite, configure a Python build for an iOS simulator (i.e., 271``--host=arm64-apple-ios-simulator`` or ``--host=x86_64-apple-ios-simulator`` 272), specifying a framework build (i.e. ``--enable-framework``). Ensure that your 273``PATH`` has been configured to include the ``iOS/Resources/bin`` folder and 274exclude any non-iOS tools, then run:: 275 276 $ make all 277 $ make install 278 $ make testios 279 280This will: 281 282* Build an iOS framework for your chosen architecture; 283* Finalize the single-platform framework; 284* Make a clean copy of the testbed project; 285* Install the Python iOS framework into the copy of the testbed project; and 286* Run the test suite on an "iPhone SE (3rd generation)" simulator. 287 288While the test suite is running, Xcode does not display any console output. 289After showing some Xcode build commands, the console output will print ``Testing 290started``, and then appear to stop. It will remain in this state until the test 291suite completes. On a 2022 M1 MacBook Pro, the test suite takes approximately 12 292minutes to run; a couple of extra minutes is required to boot and prepare the 293iOS simulator. 294 295On success, the test suite will exit and report successful completion of the 296test suite. No output of the Python test suite will be displayed. 297 298On failure, the output of the Python test suite *will* be displayed. This will 299show the details of the tests that failed. 300 301Debugging test failures 302----------------------- 303 304The easiest way to diagnose a single test failure is to open the testbed project 305in Xcode and run the tests from there using the "Product > Test" menu item. 306 307To test in Xcode, you must ensure the testbed project has a copy of a compiled 308framework. If you've configured your build with the default install location of 309``iOS/Frameworks``, you can copy from that location into the test project. To 310test on an ARM64 simulator, run:: 311 312 $ rm -rf iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator/* 313 $ cp -r iOS/Frameworks/arm64-iphonesimulator/* iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator 314 315To test on an x86-64 simulator, run:: 316 317 $ rm -rf iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator/* 318 $ cp -r iOS/Frameworks/x86_64-iphonesimulator/* iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator 319 320To test on a physical device:: 321 322 $ rm -rf iOS/testbed/Python.xcframework/ios-arm64/* 323 $ cp -r iOS/Frameworks/arm64-iphoneos/* iOS/testbed/Python.xcframework/ios-arm64 324 325Alternatively, you can configure your build to install directly into the 326testbed project. For a simulator, use:: 327 328 --enable-framework=$(pwd)/iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator 329 330For a physical device, use:: 331 332 --enable-framework=$(pwd)/iOS/testbed/Python.xcframework/ios-arm64 333 334 335Testing on an iOS device 336^^^^^^^^^^^^^^^^^^^^^^^^ 337 338To test on an iOS device, the app needs to be signed with known developer 339credentials. To obtain these credentials, you must have an iOS Developer 340account, and your Xcode install will need to be logged into your account (see 341the Accounts tab of the Preferences dialog). 342 343Once the project is open, and you're signed into your Apple Developer account, 344select the root node of the project tree (labeled "iOSTestbed"), then the 345"Signing & Capabilities" tab in the details page. Select a development team 346(this will likely be your own name), and plug in a physical device to your 347macOS machine with a USB cable. You should then be able to select your physical 348device from the list of targets in the pulldown in the Xcode titlebar. 349 350Running specific tests 351^^^^^^^^^^^^^^^^^^^^^^ 352 353As the test suite is being executed on an iOS simulator, it is not possible to 354pass in command line arguments to configure test suite operation. To work 355around this limitation, the arguments that would normally be passed as command 356line arguments are configured as part of the ``iOSTestbed-Info.plist`` file 357that is used to configure the iOS testbed app. In this file, the ``TestArgs`` 358key is an array containing the arguments that would be passed to ``python -m`` 359on the command line (including ``test`` in position 0, the name of the test 360module to be executed). 361 362Disabling automated breakpoints 363^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 364 365By default, Xcode will inserts an automatic breakpoint whenever a signal is 366raised. The Python test suite raises many of these signals as part of normal 367operation; unless you are trying to diagnose an issue with signals, the 368automatic breakpoints can be inconvenient. However, they can be disabled by 369creating a symbolic breakpoint that is triggered at the start of the test run. 370 371Select "Debug > Breakpoints > Create Symbolic Breakpoint" from the Xcode menu, and 372populate the new brewpoint with the following details: 373 374* **Name**: IgnoreSignals 375* **Symbol**: UIApplicationMain 376* **Action**: Add debugger commands for: 377 - ``process handle SIGINT -n true -p true -s false`` 378 - ``process handle SIGUSR1 -n true -p true -s false`` 379 - ``process handle SIGUSR2 -n true -p true -s false`` 380 - ``process handle SIGXFSZ -n true -p true -s false`` 381* Check the "Automatically continue after evaluating" box. 382 383All other details can be left blank. When the process executes the 384``UIApplicationMain`` entry point, the breakpoint will trigger, run the debugger 385commands to disable the automatic breakpoints, and automatically resume. 386