1# Integrating and Running ExecuTorch on Apple Platforms 2 3**Author:** [Anthony Shoumikhin](https://github.com/shoumikhin) 4 5The ExecuTorch Runtime for iOS and macOS is distributed as a collection of prebuilt [.xcframework](https://developer.apple.com/documentation/xcode/creating-a-multi-platform-binary-framework-bundle) binary targets. These targets are compatible with both iOS and macOS devices and simulators and are available in both release and debug modes: 6 7* `executorch` - Main Runtime components 8* `backend_coreml` - Core ML backend 9* `backend_mps` - MPS backend 10* `backend_xnnpack` - XNNPACK backend 11* `kernels_custom` - Custom kernels for LLMs 12* `kernels_optimized` - Optimized kernels 13* `kernels_portable` - Portable kernels (naive implementation used as a reference) 14* `kernels_quantized` - Quantized kernels 15 16Link your binary with the ExecuTorch runtime and any backends or kernels used by the exported ML model. It is recommended to link the core runtime to the components that use ExecuTorch directly, and link kernels and backends against the main app target. 17 18**Note:** To access logs, link against the Debug build of the ExecuTorch runtime, i.e., the `executorch_debug` framework. For optimal performance, always link against the Release version of the deliverables (those without the `_debug` suffix), which have all logging overhead removed. 19 20## Integration 21 22### Setup 23 24#### CMake 25 26Building the Xcode project requires CMake. Installing via homebrew does not 27typically work; instead, install the packaged application and commandline tools 28globally: 29 301. Download the macOS `.dmg` installer from https://cmake.org/download 312. Open the `.dmg` 323. Drag the CMake app to the `/Applications` folder 334. In a terminal, install the command line tools: `sudo /Applications/CMake.app/Contents/bin/cmake-gui --install` 34 35### Swift Package Manager 36 37The prebuilt ExecuTorch runtime, backend, and kernels are available as a [Swift PM](https://www.swift.org/documentation/package-manager/) package. 38 39#### Xcode 40 41In Xcode, go to `File > Add Package Dependencies`. Paste the URL of the [ExecuTorch repo](https://github.com/pytorch/executorch) into the search bar and select it. Make sure to change the branch name to the desired ExecuTorch version in format "swiftpm-<version>", (e.g. "swiftpm-0.4.0"), or a branch name in format "swiftpm-<version>.<year_month_date>" (e.g. "swiftpm-0.4.0-20241114") for a nightly build on a specific date. 42 43 44 45Then select which ExecuTorch framework should link against which target. 46 47 48 49Click the screenshot below to watch the *demo video* on how to add the package and run a simple ExecuTorch model on iOS. 50 51<a href="https://pytorch.org/executorch/main/_static/img/swiftpm_xcode.mp4"> 52 <img src="https://pytorch.org/executorch/main/_static/img/swiftpm_xcode.png" width="800" alt="Integrating and Running ExecuTorch on Apple Platforms"> 53</a> 54 55#### CLI 56 57Add a package and target dependencies on ExecuTorch to your package file like this: 58 59```swift 60// swift-tools-version:5.0 61import PackageDescription 62 63let package = Package( 64 name: "YourPackageName", 65 products: [ 66 .library(name: "YourPackageName", targets: ["YourTargetName"]), 67 ], 68 dependencies: [ 69 // Use "swiftpm-0.4.0.<year_month_day>" branch name for a nightly build. 70 .package(url: "https://github.com/pytorch/executorch.git", .branch("swiftpm-0.4.0")) 71 ], 72 targets: [ 73 .target( 74 name: "YourTargetName", 75 dependencies: [ 76 .product(name: "executorch", package: "executorch"), 77 .product(name: "xnnpack_backend", package: "executorch") 78 ]), 79 ] 80) 81``` 82 83Then check if everything works correctly: 84 85```bash 86cd path/to/your/package 87 88swift package resolve 89 90# or just build it 91swift build 92``` 93 94### Local Build 95 96Another way to integrate the ExecuTorch runtime is to build the necessary components from sources locally and link against them. This route is more involved but certainly doable. 97 981. Install [Xcode](https://developer.apple.com/xcode/resources/) 15+ and Command Line Tools: 99 100```bash 101xcode-select --install 102``` 103 1042. Clone ExecuTorch: 105 106```bash 107git clone https://github.com/pytorch/executorch.git --depth 1 --recurse-submodules --shallow-submodules && cd executorch 108``` 109 1103. Set up [Python](https://www.python.org/downloads/macos/) 3.10+ and activate a virtual environment: 111 112```bash 113python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip 114``` 115 1164. Install the required dependencies, including those needed for the backends like [Core ML](build-run-coreml.md) or [MPS](build-run-mps.md), if you plan to build them as well: 117 118```bash 119./install_requirements.sh --pybind coreml mps xnnpack 120 121# Optional dependencies for Core ML backend. 122./backends/apple/coreml/scripts/install_requirements.sh 123 124# And MPS backend. 125./backends/apple/mps/install_requirements.sh 126``` 127 1285. Install [CMake](https://cmake.org): 129 130Download the macOS binary distribution from the [CMake website](https://cmake.org/download), open the `.dmg` file, move `CMake.app` to the `/Applications` directory, and then run the following command to install the CMake command-line tools: 131 132```bash 133sudo /Applications/CMake.app/Contents/bin/cmake-gui --install 134``` 135 1366. Use the provided script to build .xcframeworks: 137 138```bash 139./build/build_apple_frameworks.sh --help 140``` 141 142For example, the following invocation will build the ExecuTorch Runtime and all currently available kernels and backends for the Apple platform: 143 144```bash 145./build/build_apple_frameworks.sh --coreml --mps --xnnpack --custom --optimized --portable --quantized 146``` 147 148Append a `--Debug` flag to the above command to build the binaries with debug symbols if needed. 149 150After the build finishes successfully, the resulting frameworks can be found in the `cmake-out` directory. 151Copy them to your project and link them against your targets. 152 153## Linkage 154 155ExecuTorch initializes its backends and kernels (operators) during app startup by registering them in a static dictionary. If you encounter errors like "unregistered kernel" or "unregistered backend" at runtime, you may need to explicitly force-load certain components. Use the `-all_load` or `-force_load` linker flags in your Xcode build configuration to ensure components are registered early. 156 157Here's an example of a Xcode configuration file (`.xcconfig`): 158 159``` 160ET_PLATFORM[sdk=iphonesimulator*] = simulator 161ET_PLATFORM[sdk=iphoneos*] = ios 162ET_PLATFORM[sdk=macos*] = macos 163 164OTHER_LDFLAGS = $(inherited) \ 165 -force_load $(BUILT_PRODUCTS_DIR)/libexecutorch-$(ET_PLATFORM)-release.a \ 166 -force_load $(BUILT_PRODUCTS_DIR)/libbackend_coreml-$(ET_PLATFORM)-release.a \ 167 -force_load $(BUILT_PRODUCTS_DIR)/libbackend_mps-$(ET_PLATFORM)-release.a \ 168 -force_load $(BUILT_PRODUCTS_DIR)/libbackend_xnnpack-$(ET_PLATFORM)-release.a \ 169 -force_load $(BUILT_PRODUCTS_DIR)/libkernels_optimized-$(ET_PLATFORM)-release.a \ 170 -force_load $(BUILT_PRODUCTS_DIR)/libkernels_quantized-$(ET_PLATFORM)-release.a 171``` 172 173For a Debug build configuration, replace `release` with `debug` in the library file names. Remember to link against the ExecuTorch runtime (`libexecutorch`) in Debug mode even if other components are built for Release to preserve logs if needed. 174 175You can assign such a config file to your target in Xcode: 176 1771. Add the `.xcconfig` file to your project. 1782. Navigate to the project’s Info tab. 1793. Select the configuration file in the build configurations for Release (or Debug) mode. 180 181## Runtime API 182 183Check out the [C++ Runtime API](extension-module.md) and [Tensors](extension-tensor.md) tutorials to learn more about how to load and run an exported model. It is recommended to use the C++ API for macOS or iOS, wrapped with Objective-C++ and Swift code if needed to expose it for other components. Please refer to the [Demo App](demo-apps-ios.md) as an example of such a setup. 184 185Once linked against the `executorch` runtime framework, the target can now import all ExecuTorch public headers. For example, in Objective-C++: 186 187```objectivecpp 188#import <ExecuTorch/ExecuTorch.h> 189#import <executorch/extension/module/module.h> 190#import <executorch/extension/tensor/tensor.h> 191``` 192 193Or in Swift: 194 195```swift 196import ExecuTorch 197``` 198 199**Note:** Importing the ExecuTorch umbrella header (or ExecuTorch module in Swift) provides access to the logging API only. You still need to import the other runtime headers explicitly as needed, e.g., `module.h`. There is no support for other runtime APIs in Objective-C or Swift beyond logging described below. 200 201**Note:** Logs are stripped in the release builds of ExecuTorch frameworks. To preserve logging, use debug builds during development. 202 203### Logging 204 205We provide extra APIs for logging in Objective-C and Swift as a lightweight wrapper of the internal ExecuTorch machinery. To use it, just import the main framework header in Objective-C. Then use the `ExecuTorchLog` interface (or the `Log` class in Swift) to subscribe your own implementation of the `ExecuTorchLogSink` protocol (or `LogSink` in Swift) to listen to log events. 206 207```objectivec 208#import <ExecuTorch/ExecuTorch.h> 209#import <os/log.h> 210 211@interface MyClass : NSObject<ExecuTorchLogSink> 212@end 213 214@implementation MyClass 215 216- (instancetype)init { 217 self = [super init]; 218 if (self) { 219#if DEBUG 220 [ExecuTorchLog.sharedLog addSink:self]; 221#endif 222 } 223 return self; 224} 225 226- (void)dealloc { 227#if DEBUG 228 [ExecuTorchLog.sharedLog removeSink:self]; 229#endif 230} 231 232#if DEBUG 233- (void)logWithLevel:(ExecuTorchLogLevel)level 234 timestamp:(NSTimeInterval)timestamp 235 filename:(NSString *)filename 236 line:(NSUInteger)line 237 message:(NSString *)message { 238 NSString *logMessage = [NSString stringWithFormat:@"%@:%lu %@", filename, (unsigned long)line, message]; 239 switch (level) { 240 case ExecuTorchLogLevelDebug: 241 os_log_with_type(OS_LOG_DEFAULT, OS_LOG_TYPE_DEBUG, "%{public}@", logMessage); 242 break; 243 case ExecuTorchLogLevelInfo: 244 os_log_with_type(OS_LOG_DEFAULT, OS_LOG_TYPE_INFO, "%{public}@", logMessage); 245 break; 246 case ExecuTorchLogLevelError: 247 os_log_with_type(OS_LOG_DEFAULT, OS_LOG_TYPE_ERROR, "%{public}@", logMessage); 248 break; 249 case ExecuTorchLogLevelFatal: 250 os_log_with_type(OS_LOG_DEFAULT, OS_LOG_TYPE_FAULT, "%{public}@", logMessage); 251 break; 252 default: 253 os_log(OS_LOG_DEFAULT, "%{public}@", logMessage); 254 break; 255 } 256} 257#endif 258 259@end 260``` 261 262Swift version: 263 264```swift 265import ExecuTorch 266import os.log 267 268public class MyClass { 269 public init() { 270 #if DEBUG 271 Log.shared.add(sink: self) 272 #endif 273 } 274 deinit { 275 #if DEBUG 276 Log.shared.remove(sink: self) 277 #endif 278 } 279} 280 281#if DEBUG 282extension MyClass: LogSink { 283 public func log(level: LogLevel, timestamp: TimeInterval, filename: String, line: UInt, message: String) { 284 let logMessage = "\(filename):\(line) \(message)" 285 switch level { 286 case .debug: 287 os_log(.debug, "%{public}@", logMessage) 288 case .info: 289 os_log(.info, "%{public}@", logMessage) 290 case .error: 291 os_log(.error, "%{public}@", logMessage) 292 case .fatal: 293 os_log(.fault, "%{public}@", logMessage) 294 default: 295 os_log("%{public}@", logMessage) 296 } 297 } 298} 299#endif 300``` 301 302**Note:** In the example, the logs are intentionally stripped out when the code is not built for Debug mode, i.e., the `DEBUG` macro is not defined or equals zero. 303 304## Debugging 305 306If you are linking against a Debug build of the ExecuTorch frameworks, configure your debugger to map the source code correctly by using the following LLDB command in the debug session: 307 308``` 309settings append target.source-map /executorch <path_to_executorch_source_code> 310``` 311 312## Troubleshooting 313 314### Slow execution 315 316Ensure the exported model is using an appropriate backend, such as XNNPACK, Core ML, or MPS. If the correct backend is invoked but performance issues persist, confirm that you are linking against the Release build of the backend runtime. 317 318For optimal performance, link the ExecuTorch runtime in Release mode too. If debugging is needed, you can keep the ExecuTorch runtime in Debug mode with minimal impact on performance, but preserve logging and debug symbols. 319 320### Swift PM 321 322If you encounter a checksum mismatch error with Swift PM, clear the package cache using the Xcode menu (`File > Packages > Reset Package Caches`) or the following command: 323 324```bash 325rm -rf <YouProjectName>.xcodeproj/project.xcworkspace/xcshareddata/swiftpm \ 326 ~/Library/org.swift.swiftpm \ 327 ~/Library/Caches/org.swift.swiftpm \ 328 ~/Library/Caches/com.apple.dt.Xcode \ 329 ~/Library/Developer/Xcode/DerivedData 330``` 331**Note:** Ensure Xcode is fully quit before running the terminal command to avoid conflicts with active processes. 332
