1# Portable C++ Programming 2 3NOTE: This document covers the code that needs to build for and execute in 4target hardware environments. This applies to the core execution runtime, as 5well as kernel and backend implementations in this repo. These rules do not 6necessarily apply to code that only runs on the development host, like authoring 7or build tools. 8 9The ExecuTorch runtime code is intendend to be portable, and should build for a 10wide variety of systems, from servers to mobile phones to DSPs, from POSIX to 11Windows to bare-metal environments. 12 13This means that it can't assume the existence of: 14- Files 15- Threads 16- Exceptions 17- `stdout`, `stderr` 18- `printf()`, `fprintf()` 19- POSIX APIs and concepts in general 20 21It also can't assume: 22- 64 bit pointers 23- The size of a given integer type 24- The signedness of `char` 25 26To keep the binary size to a minimum, and to keep tight control over memory 27allocation, the code may not use: 28- `malloc()`, `free()` 29- `new`, `delete` 30- Most `stdlibc++` types; especially container types that manage their own 31 memory like `string` and `vector`, or memory-management wrapper types like 32 `unique_ptr` and `shared_ptr`. 33 34And to help reduce complexity, the code may not depend on any external 35dependencies except: 36- `flatbuffers` (for `.pte` file deserialization) 37- `flatcc` (for event trace serialization) 38- Core PyTorch (only for ATen mode) 39 40## Platform Abstraction Layer (PAL) 41 42To avoid assuming the capabilities of the target system, the ExecuTorch runtime 43lets clients override low-level functions in its Platform Abstraction Layer 44(PAL), defined in `//executorch/runtime/platform/platform.h`, to perform operations 45like: 46- Getting the current timestamp 47- Printing a log message 48- Panicking the system 49 50## Memory Allocation 51 52Instead of using `malloc()` or `new`, the runtime code should allocate memory 53using the `MemoryManager` (`//executorch/runtime/executor/memory_manager.h`) 54provided by the client. 55 56## File Loading 57 58Instead of loading files directly, clients should provide buffers with the data 59already loaded, or wrapped in types like `DataLoader`. 60 61## Integer Types 62 63ExecuTorch runtime code should not assume anything about the sizes of primitive 64types like `int`, `short`, or `char`. For example, the C++ standard only 65guarantees that `int` will be at least 16 bits wide. And ARM toolchains treat 66`char` as unsigned, while other toolchains often treat it as signed. 67 68Instead, the runtime APIs use a set of more predictable, but still standard, 69integer types: 70- `<cstdint>` types like `uint64_t`, `int32_t`; these types guarantee the bit 71 width and signedness, regardless of the architecture. Use these types when you 72 need a very specific integer width. 73- `size_t` for counts of things, or memory offsets. `size_t` is guaranteed to be 74 big enough to represent any memory byte offset; i.e., it will be as wide as 75 the native pointer type for the target system. Prefer using this instead of 76 `uint64_t` for counts/offsets so that 32-bit systems don't need to pay for the 77 unnecessary overhead of a 64-bit value. 78- `ssize_t` for some ATen-compatibility situations where `Tensor` returns a 79 signed count. Prefer `size_t` when possible. 80 81## Floating Point Arithmetic 82 83Not every system has support for floating point arithmetic: some don't even enable 84floating point emulation in their toolchains. Therefore, the core runtime code 85must not perform any floating point arithmetic at runtime, although it is ok to 86simply create or manage `float` or `double` values (e.g., in an `EValue`). 87 88Kernels, being outside of the core runtime, are allowed to perform floating point 89arithmetic. Though some kernels may choose not to, so that they can run on systems 90without floating point support. 91 92## Logging 93 94Instead of using `printf()`, `fprintf()`, `cout`, `cerr`, or a library like 95`folly::logging` or `glog`, the ExecuTorch runtime provides the `ET_LOG` 96interface in `//executorch/runtime/platform/log.h` and the `ET_CHECK` interface in 97`//executorch/runtime/platform/assert.h`. The messages are printed using a hook in the PAL, 98which means that clients can redirect them to any underlying logging system, or 99just print them to `stderr` if available. 100 101### Logging Format Portability 102 103#### Fixed-Width Integers 104 105When you have a log statement like 106``` 107int64_t value; 108ET_LOG(Error, "Value %??? is bad", value); 109``` 110what should you put for the `%???` part, to match the `int64_t`? On different 111systems, the `int64_t` typdef might be `int`, `long int`, or `long long int`. 112Picking a format like `%d`, `%ld`, or `%lld` might work on one target, but break 113on the others. 114 115To be portable, the runtime code uses the standard (but admittedly awkward) 116helper macros from `<cinttypes>`. Each portable integer type has a corresponding 117`PRIn##` macro, like 118- `int32_t` -> `PRId32` 119- `uint32_t` -> `PRIu32` 120- `int64_t` -> `PRId64` 121- `uint64_t` -> `PRIu64` 122- See https://en.cppreference.com/w/cpp/header/cinttypes for more 123 124These macros are literal strings that can concatenate with other parts of the 125format string, like 126``` 127int64_t value; 128ET_LOG(Error, "Value %" PRId64 " is bad", value); 129``` 130Note that this requires chopping up the literal format string (the extra double 131quotes). It also requires the leading `%` before the macro. 132 133But, by using these macros, you're guaranteed that the toolchain will use the 134appropriate format pattern for the type. 135 136#### `size_t`, `ssize_t` 137 138Unlike the fixed-width integer types, format strings already have a portable 139way to handle `size_t` and `ssize_t`: 140- `size_t` -> `%zu` 141- `ssize_t` -> `%zd` 142 143#### Casting 144 145Sometimes, especially in code that straddles ATen and lean mode, the type of the 146value itself might be different across build modes. In those cases, cast the 147value to the lean mode type, like: 148``` 149ET_CHECK_MSG( 150 input.dim() == output.dim(), 151 "input.dim() %zd not equal to output.dim() %zd", 152 (ssize_t)input.dim(), 153 (ssize_t)output.dim()); 154``` 155In this case, `Tensor::dim()` returns `ssize_t` in lean mode, while 156`at::Tensor::dim()` returns `int64_t` in ATen mode. Since they both conceptually 157return (signed) counts, `ssize_t` is the most appropriate integer type. 158`int64_t` would work, but it would unnecessarily require 32-bit systems to deal 159with a 64-bit value in lean mode. 160 161This is the only situation where casting should be necessary, when lean and ATen 162modes disagree. Otherwise, use the format pattern that matches the type. 163