1# Cross-Values 2`Cross-values` is a module that delivers implementation-defined constants (data structure sizes, offsets inside data structures, etc.), which is necessary for cross-compiling. For example, this allows generating binaries for `AArch64` on `x86-64` system. 3 4`Cross-values` consists of (and may refer to) the following objects: 5* A list of [*name*, *cpp-constant-expression*] pairs ([runtime/asm_defines/asm_defines.def](../runtime/asm_defines/asm_defines.def)); 6* An auto-generated file for each platform with constants' definition of format *"ptrdiff_t name = integer_literal;"* ([build/cross_values/generated_values/...](../build/cross_values/generated_values/))) 7 * *"integer_literal"* is equal to the constant expression on the corresponding platform; 8* An auto-generated file which contain a getter function for each value ([build/cross_values/cross_values.h](../build/cross_values/cross_values.h))); 9 * These getters take [*Arch*](../libpandabase/utils/arch.h) as an argument and return the value on that platform; 10## Contents 111. [Rationale](#rationale) 122. [Interface](#interface) 133. [Generated files example](#example-of-generated-cross-values) 144. [Principal mechanism](#principal-mechanism) 155. [Verification](#verification) 166. [Supported platforms](#supported-platforms) 177. [Sources](#sources) 18## Rationale 19`Cross-values` are intended to provide the following features: 20* A common interface for platform-dependent values (see [Interface](#interface)); 21* Minimalization of manual work: 22 * Changing of actual values (e.g. adding/removing new fields in data structures) will not break `cross-values`; 23 * To add a cross-value, it is enough to write a single line containing a name and a corresponding constant-expression; 24 * Regeneration is triggered automatically based on build dependencies; 25* Freedom of toolchain choice: 26 * Depending on the toolchain, data layout may differ even on the same platform; `cross-values` are designed with that in mind. 27 28Another possible approach is to keep the same fixed layout on each platform for every data structure but it has several disadvantages: 291. Bloat of data structures (especialy on 32-bit systems); 302. Need of providing guarantees that offsets are the same across all toolchains; 313. Less flexibility of the code, problems with maintaing; 32## Interface 331. `CMake`: 34 A set of variables to control targets' ".cmake"-toolchain: 35 * *PANDA_CROSS_X86_64_TOOLCHAIN_FILE* 36 * *PANDA_CROSS_AARCH64_TOOLCHAIN_FILE* 37 * *PANDA_CROSS_AARCH32_TOOLCHAIN_FILE* 381. `C++`: 39 To use `cross-values`: 40 1. Add desired value in [asm_defines.def](../runtime/asm_defines/asm_defines.def); 41 2. Include [cross_values.h](build/cross_values/cross_values.h); 42 3. Use the getter from [cross_values.h](build/cross_values/cross_values.h); 43 44Values' names should be in `UPPER_CASE`; the name of the getter will be transformed in `UpperSnakeCase` with `Get` prefix. For convenience, names should reflect objects/things they represent and also should contain part of namespaces to avoid ambiguity. 45```cpp 46// bar.h 47namespace foo { 48 class Bar 49 { 50 data_t baz_; 51 }; 52} // namespace foo 53``` 54 55```cpp 56// asm_defines.def 57#include "bar.h" 58 59DEFINE_VALUE(FOO_BAR_BAZ_SIZE, sizeof(foo::Bar::baz_)) 60DEFINE_VALUE(FOO_BAR_BAZ_OFFSET, offsetof(foo::Bar, baz_)) 61... 62``` 63```cpp 64// platform_dependent_code.cpp 65#include "cross_values.h" 66... 67 size_t size_x86_64 = panda::cross_values::GetFooBarBazSize(Arch::X86_64); 68 size_t offs_x86_64 = panda::cross_values::GetFooBarBazOffset(Arch::X86_64); 69 70 size_t size_aarch64 = panda::cross_values::GetFooBarBazSize(Arch::AARCH64); 71 size_t offs_aarch64 = panda::cross_values::GetFooBarBazOffset(Arch::AARCH64); 72... 73``` 74 75 76## Example of generated cross-values 77```cpp 78// X86_64_values_gen.h: 79namespace panda::cross_values::X86_64 { 80 static constexpr ptrdiff_t FOO_BAR_BAZ_SIZE_VAL = 8; 81 static constexpr ptrdiff_t FOO_BAR_BAZ_OFFSET_VAL = 0; 82} // namespace panda::cross_values::X86_64 83``` 84```cpp 85// AARCH64_values_gen.h: 86namespace panda::cross_values::AARCH64 { 87 static constexpr ptrdiff_t FOO_BAR_BAZ_SIZE_VAL = 8; 88 static constexpr ptrdiff_t FOO_BAR_BAZ_OFFSET_VAL = 0; 89} // namespace panda::cross_values::AARCH64 90``` 91```cpp 92// AARCH32_values_gen.h: 93namespace panda::cross_values::AARCH32 { 94 static constexpr ptrdiff_t FOO_BAR_BAZ_SIZE_VAL = 4; 95 static constexpr ptrdiff_t FOO_BAR_BAZ_OFFSET_VAL = 0; 96} // namespace panda::cross_values::AARCH32 97``` 98 99```cpp 100// cross_values.h: 101namespace panda::cross_values { 102... 103[[maybe_unused]] static constexpr ptrdiff_t GetFooBarBazSize(Arch arch) 104{ 105 switch (arch) { 106 case Arch::X86_64: 107 return cross_values::X86_64::FOO_BAR_BAZ_SIZE_VAL; 108 case Arch::AARCH64: 109 return cross_values::AARCH64::FOO_BAR_BAZ_SIZE_VAL; 110 case Arch::AARCH32: 111 return cross_values::AARCH32::FOO_BAR_BAZ_SIZE_VAL; 112 default: 113 LOG(FATAL, COMMON) << "No cross-values generated for " << GetStringFromArch(arch); 114 UNREACHABLE(); 115 } 116} 117... 118} // namespace panda::cross_values 119``` 120 121## Principal mechanism 122The idea is to 1231) Gather values ([asm_defines.def](../runtime/asm_defines/asm_defines.def)); 1242) Translate `constexpr` to literals using cross-compilers ([defines.cpp](../runtime/asm_defines/defines.cpp) -> `libasm_defines.S`); 1253) Generate '`.h`' file with the values' definition (via [cross_values_generator.rb](../cross_values/cross_values_generator.rb)); 1264) Generate getters (via [cross_values_getters_enerator.rb](../cross_values/cross_values_getters_generator.rb) -> `cross_values.h`); 1275) Include `cross_values.h` to use the values. 128 129It is allowed to choose an arbitrary cross-compiler (see [Interface](#interface)), `gcc-toolchain` is used by default. 130 131For each target an auxiliary binary directory with the project is configured with a single `asm_defines` target being built (see [CMake-schematic](#schematic-of-cmake-based-build)). These sub-builds are configured from the same sources as the primary build and added to dependency graph via CMake's `ExternalProject` so `cross-values`' regeneration is triggered automatically. 132 133#### Schematic of CMake-based build: 134``` 135 /-root_ark_build_dir---------------------\ 136 | cross_values/ | 137 | . auxiliary_binary_dirs/ |----+--->/-aarch32_toolchain_build_dir-\ 138 | . generated_values/ | | | runtime/ | 139 | . . test/ | | | . asm_defines/ | 140 | +-> . . . values_root_build_gen.h | | | . . libasm_defines.S |------+ 141+------------|-> . . AARCH32_values_gen.h-------+ | | \-----------------------------/ | 142| +----------|-> . . AARCH64_values_gen.h-------+ | | | 143| | +--------|-> . . X86_64_values_gen.h--------+ | | | 144| | | | | cross_values.h | | +--->/-aarch64_toolchain_build_dir-\ | 145| | | | | ^-----------------(#include)-+ | | | runtime/ | | 146| | | | | | | | . asm_defines/ | | 147| | | | | runtime/ | | | . . libasm_defines.S |----+ | 148| | | | | . asm_defines | | \-----------------------------/ | | 149| | | | | . . libasm_defines.S-----------+ | | | | 150| | | | | | | | | | 151| | | | +----------------------------------+ | | | | 152| | | | | +--->/-x86-64_toolchain_build_dir--\ | | 153| | | \----------------------------------------/ | runtime/ | | | 154| | | | . asm_defines/ | | | 155| | | | . . libasm_defines.S |--+ | | 156| | | \-----------------------------/ | | | 157| | | | | | 158| | | | | | 159| | +-----------------------------------------------------------------------------------------+ | | 160| +---------------------------------------------------------------------------------------------+ | 161+-------------------------------------------------------------------------------------------------+ 162``` 163Besides `CMake`, `GN` build is supported, with some restrictions. As `GN`-build is intended for target's bootstrapping, no cross-compilation is supported; `cross-values` are generated from the root `libasm_defines.S` and there are no any of `"auxiliary_binary_dirs"` (see [GN-schematic](#schematic-of-gn-based-build)). 164#### Schematic of GN-based build: 165``` 166/-root_ark_build_dir-(AArch64)------------\ 167| cross_values/ | 168| . generated_values/ | 169| +-> . . AARCH64_values_gen.h--------+ | 170| | cross_values.h | | 171| | ^------------------(#include)-+ | 172| | | 173| | runtime/ | 174| | . asm_defines | 175| | . . libasm_defines.S------------+ | 176| | | | 177| +-----------------------------------+ | 178| | 179\-----------------------------------------/ 180``` 181## Verification 182As it was stated, even on the same platform the values may differ depending on the toolchain. To detect such situation, a `CRC-32` checksum is generated based on '`.h`'-files with values (for each architecture), stored in aot- and runtime-binaries and compared on loading of aot-files (see [entrypoints_compiler_checksum.inl.erb](../runtime/entrypoints/entrypoints_compiler_checksum.inl.erb)). 183 184There are also plain diff-compares of values that should prevent errors caused by the tangled `CMake`-configuration: 1851. between target's `cross-values` and host-tools' `cross-values` (see [HostTools.cmake](../cmake/HostTools.cmake)); 1862. between `values_root_build_gen.h` (which is similiar to `cross-values`, but generated from the root `libasm_defines.S`, see [CMake-schematic](#schematic-of-cmake-based-build)) and `cross-values` for the corresponding architecture. 187 188 189## Supported platforms 190 191Currently, the following architectures are supported: 192* x86-64 193* AArch64 194* AArch32 195 196Cross-compilation is available only for x86-64. 197 198## Sources 199* [cross_values/BUILD.gn](../cross_values/BUILD.gn) 200* [cross_values/CMakeLists.txt](../cross_values/CMakeLists.txt) 201* [cross_values/cross_values_generator.rb](../cross_values/cross_values_generator.rb) 202* [cross_values/cross_values_getters_generator.rb](../cross_values/cross_values_getters_generator.rb) 203* [cross_values/diff_check_values.sh](../cross_values/diff_check_values.sh) 204* [cross_values/wrap_cross_values_getters_generator.sh](../cross_values/wrap_cross_values_getters_generator.sh) 205* [runtime/asm_defines/asm_defines.def](../runtime/asm_defines/asm_defines.def) 206