1=================================================================== 2How to Cross Compile Compiler-rt Builtins For Arm 3=================================================================== 4 5Introduction 6============ 7 8This document contains information about building and testing the builtins part 9of compiler-rt for an Arm target, from an x86_64 Linux machine. 10 11While this document concentrates on Arm and Linux the general principles should 12apply to other targets supported by compiler-rt. Further contributions for other 13targets are welcome. 14 15The instructions in this document depend on libraries and programs external to 16LLVM, there are many ways to install and configure these dependencies so you 17may need to adapt the instructions here to fit your own local situation. 18 19Prerequisites 20============= 21 22In this use case we'll be using CMake on a Debian-based Linux system, 23cross-compiling from an x86_64 host to a hard-float Armv7-A target. We'll be 24using as many of the LLVM tools as we can, but it is possible to use GNU 25equivalents. 26 27 * ``A build of LLVM/clang for the llvm-tools and llvm-config`` 28 * ``The qemu-arm user mode emulator`` 29 * ``An arm-linux-gnueabihf sysroot`` 30 31See https://compiler-rt.llvm.org/ for more information about the dependencies 32on clang and LLVM. 33 34``qemu-arm`` should be available as a package for your Linux distribution. 35 36The most complicated of the prequisites to satisfy is the arm-linux-gnueabihf 37sysroot. The :doc:`HowToCrossCompileLLVM` has information about how to use the 38Linux distributions multiarch support to fulfill the dependencies for building 39LLVM. Alternatively, as building and testing just the compiler-rt builtins 40requires fewer dependencies than LLVM, it is possible to use the Linaro 41arm-linux-gnueabihf gcc installation as our sysroot. 42 43Building compiler-rt builtins for Arm 44===================================== 45We will be doing a standalone build of compiler-rt using the following cmake 46options. 47 48* ``path/to/llvm/projects/compiler-rt`` 49* ``-DCOMPILER_RT_BUILD_BUILTINS=ON`` 50* ``-DCOMPILER_RT_BUILD_SANITIZERS=OFF`` 51* ``-DCOMPILER_RT_BUILD_XRAY=OFF`` 52* ``-DCOMPILER_RT_BUILD_LIBFUZZER=OFF`` 53* ``-DCOMPILER_RT_BUILD_PROFILE=OFF`` 54* ``-DCMAKE_C_COMPILER=/path/to/clang`` 55* ``-DCMAKE_AR=/path/to/llvm-ar`` 56* ``-DCMAKE_NM=/path/to/llvm-nm`` 57* ``-DCMAKE_RANLIB=/path/to/llvm-ranlib`` 58* ``-DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld"`` 59* ``-DCMAKE_C_COMPILER_TARGET="arm-linux-gnueabihf"`` 60* ``-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON`` 61* ``-DLLVM_CONFIG_PATH=/path/to/llvm-config`` 62* ``-DCMAKE_C_FLAGS="build-c-flags"`` 63 64The build-c-flags need to be sufficient to pass the C-make compiler check and 65to compile compiler-rt. When using a GCC 7 Linaro arm-linux-gnueabihf 66installation the following flags are needed: 67 68* ``--target=arm-linux-gnueabihf`` 69* ``--march=armv7a`` 70* ``--gcc-toolchain=/path/to/dir/toolchain`` 71* ``--sysroot=/path/to/toolchain/arm-linux-gnueabihf/libc`` 72 73Depending on how your sysroot is laid out, you may not need ``--gcc-toolchain``. 74For example if you have added armhf as an architecture using your Linux 75distributions multiarch support then you should be able to use ``--sysroot=/``. 76 77Once cmake has completed the builtins can be built with ``ninja builtins`` 78 79Testing compiler-rt builtins using qemu-arm 80=========================================== 81To test the builtins library we need to add a few more cmake flags to enable 82testing and set up the compiler and flags for test case. We must also tell 83cmake that we wish to run the tests on ``qemu-arm``. 84 85* ``-DCOMPILER_RT_EMULATOR="qemu-arm -L /path/to/armhf/sysroot`` 86* ``-DCOMPILER_RT_INCLUDE_TESTS=ON`` 87* ``-DCOMPILER_RT_TEST_COMPILER="/path/to/clang"`` 88* ``-DCOMPILER_RT_TEST_COMPILER_CFLAGS="test-c-flags"`` 89 90The ``/path/to/armhf/sysroot`` should be the same as the one passed to 91``--sysroot`` in the "build-c-flags". 92 93The "test-c-flags" can be the same as the "build-c-flags", with the addition 94of ``"-fuse-ld=lld`` if you wish to use lld to link the tests. 95 96Once cmake has completed the tests can be built and run using 97``ninja check-builtins`` 98 99Modifications for other Targets 100=============================== 101 102Arm Soft-Float Target 103--------------------- 104The instructions for the Arm hard-float target can be used for the soft-float 105target by substituting soft-float equivalents for the sysroot and target. The 106target to use is: 107 108* ``-DCMAKE_C_COMPILER_TARGET=arm-linux-gnueabi`` 109 110Depending on whether you want to use floating point instructions or not you 111may need extra c-flags such as ``-mfloat-abi=softfp`` for use of floating-point 112instructions, and ``-mfloat-abi=soft -mfpu=none`` for software floating-point 113emulation. 114 115AArch64 Target 116-------------- 117The instructions for Arm can be used for AArch64 by substituting AArch64 118equivalents for the sysroot, emulator and target. 119 120* ``-DCMAKE_C_COMPILER_TARGET=aarch64-linux-gnu`` 121* ``-DCOMPILER_RT_EMULATOR="qemu-aarch64 -L /path/to/aarch64/sysroot`` 122 123The CMAKE_C_FLAGS and COMPILER_RT_TEST_COMPILER_CFLAGS may also need: 124``"--sysroot=/path/to/aarch64/sysroot --gcc-toolchain=/path/to/gcc-toolchain"`` 125 126Armv6-m, Armv7-m and Armv7E-M targets 127------------------------------------- 128If you wish to build, but not test compiler-rt for Armv6-M, Armv7-M or Armv7E-M 129then the easiest way is to use the BaremetalARM.cmake recipe in 130clang/cmake/caches. 131 132You will need a bare metal sysroot such as that provided by the GNU ARM 133Embedded toolchain. 134 135The libraries can be built with the cmake options: 136 137* ``-DBAREMETAL_ARMV6M_SYSROOT=/path/to/bare/metal/sysroot`` 138* ``-DBAREMETAL_ARMV7M_SYSROOT=/path/to/bare/metal/sysroot`` 139* ``-DBAREMETAL_ARMV7EM_SYSROOT=/path/to/bare/metal/sysroot`` 140* ``-C /path/to/llvm/source/tools/clang/cmake/caches/BaremetalARM.cmake`` 141 142**Note** that for the recipe to work the compiler-rt source must be checked out 143into the directory llvm/runtimes and not llvm/projects. 144 145To build and test the libraries using a similar method to Armv7-A is possible 146but more difficult. The main problems are: 147 148* There isn't a ``qemu-arm`` user-mode emulator for bare-metal systems. The ``qemu-system-arm`` can be used but this is significantly more difficult to setup. 149* The target to compile compiler-rt have the suffix -none-eabi. This uses the BareMetal driver in clang and by default won't find the libraries needed to pass the cmake compiler check. 150 151As the Armv6-M, Armv7-M and Armv7E-M builds of compiler-rt only use instructions 152that are supported on Armv7-A we can still get most of the value of running the 153tests using the same ``qemu-arm`` that we used for Armv7-A by building and 154running the test cases for Armv7-A but using the builtins compiled for 155Armv6-M, Armv7-M or Armv7E-M. This will not catch instructions that are 156supported on Armv7-A but not Armv6-M, Armv7-M and Armv7E-M. 157 158To get the cmake compile test to pass the libraries needed to successfully link 159the test application will need to be manually added to ``CMAKE_CFLAGS``. 160Alternatively if you are using version 3.6 or above of cmake you can use 161``CMAKE_TRY_COMPILE_TARGET=STATIC_LIBRARY`` to skip the link step. 162 163* ``-DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY`` 164* ``-DCOMPILER_RT_OS_DIR="baremetal"`` 165* ``-DCOMPILER_RT_BUILD_BUILTINS=ON`` 166* ``-DCOMPILER_RT_BUILD_SANITIZERS=OFF`` 167* ``-DCOMPILER_RT_BUILD_XRAY=OFF`` 168* ``-DCOMPILER_RT_BUILD_LIBFUZZER=OFF`` 169* ``-DCOMPILER_RT_BUILD_PROFILE=OFF`` 170* ``-DCMAKE_C_COMPILER=${host_install_dir}/bin/clang`` 171* ``-DCMAKE_C_COMPILER_TARGET="your *-none-eabi target"`` 172* ``-DCMAKE_AR=/path/to/llvm-ar`` 173* ``-DCMAKE_NM=/path/to/llvm-nm`` 174* ``-DCMAKE_RANLIB=/path/to/llvm-ranlib`` 175* ``-DCOMPILER_RT_BAREMETAL_BUILD=ON`` 176* ``-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON`` 177* ``-DLLVM_CONFIG_PATH=/path/to/llvm-config`` 178* ``-DCMAKE_C_FLAGS="build-c-flags"`` 179* ``-DCMAKE_ASM_FLAGS="${arm_cflags}"`` 180* ``-DCOMPILER_RT_EMULATOR="qemu-arm -L /path/to/armv7-A/sysroot"`` 181* ``-DCOMPILER_RT_INCLUDE_TESTS=ON`` 182* ``-DCOMPILER_RT_TEST_COMPILER="/path/to/clang"`` 183* ``-DCOMPILER_RT_TEST_COMPILER_CFLAGS="test-c-flags"`` 184 185The Armv6-M builtins will use the soft-float ABI. When compiling the tests for 186Armv7-A we must include ``"-mthumb -mfloat-abi=soft -mfpu=none"`` in the 187test-c-flags. We must use an Armv7-A soft-float abi sysroot for ``qemu-arm``. 188 189Unfortunately at time of writing the Armv7-M and Armv7E-M builds of 190compiler-rt will always include assembler files including floating point 191instructions. This means that building for a cpu without a floating point unit 192requires something like removing the arm_Thumb1_VFPv2_SOURCES from the 193arm_Thumb1_SOURCES in builtins/CMakeLists.txt. The float-abi of the compiler-rt 194library must be matched by the float abi of the Armv7-A sysroot used by 195qemu-arm. 196 197Depending on the linker used for the test cases you may encounter BuildAttribute 198mismatches between the M-profile objects from compiler-rt and the A-profile 199objects from the test. The lld linker does not check the BuildAttributes so it 200can be used to link the tests by adding -fuse-ld=lld to the 201``COMPILER_RT_TEST_COMPILER_CFLAGS``. 202