1.. SPDX-License-Identifier: GPL-2.0 2 3=================================== 4Using AutoFDO with the Linux kernel 5=================================== 6 7This enables AutoFDO build support for the kernel when using 8the Clang compiler. AutoFDO (Auto-Feedback-Directed Optimization) 9is a type of profile-guided optimization (PGO) used to enhance the 10performance of binary executables. It gathers information about the 11frequency of execution of various code paths within a binary using 12hardware sampling. This data is then used to guide the compiler's 13optimization decisions, resulting in a more efficient binary. AutoFDO 14is a powerful optimization technique, and data indicates that it can 15significantly improve kernel performance. It's especially beneficial 16for workloads affected by front-end stalls. 17 18For AutoFDO builds, unlike non-FDO builds, the user must supply a 19profile. Acquiring an AutoFDO profile can be done in several ways. 20AutoFDO profiles are created by converting hardware sampling using 21the "perf" tool. It is crucial that the workload used to create these 22perf files is representative; they must exhibit runtime 23characteristics similar to the workloads that are intended to be 24optimized. Failure to do so will result in the compiler optimizing 25for the wrong objective. 26 27The AutoFDO profile often encapsulates the program's behavior. If the 28performance-critical codes are architecture-independent, the profile 29can be applied across platforms to achieve performance gains. For 30instance, using the profile generated on Intel architecture to build 31a kernel for AMD architecture can also yield performance improvements. 32 33There are two methods for acquiring a representative profile: 34(1) Sample real workloads using a production environment. 35(2) Generate the profile using a representative load test. 36When enabling the AutoFDO build configuration without providing an 37AutoFDO profile, the compiler only modifies the dwarf information in 38the kernel without impacting runtime performance. It's advisable to 39use a kernel binary built with the same AutoFDO configuration to 40collect the perf profile. While it's possible to use a kernel built 41with different options, it may result in inferior performance. 42 43One can collect profiles using AutoFDO build for the previous kernel. 44AutoFDO employs relative line numbers to match the profiles, offering 45some tolerance for source changes. This mode is commonly used in a 46production environment for profile collection. 47 48In a profile collection based on a load test, the AutoFDO collection 49process consists of the following steps: 50 51#. Initial build: The kernel is built with AutoFDO options 52 without a profile. 53 54#. Profiling: The above kernel is then run with a representative 55 workload to gather execution frequency data. This data is 56 collected using hardware sampling, via perf. AutoFDO is most 57 effective on platforms supporting advanced PMU features like 58 LBR on Intel machines, ETM traces on ARM machines. 59 60#. AutoFDO profile generation: Perf output file is converted to 61 the AutoFDO profile via offline tools. 62 63The support requires a Clang compiler LLVM 17 or later. 64 65Preparation 66=========== 67 68Configure the kernel with:: 69 70 CONFIG_AUTOFDO_CLANG=y 71 72Customization 73============= 74 75The default CONFIG_AUTOFDO_CLANG setting covers kernel space objects for 76AutoFDO builds. One can, however, enable or disable AutoFDO build for 77individual files and directories by adding a line similar to the following 78to the respective kernel Makefile: 79 80- For enabling a single file (e.g. foo.o) :: 81 82 AUTOFDO_PROFILE_foo.o := y 83 84- For enabling all files in one directory :: 85 86 AUTOFDO_PROFILE := y 87 88- For disabling one file :: 89 90 AUTOFDO_PROFILE_foo.o := n 91 92- For disabling all files in one directory :: 93 94 AUTOFDO_PROFILE := n 95 96Workflow 97======== 98 99Here is an example workflow for AutoFDO kernel: 100 1011) Build the kernel on the host machine with LLVM enabled, 102 for example, :: 103 104 $ make menuconfig LLVM=1 105 106 Turn on AutoFDO build config:: 107 108 CONFIG_AUTOFDO_CLANG=y 109 110 With a configuration that with LLVM enabled, use the following command:: 111 112 $ scripts/config -e AUTOFDO_CLANG 113 114 After getting the config, build with :: 115 116 $ make LLVM=1 117 1182) Install the kernel on the test machine. 119 1203) Run the load tests. The '-c' option in perf specifies the sample 121 event period. We suggest using a suitable prime number, like 500009, 122 for this purpose. 123 124 - For Intel platforms:: 125 126 $ perf record -e BR_INST_RETIRED.NEAR_TAKEN:k -a -N -b -c <count> -o <perf_file> -- <loadtest> 127 128 - For AMD platforms: 129 130 The supported systems are: Zen3 with BRS, or Zen4 with amd_lbr_v2. To check, 131 132 For Zen3:: 133 134 $ cat proc/cpuinfo | grep " brs" 135 136 For Zen4:: 137 138 $ cat proc/cpuinfo | grep amd_lbr_v2 139 140 The following command generated the perf data file:: 141 142 $ perf record --pfm-events RETIRED_TAKEN_BRANCH_INSTRUCTIONS:k -a -N -b -c <count> -o <perf_file> -- <loadtest> 143 144 - For ARM platforms with ETM trace: 145 146 Follow the instructions in the `Linaro OpenCSD document 147 https://github.com/Linaro/OpenCSD/blob/master/decoder/tests/auto-fdo/autofdo.md`_ 148 to record ETM traces for AutoFDO:: 149 150 $ perf record -e cs_etm/@tmc_etr0/k -a -o <etm_perf_file> -- <loadtest> 151 $ perf inject -i <etm_perf_file> -o <perf_file> --itrace=i500009il 152 153 For ARM platforms running Android, follow the instructions in the 154 `Android simpleperf document 155 <https://android.googlesource.com/platform/system/extras/+/main/simpleperf/doc/collect_etm_data_for_autofdo.md>`_ 156 to record ETM traces for AutoFDO:: 157 158 $ simpleperf record -e cs-etm:k -a -o <perf_file> -- <loadtest> 159 1604) (Optional) Download the raw perf file to the host machine. 161 1625) To generate an AutoFDO profile, two offline tools are available: 163 create_llvm_prof and llvm_profgen. The create_llvm_prof tool is part 164 of the AutoFDO project and can be found on GitHub 165 (https://github.com/google/autofdo), version v0.30.1 or later. 166 The llvm_profgen tool is included in the LLVM compiler itself. It's 167 important to note that the version of llvm_profgen doesn't need to match 168 the version of Clang. It needs to be the LLVM 19 release of Clang 169 or later, or just from the LLVM trunk. :: 170 171 $ llvm-profgen --kernel --binary=<vmlinux> --perfdata=<perf_file> -o <profile_file> 172 173 or :: 174 175 $ create_llvm_prof --binary=<vmlinux> --profile=<perf_file> --format=extbinary --out=<profile_file> 176 177 Note that multiple AutoFDO profile files can be merged into one via:: 178 179 $ llvm-profdata merge -o <profile_file> <profile_1> <profile_2> ... <profile_n> 180 1816) Rebuild the kernel using the AutoFDO profile file with the same config as step 1, 182 (Note CONFIG_AUTOFDO_CLANG needs to be enabled):: 183 184 $ make LLVM=1 CLANG_AUTOFDO_PROFILE=<profile_file> 185