1======= 2ThinLTO 3======= 4 5.. contents:: 6 :local: 7 8Introduction 9============ 10 11*ThinLTO* compilation is a new type of LTO that is both scalable and 12incremental. *LTO* (Link Time Optimization) achieves better 13runtime performance through whole-program analysis and cross-module 14optimization. However, monolithic LTO implements this by merging all 15input into a single module, which is not scalable 16in time or memory, and also prevents fast incremental compiles. 17 18In ThinLTO mode, as with regular LTO, clang emits LLVM bitcode after the 19compile phase. The ThinLTO bitcode is augmented with a compact summary 20of the module. During the link step, only the summaries are read and 21merged into a combined summary index, which includes an index of function 22locations for later cross-module function importing. Fast and efficient 23whole-program analysis is then performed on the combined summary index. 24 25However, all transformations, including function importing, occur 26later when the modules are optimized in fully parallel backends. 27By default, linkers_ that support ThinLTO are set up to launch 28the ThinLTO backends in threads. So the usage model is not affected 29as the distinction between the fast serial thin link step and the backends 30is transparent to the user. 31 32For more information on the ThinLTO design and current performance, 33see the LLVM blog post `ThinLTO: Scalable and Incremental LTO 34<http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html>`_. 35While tuning is still in progress, results in the blog post show that 36ThinLTO already performs well compared to LTO, in many cases matching 37the performance improvement. 38 39Current Status 40============== 41 42Clang/LLVM 43---------- 44.. _compiler: 45 46The 3.9 release of clang includes ThinLTO support. However, ThinLTO 47is under active development, and new features, improvements and bugfixes 48are being added for the next release. For the latest ThinLTO support, 49`build a recent version of clang and LLVM 50<https://llvm.org/docs/CMake.html>`_. 51 52Linkers 53------- 54.. _linkers: 55.. _linker: 56 57ThinLTO is currently supported for the following linkers: 58 59- **gold (via the gold-plugin)**: 60 Similar to monolithic LTO, this requires using 61 a `gold linker configured with plugins enabled 62 <https://llvm.org/docs/GoldPlugin.html>`_. 63- **ld64**: 64 Starting with `Xcode 8 <https://developer.apple.com/xcode/>`_. 65- **lld**: 66 Starting with r284050 for ELF, r298942 for COFF. 67 68Usage 69===== 70 71Basic 72----- 73 74To utilize ThinLTO, simply add the -flto=thin option to compile and link. E.g. 75 76.. code-block:: console 77 78 % clang -flto=thin -O2 file1.c file2.c -c 79 % clang -flto=thin -O2 file1.o file2.o -o a.out 80 81When using lld-link, the -flto option need only be added to the compile step: 82 83.. code-block:: console 84 85 % clang-cl -flto=thin -O2 -c file1.c file2.c 86 % lld-link /out:a.exe file1.obj file2.obj 87 88As mentioned earlier, by default the linkers will launch the ThinLTO backend 89threads in parallel, passing the resulting native object files back to the 90linker for the final native link. As such, the usage model the same as 91non-LTO. 92 93With gold, if you see an error during the link of the form: 94 95.. code-block:: console 96 97 /usr/bin/ld: error: /path/to/clang/bin/../lib/LLVMgold.so: could not load plugin library: /path/to/clang/bin/../lib/LLVMgold.so: cannot open shared object file: No such file or directory 98 99Then either gold was not configured with plugins enabled, or clang 100was not built with ``-DLLVM_BINUTILS_INCDIR`` set properly. See 101the instructions for the 102`LLVM gold plugin <https://llvm.org/docs/GoldPlugin.html#how-to-build-it>`_. 103 104Controlling Backend Parallelism 105------------------------------- 106.. _parallelism: 107 108By default, the ThinLTO link step will launch as many 109threads in parallel as there are cores. If the number of 110cores can't be computed for the architecture, then it will launch 111``std::thread::hardware_concurrency`` number of threads in parallel. 112For machines with hyper-threading, this is the total number of 113virtual cores. For some applications and machine configurations this 114may be too aggressive, in which case the amount of parallelism can 115be reduced to ``N`` via: 116 117- gold: 118 ``-Wl,-plugin-opt,jobs=N`` 119- ld64: 120 ``-Wl,-mllvm,-threads=N`` 121- lld: 122 ``-Wl,--thinlto-jobs=N`` 123- lld-link: 124 ``/opt:lldltojobs=N`` 125 126Other possible values for ``N`` are: 127 128- 0: 129 Use one thread per physical core (default) 130- 1: 131 Use a single thread only (disable multi-threading) 132- all: 133 Use one thread per logical core (uses all hyper-threads) 134 135Incremental 136----------- 137.. _incremental: 138 139ThinLTO supports fast incremental builds through the use of a cache, 140which currently must be enabled through a linker option. 141 142- gold (as of LLVM 4.0): 143 ``-Wl,-plugin-opt,cache-dir=/path/to/cache`` 144- ld64 (support in clang 3.9 and Xcode 8): 145 ``-Wl,-cache_path_lto,/path/to/cache`` 146- ELF lld (as of LLVM 5.0): 147 ``-Wl,--thinlto-cache-dir=/path/to/cache`` 148- COFF lld-link (as of LLVM 6.0): 149 ``/lldltocache:/path/to/cache`` 150 151Cache Pruning 152------------- 153 154To help keep the size of the cache under control, ThinLTO supports cache 155pruning. Cache pruning is supported with gold, ld64 and ELF and COFF lld, but 156currently only gold, ELF and COFF lld allow you to control the policy with a 157policy string. The cache policy must be specified with a linker option. 158 159- gold (as of LLVM 6.0): 160 ``-Wl,-plugin-opt,cache-policy=POLICY`` 161- ELF lld (as of LLVM 5.0): 162 ``-Wl,--thinlto-cache-policy,POLICY`` 163- COFF lld-link (as of LLVM 6.0): 164 ``/lldltocachepolicy:POLICY`` 165 166A policy string is a series of key-value pairs separated by ``:`` characters. 167Possible key-value pairs are: 168 169- ``cache_size=X%``: The maximum size for the cache directory is ``X`` percent 170 of the available space on the disk. Set to 100 to indicate no limit, 171 50 to indicate that the cache size will not be left over half the available 172 disk space. A value over 100 is invalid. A value of 0 disables the percentage 173 size-based pruning. The default is 75%. 174 175- ``cache_size_bytes=X``, ``cache_size_bytes=Xk``, ``cache_size_bytes=Xm``, 176 ``cache_size_bytes=Xg``: 177 Sets the maximum size for the cache directory to ``X`` bytes (or KB, MB, 178 GB respectively). A value over the amount of available space on the disk 179 will be reduced to the amount of available space. A value of 0 disables 180 the byte size-based pruning. The default is no byte size-based pruning. 181 182 Note that ThinLTO will apply both size-based pruning policies simultaneously, 183 and changing one does not affect the other. For example, a policy of 184 ``cache_size_bytes=1g`` on its own will cause both the 1GB and default 75% 185 policies to be applied unless the default ``cache_size`` is overridden. 186 187- ``cache_size_files=X``: 188 Set the maximum number of files in the cache directory. Set to 0 to indicate 189 no limit. The default is 1000000 files. 190 191- ``prune_after=Xs``, ``prune_after=Xm``, ``prune_after=Xh``: Sets the 192 expiration time for cache files to ``X`` seconds (or minutes, hours 193 respectively). When a file hasn't been accessed for ``prune_after`` seconds, 194 it is removed from the cache. A value of 0 disables the expiration-based 195 pruning. The default is 1 week. 196 197- ``prune_interval=Xs``, ``prune_interval=Xm``, ``prune_interval=Xh``: 198 Sets the pruning interval to ``X`` seconds (or minutes, hours 199 respectively). This is intended to be used to avoid scanning the directory 200 too often. It does not impact the decision of which files to prune. A 201 value of 0 forces the scan to occur. The default is every 20 minutes. 202 203Clang Bootstrap 204--------------- 205 206To `bootstrap clang/LLVM <https://llvm.org/docs/AdvancedBuilds.html#bootstrap-builds>`_ 207with ThinLTO, follow these steps: 208 2091. The host compiler_ must be a version of clang that supports ThinLTO. 210#. The host linker_ must support ThinLTO (and in the case of gold, must be 211 `configured with plugins enabled <https://llvm.org/docs/GoldPlugin.html>`_). 212#. Use the following additional `CMake variables 213 <https://llvm.org/docs/CMake.html#options-and-variables>`_ 214 when configuring the bootstrap compiler build: 215 216 * ``-DLLVM_ENABLE_LTO=Thin`` 217 * ``-DCMAKE_C_COMPILER=/path/to/host/clang`` 218 * ``-DCMAKE_CXX_COMPILER=/path/to/host/clang++`` 219 * ``-DCMAKE_RANLIB=/path/to/host/llvm-ranlib`` 220 * ``-DCMAKE_AR=/path/to/host/llvm-ar`` 221 222 Or, on Windows: 223 224 * ``-DLLVM_ENABLE_LTO=Thin`` 225 * ``-DCMAKE_C_COMPILER=/path/to/host/clang-cl.exe`` 226 * ``-DCMAKE_CXX_COMPILER=/path/to/host/clang-cl.exe`` 227 * ``-DCMAKE_LINKER=/path/to/host/lld-link.exe`` 228 * ``-DCMAKE_RANLIB=/path/to/host/llvm-ranlib.exe`` 229 * ``-DCMAKE_AR=/path/to/host/llvm-ar.exe`` 230 231#. To use additional linker arguments for controlling the backend 232 parallelism_ or enabling incremental_ builds of the bootstrap compiler, 233 after configuring the build, modify the resulting CMakeCache.txt file in the 234 build directory. Specify any additional linker options after 235 ``CMAKE_EXE_LINKER_FLAGS:STRING=``. Note the configure may fail if 236 linker plugin options are instead specified directly in the previous step. 237 238The ``BOOTSTRAP_LLVM_ENABLE_LTO=Thin`` will enable ThinLTO for stage 2 and 239stage 3 in case the compiler used for stage 1 does not support the ThinLTO 240option. 241 242More Information 243================ 244 245* From LLVM project blog: 246 `ThinLTO: Scalable and Incremental LTO 247 <http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html>`_ 248