README.md
1# Compiler wrapper
2
3See the comments on the top of main.go.
4Build is split into 2 steps via separate commands:
5- bundle: copies the sources and the `build.py` file into
6 a folder.
7- build: builds the actual go binary, assuming it is executed
8 from the folder created by `bundle.py`.
9
10This allows to copy the sources to a ChromeOS / Android
11package, including the build script, and then
12build from there without a dependency on toolchain-utils
13itself.
14
15## Testing Inside the Chroot
16
17To test updates to the wrapper locally:
18
19Run `install_compiler_wrapper.sh` to install the new wrapper in the chroot:
20```
21(chroot) ~/trunk/src/third_party/toolchain-utils/compiler_wrapper/install_compiler_wrapper.sh
22```
23
24Then perform the tests, e.g. build with the new compiler.
25
26
27## Updating the Wrapper for ChromeOS
28
29To update the wrapper for everyone, the new wrapper configuration must be copied
30into chromiumos-overlay, and new revisions of the gcc and llvm ebuilds must be
31created.
32
33Copy over sources and `build.py` to chromiumos-overlay:
34```
35(chroot) /mnt/host/source/src/third_party/chromiumos-overlay/sys-devel/llvm/files/update_compiler_wrapper.sh
36```
37
38Rename chromiumos-overlay/sys-devel/llvm/llvm-${VERSION}.ebuild to the next
39revision number. For example, if the current version is
4011.0_pre394483_p20200618-r2:
41```
42(chroot) cd ~/trunk/src/third_party/chromiumos-overlay
43(chroot) git mv llvm-11.0_pre394483_p20200618-r2.ebuild llvm-11.0_pre394483_p20200618-r3.ebuild
44```
45
46Rename chromiumos-overlay/sys-devel/gcc/gcc-${VERSION}.ebuild to the next
47revision number. For example, if the current version is 10.2.0-r3:
48```
49(chroot) cd ~/trunk/src/third_party/chromiumos-overlay
50(chroot) git mv sys-devel/gcc/gcc-10.2.0-r3.ebuild sys-devel/gcc/gcc-10.2.0-r4.ebuild
51```
52
53Commit those changes together with the changes made by
54`update_compiler_wrapper.sh`.
55
56The changes can then be reviewed and submitted through the normal process.
57
58
59## Paths
60
61`build.py` is called by these ebuilds:
62
63- third_party/chromiumos-overlay/sys-devel/llvm/llvm-*.ebuild
64- third_party/chromiumos-overlay/sys-devel/gcc/gcc-*.ebuild
65
66Generated wrappers are stored here:
67
68- Sysroot wrapper with ccache:
69 `/usr/x86_64-pc-linux-gnu/<arch>/gcc-bin/10.2.0/sysroot_wrapper.hardened.ccache`
70- Sysroot wrapper without ccache:
71 `/usr/x86_64-pc-linux-gnu/<arch>/gcc-bin/10.2.0/sysroot_wrapper.hardened.noccache`
72- Clang host wrapper:
73 `/usr/bin/clang_host_wrapper`
74- Gcc host wrapper:
75 `/usr/x86_64-pc-linux-gnu/gcc-bin/10.2.0/host_wrapper`
76
77## Using the compiler wrapper to crash arbitrary compilations
78
79When Clang crashes, its output can be extremely useful. Often, it will provide
80the user with a stack trace, and messages like:
81
82```
83clang-15: unable to execute command: Illegal instruction
84clang-15: note: diagnostic msg: /tmp/clang_crash_diagnostics/foo-5420d2.c
85clang-15: note: diagnostic msg: /tmp/clang_crash_diagnostics/foo-5420d2.sh
86```
87
88Where the artifacts at `/tmp/clang_crash_diagnostics/foo-*` are a full,
89self-contained reproducer of the inputs that caused the crash in question.
90Often, such a reproducer is very valuable to have even for cases where a crash
91_doesn't_ happen (e.g., maybe Clang is now emitting an error where it used to
92not do so, and we want to bisect upstream LLVM with that info). Normally,
93collecting and crafting such a reproducer is a multi-step process, and can be
94error-prone; compile commands may rely on env vars, they may be done within
95`chroot`s, they may rely on being executed in a particular directory, they may
96rely on intermediate state, etc.
97
98Because of the usefulness of these crash reports, our wrapper supports crashing
99Clang even on files that ordinarily don't cause Clang to crash. For various
100reasons (b/236736327), this support currently requires rebuilding and
101redeploying the wrapper in order to work. That said, this could be a valuable
102tool for devs interested in creating a self-contained reproducer without having
103to manually reproduce the environment in which a particular build was performed.
104
bundle.README