1# bionic 2 3[bionic](https://en.wikipedia.org/wiki/Bionic_(software)) is Android's 4C library, math library, and dynamic linker. 5 6# Using bionic as an app developer 7 8See the [user documentation](docs/). 9 10# Working on bionic itself 11 12This documentation is about making changes to bionic itself. 13 14## What are the big pieces of bionic? 15 16#### libc/ --- libc.so, libc.a 17 18The C library. Stuff like `fopen(3)` and `kill(2)`. 19 20#### libm/ --- libm.so, libm.a 21 22The math library. Traditionally Unix systems kept stuff like `sin(3)` and 23`cos(3)` in a separate library to save space in the days before shared 24libraries. 25 26#### libdl/ --- libdl.so 27 28The dynamic linker interface library. This is actually just a bunch of stubs 29that the dynamic linker replaces with pointers to its own implementation at 30runtime. This is where stuff like `dlopen(3)` lives. 31 32#### libstdc++/ --- libstdc++.so 33 34The C++ ABI support functions. The C++ compiler doesn't know how to implement 35thread-safe static initialization and the like, so it just calls functions that 36are supplied by the system. Stuff like `__cxa_guard_acquire` and 37`__cxa_pure_virtual` live here. 38 39#### linker/ --- /system/bin/linker and /system/bin/linker64 40 41The dynamic linker. When you run a dynamically-linked executable, its ELF file 42has a `DT_INTERP` entry that says "use the following program to start me". On 43Android, that's either `linker` or `linker64` (depending on whether it's a 4432-bit or 64-bit executable). It's responsible for loading the ELF executable 45into memory and resolving references to symbols (so that when your code tries to 46jump to `fopen(3)`, say, it lands in the right place). 47 48#### tests/ --- unit tests 49 50The `tests/` directory contains unit tests. Roughly arranged as one file per 51publicly-exported header file. 52 53#### benchmarks/ --- benchmarks 54 55The `benchmarks/` directory contains benchmarks, with its own [documentation](benchmarks/README.md). 56 57 58## What's in libc/? 59 60``` 61libc/ 62 arch-arm/ 63 arch-arm64/ 64 arch-common/ 65 arch-x86/ 66 arch-x86_64/ 67 # Each architecture has its own subdirectory for stuff that isn't shared 68 # because it's architecture-specific. There will be a .mk file in here that 69 # drags in all the architecture-specific files. 70 bionic/ 71 # Every architecture needs a handful of machine-specific assembler files. 72 # They live here. 73 string/ 74 # Most architectures have a handful of optional assembler files 75 # implementing optimized versions of various routines. The <string.h> 76 # functions are particular favorites. 77 syscalls/ 78 # The syscalls directories contain script-generated assembler files. 79 # See 'Adding system calls' later. 80 81 include/ 82 # The public header files on everyone's include path. These are a mixture of 83 # files written by us and files taken from BSD. 84 85 kernel/ 86 # The kernel uapi header files. These are scrubbed copies of the originals 87 # in external/kernel-headers/. These files must not be edited directly. The 88 # generate_uapi_headers.sh script should be used to go from a kernel tree to 89 # external/kernel-headers/ --- this takes care of the architecture-specific 90 # details. The update_all.py script should be used to regenerate bionic's 91 # scrubbed headers from external/kernel-headers/. 92 93 private/ 94 # These are private header files meant for use within bionic itself. 95 96 dns/ 97 # Contains the DNS resolver (originates from NetBSD code). 98 99 upstream-freebsd/ 100 upstream-netbsd/ 101 upstream-openbsd/ 102 # These directories contain unmolested upstream source. Any time we can 103 # just use a BSD implementation of something unmodified, we should. 104 # The structure under these directories mimics the upstream tree, 105 # but there's also... 106 android/ 107 include/ 108 # This is where we keep the hacks necessary to build BSD source 109 # in our world. The *-compat.h files are automatically included 110 # using -include, but we also provide equivalents for missing 111 # header/source files needed by the BSD implementation. 112 113 bionic/ 114 # This is the biggest mess. The C++ files are files we own, typically 115 # because the Linux kernel interface is sufficiently different that we 116 # can't use any of the BSD implementations. The C files are usually 117 # legacy mess that needs to be sorted out, either by replacing it with 118 # current upstream source in one of the upstream directories or by 119 # switching the file to C++ and cleaning it up. 120 121 malloc_debug/ 122 # The code that implements the functionality to enable debugging of 123 # native allocation problems. 124 125 stdio/ 126 # These are legacy files of dubious provenance. We're working to clean 127 # this mess up, and this directory should disappear. 128 129 tools/ 130 # Various tools used to maintain bionic. 131 132 tzcode/ 133 # A modified superset of the IANA tzcode. Most of the modifications relate 134 # to Android's use of a single file (with corresponding index) to contain 135 # time zone data. 136 zoneinfo/ 137 # Android-format time zone data. 138 # See 'Updating tzdata' later. 139``` 140 141 142## Adding libc wrappers for system calls 143 144The first question you should ask is "should I add a libc wrapper for 145this system call?". The answer is usually "no". 146 147The answer is "yes" if the system call is part of the POSIX standard. 148 149The answer is probably "yes" if the system call has a wrapper in at 150least one other C library (typically glibc/musl or Apple's libc). 151 152The answer may be "yes" if the system call has three/four distinct 153users in different projects, and there isn't a more specific higher-level 154library that would make more sense as the place to add the wrapper. 155 156In all other cases, you should use 157[syscall(3)](http://man7.org/linux/man-pages/man2/syscall.2.html) instead. 158 159Adding a system call usually involves: 160 161 1. Add entries to SYSCALLS.TXT. 162 See SYSCALLS.TXT itself for documentation on the format. 163 2. Add constants (and perhaps types) to the appropriate header file. 164 Note that you should check to see whether the constants are already in 165 kernel uapi header files, in which case you just need to make sure that 166 the appropriate POSIX header file in libc/include/ includes the 167 relevant file or files. 168 3. Add function declarations to the appropriate header file. Don't forget 169 to include the appropriate `__INTRODUCED_IN()`. If you need to create a new 170 header file, libc/include/sys/sysinfo.h is a good short example to copy and 171 paste from. 172 4. Add basic documentation to the header file. libc/include/sys/sysinfo.h is a 173 good short example that shows the expected style. Most of the detail 174 should actually be left to the man7.org page, with only a brief 175 one-sentence explanation in our documentation. Alway include the return 176 value/error reporting details. Explicitly say which version of Android the 177 function was added to. Explicitly call out any Android-specific 178 changes/additions/limitations because they won't be on the man7.org page. 179 5. Add the function name to the correct section in libc/libc.map.txt. 180 6. Add a basic test. Don't try to test everything; concentrate on just testing 181 the code that's actually in *bionic*, not all the functionality that's 182 implemented in the kernel. For simple syscalls, that's just the 183 auto-generated argument and return value marshalling. 184 185 A trivial test that deliberately supplies an invalid argument helps check 186 that we're generating the right symbol and have the right declaration in 187 the header file, and that the change to libc.map.txt from step 5 is 188 correct. (You can use strace(1) manually to confirm that the correct 189 system call is being made.) 190 191 For testing the *kernel* side of things, we should prefer to rely on 192 https://github.com/linux-test-project/ltp for kernel testing, but you'll 193 want to check that external/ltp does contain tests for the syscall you're 194 adding. Also check that external/ltp is using the libc wrapper for the 195 syscall rather than calling it "directly" via syscall(3)! 196 197Some system calls are harder than others. The most common problem is a 64-bit 198argument such as `off64_t` (a *pointer* to a 64-bit argument is fine, since 199pointers are always the "natural" size for the architecture regardless of the 200size of the thing they point to). Whenever you have a function that takes 201`off_t` or `off64_t`, you'll need to consider whether you actually need a foo() 202and a foo64(), and whether they will use the same underlying system call or are 203implemented as two different system calls. It's usually easiest to find a 204similar system call and copy and paste from that. You'll definitely need to test 205both on 32-bit and 64-bit. (These special cases warrant more testing than the 206easy cases, even if only manual testing with strace. Sadly it isn't always 207feasible to write a working test for the interesting cases -- offsets larger 208than 2GiB, say -- so you may end up just writing a "meaningless" program whose 209only purpose is to give you patterns to look for when run under strace(1).) 210 211## Updating kernel header files 212 213As mentioned above, this is currently a two-step process: 214 215 1. Use generate_uapi_headers.sh to go from a Linux source tree to appropriate 216 contents for external/kernel-headers/. 217 2. Run update_all.py to scrub those headers and import them into bionic. 218 219Note that if you're actually just trying to expose device-specific headers to 220build your device drivers, you shouldn't modify bionic. Instead use 221`TARGET_DEVICE_KERNEL_HEADERS` and friends described in [config.mk](https://android.googlesource.com/platform/build/+/master/core/config.mk#186). 222 223 224## Updating tzdata 225 226This is handled by the libcore team, because they own icu, and that needs to be 227updated in sync with bionic). See 228[system/timezone/README.android](https://android.googlesource.com/platform/system/timezone/+/master/README.android). 229 230 231## Verifying changes 232 233If you make a change that is likely to have a wide effect on the tree (such as a 234libc header change), you should run `make checkbuild`. A regular `make` will 235_not_ build the entire tree; just the minimum number of projects that are 236required for the device. Tests, additional developer tools, and various other 237modules will not be built. Note that `make checkbuild` will not be complete 238either, as `make tests` covers a few additional modules, but generally speaking 239`make checkbuild` is enough. 240 241 242## Running the tests 243 244The tests are all built from the tests/ directory. 245 246### Device tests 247 248 $ mma # In $ANDROID_ROOT/bionic. 249 $ adb root && adb remount && adb sync 250 $ adb shell /data/nativetest/bionic-unit-tests/bionic-unit-tests 251 $ adb shell \ 252 /data/nativetest/bionic-unit-tests-static/bionic-unit-tests-static 253 # Only for 64-bit targets 254 $ adb shell /data/nativetest64/bionic-unit-tests/bionic-unit-tests 255 $ adb shell \ 256 /data/nativetest64/bionic-unit-tests-static/bionic-unit-tests-static 257 258Note that we use our own custom gtest runner that offers a superset of the 259options documented at 260<https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.md#running-test-programs-advanced-options>, 261in particular for test isolation and parallelism (both on by default). 262 263### Device tests via CTS 264 265Most of the unit tests are executed by CTS. By default, CTS runs as 266a non-root user, so the unit tests must also pass when not run as root. 267Some tests cannot do any useful work unless run as root. In this case, 268the test should check `getuid() == 0` and do nothing otherwise (typically 269we log in this case to prevent accidents!). Obviously, if the test can be 270rewritten to not require root, that's an even better solution. 271 272Currently, the list of bionic CTS tests is generated at build time by 273running a host version of the test executable and dumping the list of 274all tests. In order for this to continue to work, all architectures must 275have the same number of tests, and the host version of the executable 276must also have the same number of tests. 277 278Running the gtests directly is orders of magnitude faster than using CTS, 279but in cases where you really have to run CTS: 280 281 $ make cts # In $ANDROID_ROOT. 282 $ adb unroot # Because real CTS doesn't run as root. 283 # This will sync any *test* changes, but not *code* changes: 284 $ cts-tradefed \ 285 run singleCommand cts --skip-preconditions -m CtsBionicTestCases 286 287### Host tests 288 289The host tests require that you have `lunch`ed either an x86 or x86_64 target. 290Note that due to ABI limitations (specifically, the size of pthread_mutex_t), 29132-bit bionic requires PIDs less than 65536. To enforce this, set /proc/sys/kernel/pid_max 292to 65536. 293 294 $ ./tests/run-on-host.sh 32 295 $ ./tests/run-on-host.sh 64 # For x86_64-bit *targets* only. 296 297You can supply gtest flags as extra arguments to this script. 298 299### Against glibc 300 301As a way to check that our tests do in fact test the correct behavior (and not 302just the behavior we think is correct), it is possible to run the tests against 303the host's glibc. 304 305 $ ./tests/run-on-host.sh glibc 306 307## Gathering test coverage 308 309To get test coverage for bionic, use `//bionic/build/coverage.sh`. Before 310running, follow the instructions at the top of the file to rebuild bionic with 311coverage instrumentation. 312 313## Attaching GDB to the tests 314 315Bionic's test runner will run each test in its own process by default to prevent 316tests failures from impacting other tests. This also has the added benefit of 317running them in parallel, so they are much faster. 318 319However, this also makes it difficult to run the tests under GDB. To prevent 320each test from being forked, run the tests with the flag `--no-isolate`. 321 322 323## 32-bit ABI bugs 324 325See [32-bit ABI bugs](docs/32-bit-abi.md). 326