1Validing libc Assembler Routines 2================================ 3This document describes how to verify incoming assembler libc routines. 4 5## Quick Start 6* First, benchmark the previous version of the routine. 7* Update the routine, run the bionic unit tests to verify the routine doesn't 8have any bugs. See the [Testing](#Testing) section for details about how to 9verify that the routine is being properly tested. 10* Rerun the benchmarks using the updated image that uses the code for 11the new routine. See the [Performance](#Performance) section for details about 12benchmarking. 13* Verify that unwind information for new routine looks sane. See the [Unwind Info](#unwind-info) section for details about how to verify this. 14 15When benchmarking, it's best to verify on the latest Pixel device supported. 16Make sure that you benchmark both the big and little cores to verify that 17there is no major difference in performance on each. 18 19Benchmark 64 bit memcmp: 20 21 /data/benchmarktest64/bionic-benchmarks/bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=memcmp 22 23Benchmark 32 bit memcmp: 24 25 /data/benchmarktest/bionic-benchmarks/bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=memcmp 26 27Locking to a specific cpu: 28 29 /data/benchmarktest/bionic-benchmarks/bionic-benchmarks --bionic_cpu=2 --bionic_xml=string.xml --benchmark_filter=memcmp 30 31## Performance 32The bionic benchmarks are used to verify the performance of changes to 33routines. For most routines, there should already be benchmarks available. 34 35Building 36-------- 37The bionic benchmarks are not built by default, they must be built separately 38and pushed on to the device. The commands below show how to do this. 39 40 mmma -j bionic/benchmarks 41 adb sync data 42 43Running 44------- 45There are two bionic benchmarks executables: 46 47 /data/benchmarktest64/bionic-benchmarks/bionic-benchmarks 48 49This is for 64 bit libc routines. 50 51 /data/benchmarktest/bionic-benchmarks/bionic-benchmarks 52 53This is for 32 bit libc routines. 54 55Here is an example of how the benchmark should be executed. For this 56command to work, you need to change directory to one of the above 57directories. 58 59 bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=memcmp 60 61The last argument is the name of the one function that you want to 62benchmark. 63 64Almost all routines are already defined in the **string.xml** file in 65**bionic/benchmarks/suites**. Look at the examples in that file to see 66how to add a benchmark for a function that doesn't already exist. 67 68It can take a long time to run these tests since it attempts to test a 69large number of sizes and alignments. 70 71Results 72------- 73Bionic benchmarks is based on the [Google Benchmarks](https://github.com/google/benchmark) 74library. An example of the output looks like this: 75 76 Run on (8 X 1844 MHz CPU s) 77 CPU Caches: 78 L1 Data 32K (x8) 79 L1 Instruction 32K (x8) 80 L2 Unified 512K (x2) 81 ***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead. 82 ------------------------------------------------------------------------------------------- 83 Benchmark Time CPU Iterations 84 ------------------------------------------------------------------------------------------- 85 BM_string_memcmp/1/0/0 6 ns 6 ns 120776418 164.641MB/s 86 BM_string_memcmp/1/1/1 6 ns 6 ns 120856788 164.651MB/s 87 88The smaller the time, the better the performance. 89 90Caveats 91------- 92When running the benchmarks, CPU scaling is not normally enabled. This means 93that if the device does not get up to the maximum cpu frequency, the results 94can vary wildly. It's possible to lock the cpu to the maximum frequency, but 95is beyond the scope of this document. However, most of the benchmarks max 96out the cpu very quickly on Pixel devices, and don't affect the results. 97 98Another potential issue is that the device can overheat when running the 99benchmarks. To avoid this, you can run the device in a cool environment, 100or choose a device that is less likely to overheat. To detect these kind 101of issues, you can run a subset of the tests again. At the very least, it's 102always a good idea to rerun the suite a couple of times to verify that 103there isn't a high variation in the numbers. 104 105If you want to verify a single benchmark result, you can run a single test 106using a command like this: 107 108 bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=BM_string_memcmp/1/1/0 109 110Where the argument to the filter argument is the name of the benchmark from 111the output. Sometimes this filter can still match multiple benchmarks, to 112guarantee that you only run the single benchmark, you can execute the benchmark 113like so: 114 115 bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=BM_string_memcmp/1/1/0$ 116 117NOTE: It is assumed that these commands are executed in adb as the shell user 118on device. If you are trying to run this using adb directly from a host 119machine, you might need to escape the special shell characters such as **$**. 120 121## Testing 122 123Run the bionic tests to verify that the new routines are valid. However, 124you should verify that there is coverage of the new routines. This is 125especially important if this is the first time a routine is assembler. 126 127Caveats 128------- 129When verifying an assembler routine that operates on buffer data (such as 130memcpy/strcpy), it's important to verify these corner cases: 131 132* Verify the routine does not read past the end of the buffers. Many 133assembler routines optimize by reading multipe bytes at a time and can 134read past the end. This kind of bug results in an infrequent and difficult to 135diagnosis crash. 136* Verify the routine handles unaligned buffers properly. Usually, a failure 137can result in an unaligned exception. 138* Verify the routine handles different sized buffers. 139 140If there are not sufficient tests for a new routine, there are a set of helper 141functions that can be used to verify the above corner cases. See the 142header **bionic/tests/buffer\_tests.h** for these routines and look at 143**bionic/tests/string\_test.cpp** for examples of how to use it. 144 145## Unwind Info 146It is also important to verify that the unwind information for these 147routines are properly set up. Here is a quick checklist of what to check: 148 149* Verify that all labels are of the format .LXXX, where XXX is any valid string 150for a label. If any other label is used, entries in the symbol table 151will be generated that include these labels. In that case, you will get 152an unwind with incorrect function information. 153* Verify that all places where pop/pushes or instructions that modify the 154sp in any way have corresponding cfi information. Along with this item, 155verify that when registers are pushed on the stack that there is cfi 156information indicating how to get the register. 157* Verify that only cfi directives are being used. This only matters for 158arm32, where it's possible to use ARM specific unwind directives. 159 160This list is not meant to be exhaustive, but a minimal set of items to verify 161before submitting a new libc assembler routine. There are difficult 162to verify unwind cases, such as around branches, where unwind information 163can be drastically different for the target of the branch and for the 164code after a branch instruction. 165