1.. _module-pw_fuzzer: 2 3--------- 4pw_fuzzer 5--------- 6``pw_fuzzer`` provides developers with tools to write `libFuzzer`_ based 7fuzzers. 8 9Fuzzing or fuzz testing is style of testing that stochastically generates inputs 10to targeted interfaces in order to automatically find defects and/or 11vulnerabilities. In other words, fuzzing is simply an automated way of testing 12APIs with generated data. 13 14A fuzzer is a program that is used to fuzz a interface. It typically has three 15steps that it executes repeatedly: 16 17#. Generate a new, context-free input. This is the *fuzzing engine*. For 18 ``pw_fuzzer``, this is `libFuzzer`_. 19#. Use the input to exercise the targeted interface, or code being tested. This 20 is the *fuzz target function*. For ``pw_fuzzer``, these are the GN 21 ``sources`` and/or ``deps`` that define `LLVMFuzzerTestOneInput`_. 22#. Monitor the code being tested for any abnormal conditions. This is the 23 *instrumentation*. For ``pw_fuzzer``, these are sanitizer runtimes from 24 LLVM's `compiler_rt`_. 25 26.. note:: 27 28 ``pw_fuzzer`` is currently only supported on Linux and MacOS using clang. 29 30.. image:: doc_resources/pw_fuzzer_coverage_guided.png 31 :alt: Coverage Guided Fuzzing with libFuzzer 32 :align: left 33 34Writing fuzzers 35=============== 36 37To write a fuzzer, a developer needs to write a fuzz target function follwing 38the `fuzz target function`__ guidelines given by libFuzzer: 39 40.. code:: cpp 41 42 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { 43 DoSomethingInterestingWithMyAPI(data, size); 44 return 0; // Non-zero return values are reserved for future use. 45 } 46 47.. __: LLVMFuzzerTestOneInput_ 48 49When writing you fuzz target function, you may want to consider: 50 51- It is acceptable to return early if the input doesn't meet some constraints, 52 e.g. it is too short. 53- If your fuzzer accepts data with a well-defined format, you can bootstrap 54 coverage by crafting examples and adding them to a `corpus`_. 55- There are tools to `split a fuzzing input`_ into multiple fields if needed; 56 the `FuzzedDataProvider`_ is particularly easy to use. 57- If your code acts on "transformed" inputs, such as encoded or compressed 58 inputs, you may want to try `structure aware fuzzing`. 59- You can do `startup initialization`_ if you need to. 60- If your code is non-deterministic or uses checksums, you may want to disable 61 those **only** when fuzzing by using LLVM's 62 `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION`_ 63 64.. _build: 65 66Building fuzzers with GN 67======================== 68 69To build a fuzzer, do the following: 70 711. Add the GN target using ``pw_fuzzer`` GN template, and add it to your the 72 test group of the module: 73 74.. code:: 75 76 # In $dir_my_module/BUILD.gn 77 import("$dir_pw_fuzzer/fuzzer.gni") 78 79 pw_fuzzer("my_fuzzer") { 80 sources = [ "my_fuzzer.cc" ] 81 deps = [ ":my_lib" ] 82 } 83 84 pw_test_group("tests") { 85 tests = [ 86 ":existing_tests", ... 87 ":my_fuzzer", # <- Added! 88 ] 89 } 90 912. Select your choice of sanitizers ("address" is also the current default). 92 See LLVM for `valid options`_. 93 94.. code:: sh 95 96 $ gn gen out --args='pw_toolchain_SANITIZERS=["address"]' 97 983. Build normally, e.g. using ``pw watch``. 99 100.. _run: 101 102Building and running fuzzers with Bazel 103======================================= 104To build a fuzzer, do the following: 105 1061. Add the Bazel target using ``pw_cc_fuzz_test`` macro. 107 108.. code:: py 109 110 load("@pigweed//pw_fuzzer:fuzzer.bzl", "pw_cc_fuzz_test") 111 112 pw_cc_fuzz_test( 113 name = "my_fuzz_test", 114 srcs = ["my_fuzzer.cc"], 115 deps = [ 116 "@pigweed//pw_fuzzer", 117 ":my_lib", 118 ], 119 ) 120 1212. Build and run the fuzzer. 122 123.. code:: sh 124 125 bazel test //my_module:my_fuzz_test 126 1273. Swap fuzzer backend to use ASAN fuzzing engine. 128 129.. code:: 130 131 # .bazelrc 132 # Define the --config=asan-libfuzzer configuration. 133 build:asan-libfuzzer \ 134 --@rules_fuzzing//fuzzing:cc_engine=@rules_fuzzing//fuzzing/engines:libfuzzer 135 build:asan-libfuzzer \ 136 --@rules_fuzzing//fuzzing:cc_engine_instrumentation=libfuzzer 137 build:asan-libfuzzer --@rules_fuzzing//fuzzing:cc_engine_sanitizer=asan 138 1394. Re-run fuzz tests. 140 141.. code:: 142 143 bazel test //my_module:my_fuzz_test --config asan-libfuzzer 144 145Running fuzzers locally 146======================= 147 148Based on the example above, the fuzzer output will be at 149``out/host/obj/my_module/my_fuzzer``. It can be invoked using the normal 150`libFuzzer options`_ and `sanitizer runtime flags`_. For even more details, see 151the libFuzzer section on `running a fuzzer`_. 152 153For example, the following invocation disables "one definition rule" detection, 154saves failing inputs to ``artifacts/``, treats any input that takes longer than 15510 seconds as a failure, and stores the working corpus in ``corpus/``. 156 157.. code:: 158 159 $ mkdir -p corpus 160 $ ASAN_OPTIONS=detect_odr_violation=0 \ 161 out/host_clang_fuzz/obj/pw_fuzzer/bin/toy_fuzzer \ 162 -artifact_prefix=artifacts/ \ 163 -timeout=10 \ 164 corpus 165 INFO: Seed: 305325345 166 INFO: Loaded 1 modules (46 inline 8-bit counters): 46 [0x38dfc0, 0x38dfee), 167 INFO: Loaded 1 PC tables (46 PCs): 46 [0x23aaf0,0x23add0), 168 INFO: 0 files found in corpus 169 INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes 170 INFO: A corpus is not provided, starting from an empty corpus 171 #2 INITED cov: 2 ft: 3 corp: 1/1b exec/s: 0 rss: 27Mb 172 #4 NEW cov: 3 ft: 4 corp: 2/3b lim: 4 exec/s: 0 rss: 27Mb L: 2/2 MS: 2 ShuffleBytes-InsertByte- 173 #11 NEW cov: 7 ft: 8 corp: 3/7b lim: 4 exec/s: 0 rss: 27Mb L: 4/4 MS: 2 EraseBytes-CrossOver- 174 #27 REDUCE cov: 7 ft: 8 corp: 3/6b lim: 4 exec/s: 0 rss: 27Mb L: 3/3 MS: 1 EraseBytes- 175 #29 REDUCE cov: 7 ft: 8 corp: 3/5b lim: 4 exec/s: 0 rss: 27Mb L: 2/2 MS: 2 ChangeBit-EraseBytes- 176 #445 REDUCE cov: 9 ft: 10 corp: 4/13b lim: 8 exec/s: 0 rss: 27Mb L: 8/8 MS: 1 InsertRepeatedBytes- 177 #12104 NEW cov: 11 ft: 12 corp: 5/24b lim: 122 exec/s: 0 rss: 28Mb L: 11/11 MS: 4 CMP-InsertByte-ShuffleBytes-ChangeByte- DE: "\xff\xff"- 178 #12321 NEW cov: 12 ft: 13 corp: 6/31b lim: 122 exec/s: 0 rss: 28Mb L: 7/11 MS: 2 CopyPart-EraseBytes- 179 #12459 REDUCE cov: 12 ft: 13 corp: 6/28b lim: 122 exec/s: 0 rss: 28Mb L: 8/8 MS: 3 CMP-InsertByte-EraseBytes- DE: "\x00\x00"- 180 #12826 REDUCE cov: 12 ft: 13 corp: 6/26b lim: 122 exec/s: 0 rss: 28Mb L: 5/8 MS: 2 ShuffleBytes-EraseBytes- 181 #14824 REDUCE cov: 12 ft: 13 corp: 6/25b lim: 135 exec/s: 0 rss: 28Mb L: 4/8 MS: 3 PersAutoDict-ShuffleBytes-EraseBytes- DE: "\x00\x00"- 182 #15106 REDUCE cov: 12 ft: 13 corp: 6/24b lim: 135 exec/s: 0 rss: 28Mb L: 3/8 MS: 2 ChangeByte-EraseBytes- 183 ... 184 #197809 REDUCE cov: 35 ft: 36 corp: 22/129b lim: 1800 exec/s: 0 rss: 79Mb L: 9/9 MS: 1 InsertByte- 185 #216250 REDUCE cov: 35 ft: 36 corp: 22/128b lim: 1980 exec/s: 0 rss: 87Mb L: 8/8 MS: 1 EraseBytes- 186 #242761 REDUCE cov: 35 ft: 36 corp: 22/127b lim: 2237 exec/s: 0 rss: 101Mb L: 7/8 MS: 1 EraseBytes- 187 ==126148== ERROR: libFuzzer: deadly signal 188 #0 0x35b981 in __sanitizer_print_stack_trace ../recipe_cleanup/clangFu99hg/llvm_build_dir/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/asan/asan_stack.cpp:86:3 189 #1 0x2bcdb5 in fuzzer::PrintStackTrace() (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2bcdb5) 190 #2 0x2a2ac9 in fuzzer::Fuzzer::CrashCallback() (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a2ac9) 191 #3 0x7f866684151f (/lib/x86_64-linux-gnu/libpthread.so.0+0x1351f) 192 #4 0x3831df in (anonymous namespace)::toy_example(char const*, char const*) /home/aarongreen/src/pigweed/out/host/../../pw_fuzzer/examples/toy_fuzzer.cc:49:15 193 #5 0x3831df in LLVMFuzzerTestOneInput /home/aarongreen/src/pigweed/out/host/../../pw_fuzzer/examples/toy_fuzzer.cc:80:3 194 #6 0x2a4025 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a4025) 195 #7 0x2a3774 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a3774) 196 #8 0x2a5769 in fuzzer::Fuzzer::MutateAndTestOne() (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a5769) 197 #9 0x2a6185 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a6185) 198 #10 0x294c8a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x294c8a) 199 #11 0x2bd422 in main ../recipe_cleanup/clangFu99hg/llvm_build_dir/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10 200 #12 0x7f8666684bba in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26bba) 201 #13 0x26ae19 in _start (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x26ae19) 202 203 NOTE: libFuzzer has rudimentary signal handlers. 204 Combine libFuzzer with AddressSanitizer or similar for better crash reports. 205 SUMMARY: libFuzzer: deadly signal 206 MS: 1 CrossOver-; base unit: 9f479f7a6e3a21363397a25da3168218ba182a16 207 0x68,0x65,0x6c,0x6c,0x6f,0x0,0x77,0x6f,0x72,0x6c,0x64,0x0,0x0,0x0, 208 hello\x00world\x00\x00\x00 209 artifact_prefix='artifacts'; Test unit written to artifacts/crash-6e4fdc7ffd04131ea15dd243a0890b1b606f4831 210 Base64: aGVsbG8Ad29ybGQAAAA= 211 212Running fuzzers on OSS-Fuzz 213=========================== 214 215Pigweed is integrated with `OSS-Fuzz`_, a continuous fuzzing infrastructure for 216open source software. Fuzzers listed in in ``pw_test_groups`` will automatically 217start being run within a day or so of appearing in the git repository. 218 219Bugs produced by OSS-Fuzz can be found in its `Monorail instance`_. These bugs 220include: 221 222* A detailed report, including a symbolized backtrace. 223* A revision range indicating when the bug has been detected. 224* A minimized testcase, which is a fuzzer input that can be used to reproduce 225 the bug. 226 227To reproduce a bug: 228 229#. Build_ the fuzzers as described above. 230#. Download the minimized testcase. 231#. Run_ the fuzzer with the testcase as an argument. 232 233For example, if the testcase is saved as "~/Downloads/testcase" 234and the fuzzer is the same as in the examples above, you could run: 235 236.. code:: 237 238 $ ./out/host/obj/pw_fuzzer/toy_fuzzer ~/Downloads/testcase 239 240If you need to recreate the OSS-Fuzz environment locally, you can use its 241documentation on `reproducing`_ issues. 242 243In particular, you can recreate the OSS-Fuzz environment using: 244 245.. code:: 246 247 $ python infra/helper.py pull_images 248 $ python infra/helper.py build_image pigweed 249 $ python infra/helper.py build_fuzzers --sanitizer <address/undefined> pigweed 250 251With that environment, you can run the reproduce bugs using: 252 253.. code:: 254 255 python infra/helper.py reproduce pigweed <pw_module>_<fuzzer_name> ~/Downloads/testcase 256 257You can even verify fixes in your local source checkout: 258 259.. code:: 260 261 $ python infra/helper.py build_fuzzers --sanitizer <address/undefined> pigweed $PW_ROOT 262 $ python infra/helper.py reproduce pigweed <pw_module>_<fuzzer_name> ~/Downloads/testcase 263 264.. _compiler_rt: https://compiler-rt.llvm.org/ 265.. _corpus: https://llvm.org/docs/LibFuzzer.html#corpus 266.. _FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION: https://llvm.org/docs/LibFuzzer.html#fuzzer-friendly-build-mode 267.. _FuzzedDataProvider: https://github.com/llvm/llvm-project/blob/HEAD/compiler-rt/include/fuzzer/FuzzedDataProvider.h 268.. _libFuzzer: https://llvm.org/docs/LibFuzzer.html 269.. _libFuzzer options: https://llvm.org/docs/LibFuzzer.html#options 270.. _LLVMFuzzerTestOneInput: https://llvm.org/docs/LibFuzzer.html#fuzz-target 271.. _monorail instance: https://bugs.chromium.org/p/oss-fuzz 272.. _oss-fuzz: https://github.com/google/oss-fuzz 273.. _reproducing: https://google.github.io/oss-fuzz/advanced-topics/reproducing/ 274.. _running a fuzzer: https://llvm.org/docs/LibFuzzer.html#running 275.. _sanitizer runtime flags: https://github.com/google/sanitizers/wiki/SanitizerCommonFlags 276.. _split a fuzzing input: https://github.com/google/fuzzing/blob/HEAD/docs/split-inputs.md 277.. _startup initialization: https://llvm.org/docs/LibFuzzer.html#startup-initialization 278.. _structure aware fuzzing: https://github.com/google/fuzzing/blob/HEAD/docs/structure-aware-fuzzing.md 279.. _valid options: https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html 280
