1.. _module-pw_fuzzer: 2 3--------- 4pw_fuzzer 5--------- 6``pw_fuzzer`` provides developers with tools to write `libFuzzer`_ based 7fuzzers. 8 9Fuzzing or fuzz testing is style of testing that stochastically generates inputs 10to targeted interfaces in order to automatically find defects and/or 11vulnerabilities. In other words, fuzzing is simply an automated way of testing 12APIs with generated data. 13 14A fuzzer is a program that is used to fuzz a interface. It typically has three 15steps that it executes repeatedly: 16 17#. Generate a new, context-free input. This is the *fuzzing engine*. For 18 ``pw_fuzzer``, this is `libFuzzer`_. 19#. Use the input to exercise the targeted interface, or code being tested. This 20 is the *fuzz target function*. For ``pw_fuzzer``, these are the GN 21 ``sources`` and/or ``deps`` that define `LLVMFuzzerTestOneInput`_. 22#. Monitor the code being tested for any abnormal conditions. This is the 23 *instrumentation*. For ``pw_fuzzer``, these are sanitizer runtimes from 24 LLVM's `compiler_rt`_. 25 26.. note:: 27 28 ``pw_fuzzer`` is currently only supported on Linux and MacOS using clang. 29 30.. image:: doc_resources/pw_fuzzer_coverage_guided.png 31 :alt: Coverage Guided Fuzzing with libFuzzer 32 :align: left 33 34Writing fuzzers 35=============== 36 37To write a fuzzer, a developer needs to write a fuzz target function follwing 38the `fuzz target function`__ guidelines given by libFuzzer: 39 40.. code:: cpp 41 42 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { 43 DoSomethingInterestingWithMyAPI(data, size); 44 return 0; // Non-zero return values are reserved for future use. 45 } 46 47.. __: LLVMFuzzerTestOneInput_ 48 49When writing you fuzz target function, you may want to consider: 50 51- It is acceptable to return early if the input doesn't meet some constraints, 52 e.g. it is too short. 53- If your fuzzer accepts data with a well-defined format, you can bootstrap 54 coverage by crafting examples and adding them to a `corpus`_. 55- There are tools to `split a fuzzing input`_ into multiple fields if needed; 56 the `FuzzedDataProvider`_ is particularly easy to use. 57- If your code acts on "transformed" inputs, such as encoded or compressed 58 inputs, you may want to try `structure aware fuzzing`. 59- You can do `startup initialization`_ if you need to. 60- If your code is non-deterministic or uses checksums, you may want to disable 61 those **only** when fuzzing by using LLVM's 62 `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION`_ 63 64.. _build: 65 66Building fuzzers with GN 67======================== 68 69To build a fuzzer, do the following: 70 711. Add the GN target to the module using ``pw_fuzzer`` GN template. If you wish 72 to limit when the generated unit test is run, you can set `enable_test_if` in 73 the same manner as `enable_if` for `pw_test`: 74 75.. code:: 76 77 # In $dir_my_module/BUILD.gn 78 import("$dir_pw_fuzzer/fuzzer.gni") 79 80 pw_fuzzer("my_fuzzer") { 81 sources = [ "my_fuzzer.cc" ] 82 deps = [ ":my_lib" ] 83 enable_test_if = device_has_1m_flash 84 } 85 862. Add the generated unit test to the module's test group. This test verifies 87 the fuzzer can build and run, even when not being built in a fuzzing 88 toolchain. 89 90.. code:: 91 92 # In $dir_my_module/BUILD.gn 93 pw_test_group("tests") { 94 tests = [ 95 ... 96 ":my_fuzzer_test", 97 ] 98 } 99 1003. If your module does not already have a group of fuzzers, add it and include 101 it in the top level fuzzers target. Depending on your project, the specific 102 toolchain may differ. Fuzzer toolchains are those with 103 ``pw_toolchain_FUZZING_ENABLED`` set to true. Examples include 104 ``host_clang_fuzz`` and any toolchains that extend it. 105 106.. code:: 107 108 # In //BUILD.gn 109 group("fuzzers") { 110 deps = [ 111 ... 112 "$dir_my_module:fuzzers($dir_pigweed/targets/host:host_clang_fuzz)", 113 ] 114 } 115 1164. Add your fuzzer to the module's group of fuzzers. 117 118.. code:: 119 120 group("fuzzers") { 121 deps = [ 122 ... 123 ":my_fuzzer", 124 ] 125 } 126 1275. If desired, select a sanitizer runtime. By default, 128 `//targets/host:host_clang_fuzz` uses "address" if no sanitizer is specified. 129 See LLVM for `valid options`_. 130 131.. code:: sh 132 133 $ gn gen out --args='pw_toolchain_SANITIZERS=["address"]' 134 1356. Build the fuzzers! 136 137.. code:: sh 138 139 $ ninja -C out fuzzers 140 141.. _bazel: 142 143Building and running fuzzers with Bazel 144======================================= 145To build a fuzzer, do the following: 146 1471. Add the Bazel target using ``pw_cc_fuzz_test`` macro. 148 149.. code:: py 150 151 load("@pigweed//pw_fuzzer:fuzzer.bzl", "pw_cc_fuzz_test") 152 153 pw_cc_fuzz_test( 154 name = "my_fuzz_test", 155 srcs = ["my_fuzzer.cc"], 156 deps = [ 157 "@pigweed//pw_fuzzer", 158 ":my_lib", 159 ], 160 ) 161 1622. Build and run the fuzzer. 163 164.. code:: sh 165 166 bazel test //my_module:my_fuzz_test 167 1683. Swap fuzzer backend to use ASAN fuzzing engine. 169 170.. code:: 171 172 # .bazelrc 173 # Define the --config=asan-libfuzzer configuration. 174 build:asan-libfuzzer \ 175 --@rules_fuzzing//fuzzing:cc_engine=@rules_fuzzing//fuzzing/engines:libfuzzer 176 build:asan-libfuzzer \ 177 --@rules_fuzzing//fuzzing:cc_engine_instrumentation=libfuzzer 178 build:asan-libfuzzer --@rules_fuzzing//fuzzing:cc_engine_sanitizer=asan 179 1804. Re-run fuzz tests. 181 182.. code:: 183 184 bazel test //my_module:my_fuzz_test --config asan-libfuzzer 185 186.. _run: 187 188Running fuzzers locally 189======================= 190 191Based on the example above, the fuzzer output will be at 192``out/host/obj/my_module/my_fuzzer``. It can be invoked using the normal 193`libFuzzer options`_ and `sanitizer runtime flags`_. For even more details, see 194the libFuzzer section on `running a fuzzer`_. 195 196For example, the following invocation disables "one definition rule" detection, 197saves failing inputs to ``artifacts/``, treats any input that takes longer than 19810 seconds as a failure, and stores the working corpus in ``corpus/``. 199 200.. code:: 201 202 $ mkdir -p corpus 203 $ ASAN_OPTIONS=detect_odr_violation=0 \ 204 out/host_clang_fuzz/obj/pw_fuzzer/bin/toy_fuzzer \ 205 -artifact_prefix=artifacts/ \ 206 -timeout=10 \ 207 corpus 208 INFO: Seed: 305325345 209 INFO: Loaded 1 modules (46 inline 8-bit counters): 46 [0x38dfc0, 0x38dfee), 210 INFO: Loaded 1 PC tables (46 PCs): 46 [0x23aaf0,0x23add0), 211 INFO: 0 files found in corpus 212 INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes 213 INFO: A corpus is not provided, starting from an empty corpus 214 #2 INITED cov: 2 ft: 3 corp: 1/1b exec/s: 0 rss: 27Mb 215 #4 NEW cov: 3 ft: 4 corp: 2/3b lim: 4 exec/s: 0 rss: 27Mb L: 2/2 MS: 2 ShuffleBytes-InsertByte- 216 #11 NEW cov: 7 ft: 8 corp: 3/7b lim: 4 exec/s: 0 rss: 27Mb L: 4/4 MS: 2 EraseBytes-CrossOver- 217 #27 REDUCE cov: 7 ft: 8 corp: 3/6b lim: 4 exec/s: 0 rss: 27Mb L: 3/3 MS: 1 EraseBytes- 218 #29 REDUCE cov: 7 ft: 8 corp: 3/5b lim: 4 exec/s: 0 rss: 27Mb L: 2/2 MS: 2 ChangeBit-EraseBytes- 219 #445 REDUCE cov: 9 ft: 10 corp: 4/13b lim: 8 exec/s: 0 rss: 27Mb L: 8/8 MS: 1 InsertRepeatedBytes- 220 #12104 NEW cov: 11 ft: 12 corp: 5/24b lim: 122 exec/s: 0 rss: 28Mb L: 11/11 MS: 4 CMP-InsertByte-ShuffleBytes-ChangeByte- DE: "\xff\xff"- 221 #12321 NEW cov: 12 ft: 13 corp: 6/31b lim: 122 exec/s: 0 rss: 28Mb L: 7/11 MS: 2 CopyPart-EraseBytes- 222 #12459 REDUCE cov: 12 ft: 13 corp: 6/28b lim: 122 exec/s: 0 rss: 28Mb L: 8/8 MS: 3 CMP-InsertByte-EraseBytes- DE: "\x00\x00"- 223 #12826 REDUCE cov: 12 ft: 13 corp: 6/26b lim: 122 exec/s: 0 rss: 28Mb L: 5/8 MS: 2 ShuffleBytes-EraseBytes- 224 #14824 REDUCE cov: 12 ft: 13 corp: 6/25b lim: 135 exec/s: 0 rss: 28Mb L: 4/8 MS: 3 PersAutoDict-ShuffleBytes-EraseBytes- DE: "\x00\x00"- 225 #15106 REDUCE cov: 12 ft: 13 corp: 6/24b lim: 135 exec/s: 0 rss: 28Mb L: 3/8 MS: 2 ChangeByte-EraseBytes- 226 ... 227 #197809 REDUCE cov: 35 ft: 36 corp: 22/129b lim: 1800 exec/s: 0 rss: 79Mb L: 9/9 MS: 1 InsertByte- 228 #216250 REDUCE cov: 35 ft: 36 corp: 22/128b lim: 1980 exec/s: 0 rss: 87Mb L: 8/8 MS: 1 EraseBytes- 229 #242761 REDUCE cov: 35 ft: 36 corp: 22/127b lim: 2237 exec/s: 0 rss: 101Mb L: 7/8 MS: 1 EraseBytes- 230 ==126148== ERROR: libFuzzer: deadly signal 231 #0 0x35b981 in __sanitizer_print_stack_trace ../recipe_cleanup/clangFu99hg/llvm_build_dir/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/asan/asan_stack.cpp:86:3 232 #1 0x2bcdb5 in fuzzer::PrintStackTrace() (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2bcdb5) 233 #2 0x2a2ac9 in fuzzer::Fuzzer::CrashCallback() (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a2ac9) 234 #3 0x7f866684151f (/lib/x86_64-linux-gnu/libpthread.so.0+0x1351f) 235 #4 0x3831df in (anonymous namespace)::toy_example(char const*, char const*) /home/aarongreen/src/pigweed/out/host/../../pw_fuzzer/examples/toy_fuzzer.cc:49:15 236 #5 0x3831df in LLVMFuzzerTestOneInput /home/aarongreen/src/pigweed/out/host/../../pw_fuzzer/examples/toy_fuzzer.cc:80:3 237 #6 0x2a4025 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a4025) 238 #7 0x2a3774 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a3774) 239 #8 0x2a5769 in fuzzer::Fuzzer::MutateAndTestOne() (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a5769) 240 #9 0x2a6185 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x2a6185) 241 #10 0x294c8a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x294c8a) 242 #11 0x2bd422 in main ../recipe_cleanup/clangFu99hg/llvm_build_dir/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10 243 #12 0x7f8666684bba in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26bba) 244 #13 0x26ae19 in _start (/home/aarongreen/src/pigweed/out/host/obj/pw_fuzzer/toy_fuzzer+0x26ae19) 245 246 NOTE: libFuzzer has rudimentary signal handlers. 247 Combine libFuzzer with AddressSanitizer or similar for better crash reports. 248 SUMMARY: libFuzzer: deadly signal 249 MS: 1 CrossOver-; base unit: 9f479f7a6e3a21363397a25da3168218ba182a16 250 0x68,0x65,0x6c,0x6c,0x6f,0x0,0x77,0x6f,0x72,0x6c,0x64,0x0,0x0,0x0, 251 hello\x00world\x00\x00\x00 252 artifact_prefix='artifacts'; Test unit written to artifacts/crash-6e4fdc7ffd04131ea15dd243a0890b1b606f4831 253 Base64: aGVsbG8Ad29ybGQAAAA= 254 255Running fuzzers on OSS-Fuzz 256=========================== 257 258Pigweed is integrated with `OSS-Fuzz`_, a continuous fuzzing infrastructure for 259open source software. Fuzzers listed in in ``pw_test_groups`` will automatically 260start being run within a day or so of appearing in the git repository. 261 262Bugs produced by OSS-Fuzz can be found in its `Monorail instance`_. These bugs 263include: 264 265* A detailed report, including a symbolized backtrace. 266* A revision range indicating when the bug has been detected. 267* A minimized testcase, which is a fuzzer input that can be used to reproduce 268 the bug. 269 270To reproduce a bug: 271 272#. Build_ the fuzzers as described above. 273#. Download the minimized testcase. 274#. Run_ the fuzzer with the testcase as an argument. 275 276For example, if the testcase is saved as "~/Downloads/testcase" 277and the fuzzer is the same as in the examples above, you could run: 278 279.. code:: 280 281 $ ./out/host/obj/pw_fuzzer/toy_fuzzer ~/Downloads/testcase 282 283If you need to recreate the OSS-Fuzz environment locally, you can use its 284documentation on `reproducing`_ issues. 285 286In particular, you can recreate the OSS-Fuzz environment using: 287 288.. code:: 289 290 $ python infra/helper.py pull_images 291 $ python infra/helper.py build_image pigweed 292 $ python infra/helper.py build_fuzzers --sanitizer <address/undefined> pigweed 293 294With that environment, you can run the reproduce bugs using: 295 296.. code:: 297 298 python infra/helper.py reproduce pigweed <pw_module>_<fuzzer_name> ~/Downloads/testcase 299 300You can even verify fixes in your local source checkout: 301 302.. code:: 303 304 $ python infra/helper.py build_fuzzers --sanitizer <address/undefined> pigweed $PW_ROOT 305 $ python infra/helper.py reproduce pigweed <pw_module>_<fuzzer_name> ~/Downloads/testcase 306 307.. _compiler_rt: https://compiler-rt.llvm.org/ 308.. _corpus: https://llvm.org/docs/LibFuzzer.html#corpus 309.. _FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION: https://llvm.org/docs/LibFuzzer.html#fuzzer-friendly-build-mode 310.. _FuzzedDataProvider: https://github.com/llvm/llvm-project/blob/HEAD/compiler-rt/include/fuzzer/FuzzedDataProvider.h 311.. _libFuzzer: https://llvm.org/docs/LibFuzzer.html 312.. _libFuzzer options: https://llvm.org/docs/LibFuzzer.html#options 313.. _LLVMFuzzerTestOneInput: https://llvm.org/docs/LibFuzzer.html#fuzz-target 314.. _monorail instance: https://bugs.chromium.org/p/oss-fuzz 315.. _oss-fuzz: https://github.com/google/oss-fuzz 316.. _reproducing: https://google.github.io/oss-fuzz/advanced-topics/reproducing/ 317.. _running a fuzzer: https://llvm.org/docs/LibFuzzer.html#running 318.. _sanitizer runtime flags: https://github.com/google/sanitizers/wiki/SanitizerCommonFlags 319.. _split a fuzzing input: https://github.com/google/fuzzing/blob/HEAD/docs/split-inputs.md 320.. _startup initialization: https://llvm.org/docs/LibFuzzer.html#startup-initialization 321.. _structure aware fuzzing: https://github.com/google/fuzzing/blob/HEAD/docs/structure-aware-fuzzing.md 322.. _valid options: https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html 323
