1# Best practices 2 3## Contents 4 5### Targets 6 7* [Fuzzing a target with source code available](#fuzzing-a-target-with-source-code-available) 8* [Fuzzing a target with dlopen() instrumented libraries](#fuzzing-a-target-with-dlopen-instrumented-libraries) 9* [Fuzzing a binary-only target](#fuzzing-a-binary-only-target) 10* [Fuzzing a GUI program](#fuzzing-a-gui-program) 11* [Fuzzing a network service](#fuzzing-a-network-service) 12 13### Improvements 14 15* [Improving speed](#improving-speed) 16* [Improving stability](#improving-stability) 17 18## Targets 19 20### Fuzzing a target with source code available 21 22To learn how to fuzz a target if source code is available, see 23[fuzzing_in_depth.md](fuzzing_in_depth.md). 24 25### Fuzzing a target with dlopen instrumented libraries 26 27If a source code based fuzzing target loads instrumented libraries with 28dlopen() after the forkserver has been activated and non-colliding coverage 29instrumentation is used (PCGUARD (which is the default), or LTO), then this 30an issue, because this would enlarge the coverage map, but afl-fuzz doesn't 31know about it. 32 33The solution is to use `AFL_PRELOAD` for all dlopen()'ed libraries to 34ensure that all coverage targets are present on startup in the target, 35even if accessed only later with dlopen(). 36 37For PCGUARD instrumentation `abort()` is called if this is detected, for LTO 38there will either be no coverage for the instrumented dlopen()'ed libraries or 39you will see lots of crashes in the UI. 40 41Note that this is not an issue if you use the inferiour `afl-gcc-fast`, 42`afl-gcc` or`AFL_LLVM_INSTRUMENT=CLASSIC/NGRAM/CTX afl-clang-fast` 43instrumentation. 44 45### Fuzzing a binary-only target 46 47For a comprehensive guide, see 48[fuzzing_binary-only_targets.md](fuzzing_binary-only_targets.md). 49 50### Fuzzing a GUI program 51 52If the GUI program can read the fuzz data from a file (via the command line, a 53fixed location or via an environment variable) without needing any user 54interaction, then it would be suitable for fuzzing. 55 56Otherwise, it is not possible without modifying the source code - which is a 57very good idea anyway as the GUI functionality is a huge CPU/time overhead for 58the fuzzing. 59 60So create a new `main()` that just reads the test case and calls the 61functionality for processing the input that the GUI program is using. 62 63### Fuzzing a network service 64 65Fuzzing a network service does not work "out of the box". 66 67Using a network channel is inadequate for several reasons: 68- it has a slow-down of x10-20 on the fuzzing speed 69- it does not scale to fuzzing multiple instances easily, 70- instead of one initial data packet often a back-and-forth interplay of packets 71 is needed for stateful protocols (which is totally unsupported by most 72 coverage aware fuzzers). 73 74The established method to fuzz network services is to modify the source code to 75read from a file or stdin (fd 0) (or even faster via shared memory, combine this 76with persistent mode 77[instrumentation/README.persistent_mode.md](../instrumentation/README.persistent_mode.md) 78and you have a performance gain of x10 instead of a performance loss of over x10 79- that is a x100 difference!). 80 81If modifying the source is not an option (e.g., because you only have a binary 82and perform binary fuzzing) you can also use a shared library with AFL_PRELOAD 83to emulate the network. This is also much faster than the real network would be. 84See [utils/socket_fuzzing/](../utils/socket_fuzzing/). 85 86There is an outdated AFL++ branch that implements networking if you are 87desperate though: 88[https://github.com/AFLplusplus/AFLplusplus/tree/networking](https://github.com/AFLplusplus/AFLplusplus/tree/networking) 89- however, a better option is AFLnet 90([https://github.com/aflnet/aflnet](https://github.com/aflnet/aflnet)) which 91allows you to define network state with different type of data packets. 92 93## Improvements 94 95### Improving speed 96 971. Use [llvm_mode](../instrumentation/README.llvm.md): afl-clang-lto (llvm >= 98 11) or afl-clang-fast (llvm >= 9 recommended). 992. Use [persistent mode](../instrumentation/README.persistent_mode.md) (x2-x20 100 speed increase). 1013. Instrument just what you are interested in, see 102 [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md). 1034. If you do not use shmem persistent mode, use `AFL_TMPDIR` to put the input 104 file directory on a tempfs location, see 105 [env_variables.md](env_variables.md). 1065. Improve Linux kernel performance: modify `/etc/default/grub`, set 107 `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off 108 mitigations=off no_stf_barrier noibpb noibrs nopcid nopti 109 nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off 110 spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then 111 `update-grub` and `reboot` (warning: makes the system less secure). 1126. Running on an `ext2` filesystem with `noatime` mount option will be a bit 113 faster than on any other journaling filesystem. 1147. Use your cores 115 ([fuzzing_in_depth.md:3c) Using multiple cores](fuzzing_in_depth.md#c-using-multiple-cores))! 116 117### Improving stability 118 119For fuzzing, a 100% stable target that covers all edges is the best case. A 90% 120stable target that covers all edges is, however, better than a 100% stable 121target that ignores 10% of the edges. 122 123With instability, you basically have a partial coverage loss on an edge, with 124ignored functions you have a full loss on that edges. 125 126There are functions that are unstable, but also provide value to coverage, e.g., 127init functions that use fuzz data as input. If, however, a function that has 128nothing to do with the input data is the source of instability, e.g., checking 129jitter, or is a hash map function etc., then it should not be instrumented. 130 131To be able to exclude these functions (based on AFL++'s measured stability), the 132following process will allow to identify functions with variable edges. 133 134Four steps are required to do this and it also requires quite some knowledge of 135coding and/or disassembly and is effectively possible only with `afl-clang-fast` 136`PCGUARD` and `afl-clang-lto` `LTO` instrumentation. 137 138 1. Instrument to be able to find the responsible function(s): 139 140 a) For LTO instrumented binaries, this can be documented during compile 141 time, just set `export AFL_LLVM_DOCUMENT_IDS=/path/to/a/file`. This file 142 will have one assigned edge ID and the corresponding function per line. 143 144 b) For PCGUARD instrumented binaries, it is much more difficult. Here you 145 can either modify the `__sanitizer_cov_trace_pc_guard` function in 146 `instrumentation/afl-llvm-rt.o.c` to write a backtrace to a file if the 147 ID in `__afl_area_ptr[*guard]` is one of the unstable edge IDs. (Example 148 code is already there). Then recompile and reinstall `llvm_mode` and 149 rebuild your target. Run the recompiled target with `afl-fuzz` for a 150 while and then check the file that you wrote with the backtrace 151 information. Alternatively, you can use `gdb` to hook 152 `__sanitizer_cov_trace_pc_guard_init` on start, check to which memory 153 address the edge ID value is written, and set a write breakpoint to that 154 address (`watch 0x.....`). 155 156 c) In other instrumentation types, this is not possible. So just recompile 157 with the two mentioned above. This is just for identifying the functions 158 that have unstable edges. 159 160 2. Identify which edge ID numbers are unstable. 161 162 Run the target with `export AFL_DEBUG=1` for a few minutes then terminate. 163 The out/fuzzer_stats file will then show the edge IDs that were identified 164 as unstable in the `var_bytes` entry. You can match these numbers directly 165 to the data you created in the first step. Now you know which functions are 166 responsible for the instability 167 168 3. Create a text file with the filenames/functions 169 170 Identify which source code files contain the functions that you need to 171 remove from instrumentation, or just specify the functions you want to skip 172 for instrumentation. Note that optimization might inline functions! 173 174 Follow this document on how to do this: 175 [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md). 176 177 If `PCGUARD` is used, then you need to follow this guide (needs llvm 12+!): 178 [https://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](https://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) 179 180 Only exclude those functions from instrumentation that provide no value for 181 coverage - that is if it does not process any fuzz data directly or 182 indirectly (e.g., hash maps, thread management etc.). If, however, a 183 function directly or indirectly handles fuzz data, then you should not put 184 the function in a deny instrumentation list and rather live with the 185 instability it comes with. 186 187 4. Recompile the target 188 189 Recompile, fuzz it, be happy :) 190 191 This link explains this process for 192 [Fuzzbench](https://github.com/google/fuzzbench/issues/677). 193