Lines Matching +full:valgrind +full:- +full:fuzz +full:- +full:test
3 The following describes how to fuzz with a target if source code is available.
4 If you have a binary-only target, go to
5 [fuzzing_binary-only_targets.md](fuzzing_binary-only_targets.md).
7 Fuzzing source code is a three-step process:
18 Please keep in mind that, similarly to many other computationally-intensive
21 - Your CPU will run hot and will need adequate cooling. In most cases, if
27 - Targeted programs may end up erratically grabbing gigabytes of memory or
33 - Fuzzing involves billions of reads and writes to the filesystem. On modern
35 "physical" I/O - but there are many factors that may alter this equation. It
42 $ iostat -d 3 -x -k [...optional disk ID...]
45 Using the `AFL_TMPDIR` environment variable and a RAM-disk, you can have the
50 …# docker run -ti --mount type=tmpfs,destination=/ramdisk -e AFL_TMPDIR=/ramdisk aflplusplus/aflplu…
57 AFL++ comes with a central compiler `afl-cc` that incorporates various different
65 +--------------------------------+
66 | clang/clang++ 11+ is available | --> use LTO mode (afl-clang-lto/afl-clang-lto++)
67 +--------------------------------+ see [instrumentation/README.lto.md](instrumentation/README.l…
69 | if not, or if the target fails with LTO afl-clang-lto/++
72 +---------------------------------+
73 | clang/clang++ 3.8+ is available | --> use LLVM mode (afl-clang-fast/afl-clang-fast++)
74 +---------------------------------+ see [instrumentation/README.llvm.md](instrumentation/README…
76 | if not, or if the target fails with LLVM afl-clang-fast/++
79 +--------------------------------+
80 | gcc 5+ is available | -> use GCC_PLUGIN mode (afl-gcc-fast/afl-g++-fast)
81 …+--------------------------------+ see [instrumentation/README.gcc_plugin.md](instrumentation/R…
87 use GCC mode (afl-gcc/afl-g++) (or afl-clang/afl-clang++ for clang)
92 * [LTO mode - afl-clang-lto](../instrumentation/README.lto.md)
93 * [LLVM mode - afl-clang-fast](../instrumentation/README.llvm.md)
94 * [GCC_PLUGIN mode - afl-gcc-fast](../instrumentation/README.gcc_plugin.md)
95 * GCC/CLANG modes (afl-gcc/afl-clang) have no README as they have no own
98 You can select the mode for the afl-cc compiler by one of the following methods:
100 * Using a symlink to afl-cc: afl-gcc, afl-g++, afl-clang, afl-clang++,
101 afl-clang-fast, afl-clang-fast++, afl-clang-lto, afl-clang-lto++,
102 afl-gcc-fast, afl-g++-fast (recommended!).
104 * Passing --afl-`MODE` command line options to the compiler via
109 * LTO (afl-clang-lto*)
110 * LLVM (afl-clang-fast*)
111 * GCC_PLUGIN (afl-g*-fast) or GCC (afl-gcc/afl-g++)
112 * CLANG(afl-clang/afl-clang++)
114 Because no AFL++ specific command-line options are accepted (beside the
115 --afl-MODE command), the compile-time tools make fairly broad use of environment
116 variables, which can be listed with `afl-cc -hh` or looked up in
121 If you instrument with LTO mode (afl-clang-fast/afl-clang-lto), the following
126 large input corpus. This technique is called laf-intel or COMPCOV. To use
129 [instrumentation/README.laf-intel.md](../instrumentation/README.laf-intel.md).
130 * A different technique (and usually a better one than laf-intel) is to
133 locations. This technique is very fast and good - if the target does not
137 setting `AFL_LLVM_CMPLOG=1`, and pass this binary to afl-fuzz via the `-c`
144 (afl-clang-fast/afl-clang-lto/afl-gcc-fast), you have the option to selectively
145 instrument _parts_ of the target that you are interested in. For afl-clang-fast,
186 …ensitive Branch Coverage](../instrumentation/README.llvm.md#6-afl-context-sensitive-branch-coverag…
187 …trumentation/README.llvm.md#7) AFL++ N-Gram Branch Coverage](../instrumentation/README.llvm.md#7-a…
191 * [instrumentation/README.llvm.md#8-neverzero-counters](../instrumentation/README.llvm.md#8-neverze…
199 and RAM usage. Also, you should only run one afl-fuzz instance per sanitizer
200 type. This is enough because e.g. a use-after-free bug will be picked up by ASAN
201 (address sanitizer) anyway after syncing test cases from other fuzzing
204 The following sanitizers have built-in support in AFL++:
207 use-after-free, NULL pointer dereference, buffer overruns, etc. Enabled with
212 * UBSAN = Undefined Behavior SANitizer, finds instances where - by the C and C++
213 standards - undefined behavior happens, e.g., adding two signed integers where
219 is mostly reduced to detecting type confusion vulnerabilities - which is,
228 `export AFL_USE_LSAN=1` before compiling. To ignore the memory-leaking check
233 It is possible to further modify the behavior of the sanitizers at run-time by
234 setting `ASAN_OPTIONS=...`, `LSAN_OPTIONS` etc. - the available parameters can
235 be looked up in the sanitizer documentation of llvm/clang. afl-fuzz, however,
254 // say that the checksum or HMAC was fine - or whatever is required
267 compiler is used. Also - if possible - you should always configure the build
273 could accidentally type "make install" and install them system wide - so don't.
282 output (and think this means a test failure) - which is something AFL++ likes
286 2. Sometimes configure and build systems error on warnings - these should be
287 disabled (e.g., `--disable-werror` for some configure scripts).
299 CC=afl-clang-fast CXX=afl-clang-fast++ ./configure --disable-shared
302 Note that if you are using the (better) afl-clang-lto compiler, you also have to
303 set `AR` to llvm-ar[-VERSION] and `RANLIB` to llvm-ranlib[-VERSION] - as is
311 mkdir build; cd build; cmake -DCMAKE_C_COMPILER=afl-cc -DCMAKE_CXX_COMPILER=afl-c++ ..
314 Note that if you are using the (better) afl-clang-lto compiler you also have to
315 set AR to llvm-ar[-VERSION] and RANLIB to llvm-ranlib[-VERSION] - as is
324 CC=afl-cc CXX=afl-c++ meson
330 `RANLIB`/`AR` that is needed - because this was just not foreseen by the
331 developer of the target. Or they have non-standard options. Figure out if there
332 is a non-standard way to set this, otherwise set up the build normally and edit
359 If you just fuzz a target program as-is, you are wasting a great opportunity for
362 This variant requires the usage of afl-clang-lto, afl-clang-fast or
363 afl-gcc-fast.
365 It is the so-called `persistent mode`, which is much, much faster but requires
367 that you want to fuzz, plus a few specific AFL++ functions around it. See
371 Basically, if you do not fuzz a target in persistent mode, then you are just
372 doing it for a hobby and not professionally :-).
382 afl-clang-fast++ -fsanitize=fuzzer -o harness harness.cpp targetlib.a
388 The generated binary is fuzzed with afl-fuzz like any other fuzz target.
391 shared-memory test cases and hence gives you the fastest speed possible.
398 As you fuzz the target with mutated input, having as diverse inputs for the
409 reported bugs, test suites, random downloads from the internet, unit test case
410 data - from all kind of PNG software.
420 Use the AFL++ tool `afl-cmin` to remove inputs from the corpus that do not
423 1. Put all files from [step a](#a-collecting-inputs) into one directory, e.g.,
425 2. Run afl-cmin:
431 afl-cmin -i INPUTS -o INPUTS_UNIQUE -- bin/target -someopt @@
438 afl-cmin -i INPUTS -o INPUTS_UNIQUE -- bin/target -someopt
447 the better the fuzzing will be. This minimization is done with `afl-tmin`,
454 afl-tmin -i "$i" -o "../input/$i" -- bin/target -someopt @@
464 The INPUTS_UNIQUE/ directory from [step b](#b-making-the-input-corpus-unique) -
466 [step c](#c-minimizing-all-corpus-files) - is the resulting input corpus
467 directory to be used in fuzzing! :-)
471 In this final step, fuzz the target. There are not that many important options
472 to run the target - unless you want to use many CPU cores/threads for the
476 not seriously :-)
478 ### a) Running afl-fuzz
480 Before you do even a test run of afl-fuzz, execute `sudo afl-system-config` (on
481 the host if you execute afl-fuzz in a Docker container). This reconfigures the
482 system for optimal speed - which afl-fuzz checks and bails otherwise. Set
483 `export AFL_SKIP_CPUFREQ=1` for afl-fuzz to skip this check if you cannot run
484 afl-system-config with root privileges on the host for whatever reason.
488 * There is also `sudo afl-persistent-config` which sets additional permanent
494 If you have an input corpus from [step 2](#2-preparing-the-fuzzing-campaign),
495 then specify this directory with the `-i` option. Otherwise, create a new
496 directory and create a file with any content as test data in there.
500 step [2a) Collecting inputs](#a-collecting-inputs):
503 afl-fuzz -i input -o output -- bin/target -someopt @@
506 Note that the directory specified with `-o` will be created if it does not
509 It can be valuable to run afl-fuzz in a `screen` or `tmux` shell so you can log
510 off, or afl-fuzz is not aborted if you are running it in a remote ssh session
512 verified that your fuzzing setup works! Run it like `screen -dmS afl-main --
513 afl-fuzz -M main-$HOSTNAME -i ...` and it will start away in a screen session.
514 To enter this session, type `screen -r afl-main`. You see - it makes sense to
515 name the screen session same as the afl-fuzz `-M`/`-S` naming :-) For more
518 If you need to stop and re-start the fuzzing, use the same command line options
520 mode!) and switch the input directory with a dash (`-`):
523 afl-fuzz -i - -o output -- bin/target -someopt @@
530 data format, and tell afl-fuzz to load that dictionary by adding `-x
532 * With `afl-clang-lto`, you have an autodictionary generation for which you need
533 to do nothing except to use afl-clang-lto as the compiler.
534 * With `afl-clang-fast`, you can set
544 afl-fuzz has a variety of options that help to workaround target quirks like
545 very specific locations for the input file (`-f`), performing deterministic
546 fuzzing (`-D`) and many more. Check out `afl-fuzz -h`.
548 We highly recommend that you set a memory limit for running the target with `-m`
549 which defines the maximum memory in MB. This prevents a potential out-of-memory
551 handling in the target. Play around with various `-m` values until you find one
555 By default, afl-fuzz never stops fuzzing. To terminate AFL++, press Control-C or
559 When you start afl-fuzz, you will see a user interface that shows what the
565 [afl-fuzz_approach.md#understanding-the-status-screen](afl-fuzz_approach.md#understanding-the-statu…
569 Memory limits are not enforced by afl-fuzz by default and the system may run out
570 of memory. You can decrease the memory with the `-m` option, the value is in MB.
571 If this is too small for the target, you can usually see this by afl-fuzz
574 Consider setting low values for `-m` and `-t`.
577 can also try setting `-t` values that are more punishing than what `afl-fuzz`
578 dares to use on its own. On fast and idle machines, going down to `-t 5` may be
581 The `-m` parameter is worth looking at, too. Some programs can end up spending a
583 presented with pathological inputs. Low `-m` values can make them give up sooner
588 If you want to seriously fuzz, then use as many cores/threads as possible to
589 fuzz your target.
591 On the same machine - due to the design of how AFL++ works - there is a maximum
597 of the test cases. Depending on the average test case size (and those found
598 during fuzzing) and their number, a value between 50-500MB is recommended. You
602 There should be one main fuzzer (`-M main-$HOSTNAME` option - set also
603 `AFL_FINAL_SYNC=1`) and as many secondary fuzzers (e.g., `-S variant1`) as you
604 have cores that you use. Every `-M`/`-S` entry needs a unique name (that can be
605 whatever), however, the same `-o` output directory location has to be used for
609 * one should fuzz the target that was compiled with sanitizers activated
611 * one or two should fuzz the target with CMPLOG/redqueen (see above), at least
612 one cmplog instance should follow transformations (`-l 2AT`)
613 * one to three fuzzers should fuzz a target compiled with laf-intel/COMPCOV (see
614 above). Important note: If you run more than one laf-intel/COMPCOV fuzzer and
615 you want them to share their intermediate results, the main fuzzer (`-M`) must
619 * 10% with the MOpt mutator enabled: `-L 0`
620 * 10% should use the old queue cycling with `-Z`
621 * 50-70% should run with `AFL_DISABLE_TRIM`
622 * 40% should run with `-P explore` and 20% with `-P exploit`
623 * If you use `-a` then set 30% of the instances to not use `-a`; if you did
624 not set `-a` (why??), then set 30% to `-a ascii` and 30% to `-a binary`.
627 the `-p` option, e.g., `-p explore`. See the
628 [FAQ](FAQ.md#what-are-power-schedules) for details.
633 Also, it is recommended to set `export AFL_IMPORT_FIRST=1` to load test cases
635 of the first fuzz by quite a lot of you have many fuzzers and/or many seeds.
641 phase and start fuzzing at once - but only do this if the calibration phase
642 would be too long for your fuzz run time.
645 conforming fuzzers, then just use the same -o directory and give it a unique
646 `-S` name. Examples are:
647 * [Fuzzolic](https://github.com/season-lab/fuzzolic)
648 * [symcc](https://github.com/eurecom-s3/symcc/)
649 * [Eclipser](https://github.com/SoftSec-KAIST/Eclipser/)
651 * [FairFuzz](https://github.com/carolemieux/afl-rb)
656 [https://github.com/Microsvuln/Awesome-AFL](https://github.com/Microsvuln/Awesome-AFL).
658 However, you can also sync AFL++ with honggfuzz, libfuzzer with `-entropic=1`,
659 etc. Just show the main fuzzer (`-M`) with the `-F` option where the queue/work
660 directory of a different fuzzer is, e.g., `-F /src/target/honggfuzz`. Using
661 honggfuzz (with `-n 1` or `-n 2`) and libfuzzer in parallel is highly
666 Maybe you have more than one machine you want to fuzz the same target on. Start
667 the `afl-fuzz` (and perhaps libfuzzer, honggfuzz, ...) orchestra as you like,
668 just ensure that your have one and only one `-M` instance per server, and that
669 its name is unique, hence the recommendation for `-M main-$HOSTNAME`.
682 The syncing process itself is very simple. As the `-M main-$HOSTNAME` instance
683 syncs to all `-S` secondaries as well as to other fuzzers, you have to copy only
686 Let's say all servers have the `-o out` directory in /target/foo/out, and you
693 rsync -rlpogtz --rsh=ssh $FROM:/target/foo/out/main-$FROM $TO:target/foo/out/
698 You can run this manually, per cron job - as you need it. There is a more
702 ### e) The status of the fuzz campaign
704 AFL++ comes with the `afl-whatsup` script to show the status of the fuzzing
707 Just supply the directory that afl-fuzz is given with the `-o` option and you
710 To have only the summary, use the `-s` switch, e.g., `afl-whatsup -s out/`.
716 afl-plot, which generates an index.html file and graphs that show how the
717 fuzzing instance is performing. The syntax is `afl-plot instance_dir web_dir`,
718 e.g., `afl-plot out/default /srv/www/htdocs/plot`.
722 To stop an afl-fuzz run, press Control-C.
724 To restart an afl-fuzz run, just reuse the same command line but replace the `-i
725 directory` with `-i -` or set `AFL_AUTORESUME=1`.
728 fuzzing instance, e.g., when your main fuzzer is using `-o out` and the new
732 AFL_BENCH_JUST_ONE=1 AFL_FAST_CAL=1 afl-fuzz -i newseeds -o out -S newseeds -- ./target
740 A better indicator - if you use default llvm instrumentation with at least
741 version 9 - is to use `afl-showmap` with the collect coverage option `-C` on the
745 $ afl-showmap -C -i out -o /dev/null -- ./target -params @@
755 It is even better to check out the exact lines of code that have been reached -
759 [https://github.com/vanhauser-thc/afl-cov](https://github.com/vanhauser-thc/afl-cov),
772 fuzz a library to convert image formats and your target is the png to tiff API,
775 ### h) How long to fuzz a target?
784 and use them to seed other good fuzzers like libfuzzer with the -entropic switch
789 * Use [persistent mode](../instrumentation/README.persistent_mode.md) (x2-x20
798 `update-grub` and `reboot` (warning: makes the system more insecure) - you can
799 also just run `sudo afl-persistent-config`.
802 * Use your cores! See [3c) Using multiple cores](#c-using-multiple-cores).
803 * Run `sudo afl-system-config` before starting the first afl-fuzz instance after
808 Fuzzing is a wonderful and underutilized technique for discovering non-crashing
812 - Two bignum libraries produce different outputs when given the same
813 fuzzer-generated input.
815 - An image library produces different outputs when asked to decode the same
818 - A serialization/deserialization library fails to produce stable outputs when
819 iteratively serializing and deserializing fuzzer-supplied data.
821 - A compression library produces an output inconsistent with the input file when
834 - AFL++ detects faults by checking for the first spawned process dying due to a
840 - As with any other brute-force tool, the fuzzer offers limited coverage if
850 - There are some unfortunate trade-offs with ASAN and 64-bit binaries. This
851 isn't due to any specific fault of afl-fuzz.
853 - There is no direct support for fuzzing network services, background daemons,
856 libdesock may offer a relatively simple option, too - see:
858 [https://github.com/fkie-cad/libdesock](https://github.com/fkie-cad/libdesock)
860 Some useful tips for modifying network-based services can be also found at:
861 …s://www.fastly.com/blog/how-to-fuzz-server-american-fuzzy-lop](https://www.fastly.com/blog/how-to-…
863 - Occasionally, sentient machines rise against their creators. If this happens
867 Beyond this, see [INSTALL.md](INSTALL.md) for platform-specific tips.
871 The coverage-based grouping of crashes usually produces a small data set that
872 can be quickly triaged manually or with a very simple GDB or Valgrind script.
873 Every crash is also traceable to its parent non-crashing test case in the queue,
878 code analysis work. To assist with this task, afl-fuzz supports a very unique
879 "crash exploration" mode enabled with the `-C` flag.
881 In this mode, the fuzzer takes one or more crashing test cases as the input and
882 uses its feedback-driven fuzzing strategies to very quickly enumerate all code
890 is possible to get past an initial out-of-bounds read - and see what lies
893 Oh, one more thing: for test case minimization, give afl-tmin a try. The tool
897 ./afl-tmin -i test_case -o minimized_result -- /path/to/program [...]
900 The tool works with crashing and non-crashing test cases alike. In the crash
901 mode, it will happily accept instrumented and non-instrumented binaries. In the
902 non-crashing mode, the minimizer relies on standard AFL++ instrumentation to
905 The minimizer accepts the `-m`, `-t`, `-f`, and `@@` syntax in a manner
906 compatible with afl-fuzz.
908 Another tool in AFL++ is the afl-analyze tool. It takes an input file, attempts
910 then color-codes the input based on which sections appear to be critical and
914 `casr-afl` from [CASR](https://github.com/ispras/casr) tools provides
918 casr-afl -i /path/to/afl/out/dir -o /path/to/casr/out/dir
923 Some notes on continuous integration (CI) fuzzing - this fuzzing is different to
929 too long for your overall available fuzz run time.
932 * LTO has a much longer compile time which is diametrical to short fuzzing -
933 hence use afl-clang-fast instead.
935 that compiled target with the `-c` option and as the main fuzz target.
937 * `AFL_FAST_CAL` - enables fast calibration, this halves the time the
939 * `AFL_CMPLOG_ONLY_NEW` - only perform cmplog on new finds, not the initial
941 * Keep the generated corpus, use afl-cmin and reuse it every time!
947 3. Also randomize the afl-fuzz runtime options, e.g.:
951 * 10% use MOpt (`-L 0`)
953 * 20% for old queue processing (`-Z`)
954 * for CMPLOG targets, 70% for `-l 2`, 10% for `-l 3`, 20% for `-l 2AT`
956 4. Do *not* run any `-M` modes, just running `-S` modes is better for CI
957 fuzzing. `-M` enables old queue handling etc. which is good for a fuzzing
961 [oss-fuzz](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/compile_af…
968 even have known you had ;-) ).