1#Fuzzing 2In this folder, we keep our _fuzzers_ (bits of code that takes a randomized input and executes code 3randomly, focusing on specific APIs). For example, we have a codec fuzzer which takes a mutated 4png/jpeg or similar file and attempts to turn it into an `SkImage`. We also have a canvas fuzzer 5which takes in a random set of bytes and turns them into calls on `SkCanvas`. 6 7## Executables 8These fuzzers are packaged in two different ways (see //BUILD.gn). There is a `fuzz` executable 9that contains all fuzzers and is a convenient way to reproduce fuzzer-reported bugs. There are also 10single fuzzer executables containing exactly one fuzzer, which are convenient to build with 11[libfuzzer](https://llvm.org/docs/LibFuzzer.html). 12 13See [../site/dev/testing/fuzz.md] for more information on building and running fuzzers using the 14`fuzz` executable. 15 16## Continuous Running 17We fuzz Skia using [OSS-Fuzz](https://github.com/google/oss-fuzz), which in turn uses fuzzing 18engines such as libfuzzer, afl-fuzz, hong-fuzz, and others to fuzz Skia. OSS-fuzz will automatically 19[file and close bugs](https://bugs.chromium.org/p/oss-fuzz/issues/list?q=label:Proj-skia) when 20it finds issues. 21 22There is a [Skia folder](https://github.com/google/oss-fuzz/tree/master/projects/skia) 23in the OSS-Fuzz repo that we make changes to when we want to add/remove/change the fuzzers that 24are automatically run. 25[This](https://google.github.io/oss-fuzz/getting-started/new-project-guide/#testing-locally) 26describes how to test the OSS-Fuzz build and fuzzers locally using Docker. 27 28When enabling a fuzzer in OSS-Fuzz, we typically need to follow these steps: 29 1. *Add a seed corpus to `gs://skia-fuzzer/oss-fuzz/` (in the 30 [skia-public project](https://console.cloud.google.com/storage/browser/skia-fuzzer?project=skia-public)). 31 Make sure the corpus file is public-readable. It is easiest to add this permission via the web 32 UI. This is done by granting the allUsers "name" the Reader role to the zip file. See the infra 33 team if you do not have access to this bucket. 34 2. *Update [the Dockerfile](https://github.com/google/oss-fuzz/blob/master/projects/skia/Dockerfile) 35 to download the seed corpus to the build image. 36 3. Update [build.sh](https://github.com/google/oss-fuzz/blob/628264df27f53cc60fcb27406a2da05d2197c025/projects/skia/build.sh#L99) 37 to build the desired fuzzer target and move it into $OUT. If there is a seed corpus, move 38 it into $OUT and make sure it is the same name as the fuzzer executable with `_seed_corpus.zip` 39 as a suffix. 40 41*For fuzzers who depend strongly on the format of the randomized data, e.g. image decoding, SkSL 42parsing. These are called _binary fuzzers_, as opposed to _API fuzzers_. 43 44Example PRs for adding fuzzers: [binary](https://github.com/google/oss-fuzz/pull/4108), 45[API](https://github.com/google/oss-fuzz/pull/5657) 46 47There is also an [OSS-fuzz folder](https://github.com/google/oss-fuzz/tree/master/projects/skcms) 48set up for the [skcms repo](https://skia.googlesource.com/skcms/). The build process is similar, 49except instead of compiling using GN targets, the build.sh script compiles the fuzz executables 50directly. 51 52### OSS-Fuzz dashboard 53<https://oss-fuzz.com/fuzzer-stats> is useful to see metrics on how our fuzzers are running. It 54shows things like executions per second (higher is better), edge coverage percent per fuzzer, 55what percent of fuzzing runs end in OOM/timeout/crash, the entire corpus of fuzzed inputs 56(corpus_backup), etc. Contact aarya@ to get permission to view this dashboard if necessary. 57Here are some example dashboards: 58 59 - [Per Fuzzer summary for all Skia fuzzers driven by libFuzzer](https://oss-fuzz.com/fuzzer-stats?group_by=by-fuzzer&date_start=2021-08-16&date_end=2021-08-22&fuzzer=libFuzzer&job=libfuzzer_asan_skia&project=skia) 60 - [Five day summary of sksl2glsl driven by afl-fuzz](https://oss-fuzz.com/fuzzer-stats?group_by=by-day&date_start=2021-08-16&date_end=2021-08-22&fuzzer=afl_skia_sksl2glsl&job=afl_asan_skia&project=skia) 61 62That dashboard also has a Coverage Report. Even though it appears the Coverage report is per fuzzer, 63the reports always show the aggregated coverage from all fuzzers. 64[Example coverage report from 2021 Aug 22](https://storage.googleapis.com/oss-fuzz-coverage/skia/reports/20210822/linux/report.html) 65 66## See Also 67 - [Creating a binary fuzzer](https://docs.google.com/document/d/1QDX0o8yDdmhbjoudNsXc66iuRXRF5XNNqGnzDzX7c2I/edit) 68 - [Creating an API fuzzer](https://docs.google.com/document/d/1e3ikXO7SwoBsbsi1MF06vydXRlXvYalVORaiUuOXk2Y/edit)