1JFuzz 2===== 3 4JFuzz is a tool for generating random programs with the objective 5of fuzz testing the ART infrastructure. Each randomly generated program 6can be run under various modes of execution, such as using the interpreter, 7using the optimizing compiler, using an external reference implementation, 8or using various target architectures. Any difference between the outputs 9(**divergence**) may indicate a bug in one of the execution modes. 10 11JFuzz can be combined with DexFuzz to get multi-layered fuzz testing. 12 13How to run JFuzz 14================ 15 16 jfuzz [-s seed] [-d expr-depth] [-l stmt-length] 17 [-i if-nest] [-n loop-nest] [-v] [-h] 18 19where 20 21 -s : defines a deterministic random seed 22 (randomized using time by default) 23 -d : defines a fuzzing depth for expressions 24 (higher values yield deeper expressions) 25 -l : defines a fuzzing length for statement lists 26 (higher values yield longer statement sequences) 27 -i : defines a fuzzing nest for if/switch statements 28 (higher values yield deeper nested conditionals) 29 -n : defines a fuzzing nest for for/while/do-while loops 30 (higher values yield deeper nested loops) 31 -t : defines a fuzzing nest for try-catch-finally blocks 32 (higher values yield deeper nested try-catch-finally blocks) 33 -v : prints version number and exits 34 -h : prints help and exits 35 36The current version of JFuzz sends all output to stdout, and uses 37a fixed testing class named Test. So a typical test run looks as follows. 38 39 jfuzz > Test.java 40 mkdir classes 41 javac -d classes Test.java 42 dx --dex --output=classes.dex classes 43 art -cp classes.dex Test 44 45How to start JFuzz testing 46========================== 47 48 run_jfuzz_test.py 49 [--num_tests=NUM_TESTS] 50 [--device=DEVICE] 51 [--mode1=MODE] [--mode2=MODE] 52 [--report_script=SCRIPT] 53 [--jfuzz_arg=ARG] 54 [--true_divergence] 55 [--dexer=DEXER] 56 [--debug_info] 57 58where 59 60 --num_tests : number of tests to run (10000 by default) 61 --device : target device serial number (passed to adb -s) 62 --mode1 : m1 63 --mode2 : m2, with m1 != m2, and values one of 64 ri = reference implementation on host (default for m1) 65 hint = Art interpreter on host 66 hopt = Art optimizing on host (default for m2) 67 tint = Art interpreter on target 68 topt = Art optimizing on target 69 --report_script : path to script called for each divergence 70 --jfuzz_arg : argument for jfuzz 71 --true_divergence : don't bisect timeout divergences 72 --dexer=DEXER : use either dx or d8 to obtain dex files 73 --debug_info : include debugging info 74 75How to start JFuzz nightly testing 76================================== 77 78 run_jfuzz_test_nightly.py 79 [--num_proc NUM_PROC] 80 81where 82 83 --num_proc : number of run_jfuzz_test.py instances to run (8 by default) 84 85Remaining arguments are passed to run\_jfuzz_test.py. 86 87How to start J/DexFuzz testing (multi-layered) 88============================================== 89 90 run_dex_fuzz_test.py 91 [--num_tests=NUM_TESTS] 92 [--num_inputs=NUM_INPUTS] 93 [--device=DEVICE] 94 [--dexer=DEXER] 95 [--debug_info] 96 97where 98 99 --num_tests : number of tests to run (10000 by default) 100 --num_inputs : number of JFuzz programs to generate 101 --device : target device serial number (passed to adb -s) 102 --dexer=DEXER : use either dx or d8 to obtain dex files 103 --debug_info : include debugging info 104 105Background 106========== 107 108Although test suites are extremely useful to validate the correctness of a 109system and to ensure that no regressions occur, any test suite is necessarily 110finite in size and scope. Tests typically focus on validating particular 111features by means of code sequences most programmers would expect. Regression 112tests often use slightly less idiomatic code sequences, since they reflect 113problems that were not anticipated originally, but occurred “in the field”. 114Still, any test suite leaves the developer wondering whether undetected bugs 115and flaws still linger in the system. 116 117Over the years, fuzz testing has gained popularity as a testing technique for 118discovering such lingering bugs, including bugs that can bring down a system 119in an unexpected way. Fuzzing refers to feeding a large amount of random data 120as input to a system in an attempt to find bugs or make it crash. Generation- 121based fuzz testing constructs random, but properly formatted input data. 122Mutation-based fuzz testing applies small random changes to existing inputs 123in order to detect shortcomings in a system. Profile-guided or coverage-guided 124fuzzing adds a direction to the way these random changes are applied. Multi- 125layered approaches generate random inputs that are subsequently mutated at 126various stages of execution. 127 128The randomness of fuzz testing implies that the size and scope of testing is no 129longer bounded. Every new run can potentially discover bugs and crashes that were 130hereto undetected. 131