1JFuzz 2===== 3 4JFuzz is a tool for generating random programs with the objective 5of fuzz testing the ART infrastructure. Each randomly generated program 6can be run under various modes of execution, such as using the interpreter, 7using the optimizing compiler, using an external reference implementation, 8or using various target architectures. Any difference between the outputs 9(**divergence**) may indicate a bug in one of the execution modes. 10 11JFuzz can be combined with DexFuzz to get multi-layered fuzz testing. 12 13How to run JFuzz 14================ 15 16 jfuzz [-s seed] [-d expr-depth] [-l stmt-length] 17 [-i if-nest] [-n loop-nest] [-v] [-h] 18 19where 20 21 -s : defines a deterministic random seed 22 (randomized using time by default) 23 -d : defines a fuzzing depth for expressions 24 (higher values yield deeper expressions) 25 -l : defines a fuzzing length for statement lists 26 (higher values yield longer statement sequences) 27 -i : defines a fuzzing nest for if/switch statements 28 (higher values yield deeper nested conditionals) 29 -n : defines a fuzzing nest for for/while/do-while loops 30 (higher values yield deeper nested loops) 31 -v : prints version number and exits 32 -h : prints help and exits 33 34The current version of JFuzz sends all output to stdout, and uses 35a fixed testing class named Test. So a typical test run looks as follows. 36 37 jfuzz > Test.java 38 jack -cp ${JACK_CLASSPATH} --output-dex . Test.java 39 art -classpath classes.dex Test 40 41How to start JFuzz testing 42========================== 43 44 run_jfuzz_test.py 45 [--num_tests=NUM_TESTS] 46 [--device=DEVICE] 47 [--mode1=MODE] [--mode2=MODE] 48 [--report_script=SCRIPT] 49 [--jfuzz_arg=ARG] 50 [--true_divergence] 51 52where 53 54 --num_tests : number of tests to run (10000 by default) 55 --device : target device serial number (passed to adb -s) 56 --mode1 : m1 57 --mode2 : m2, with m1 != m2, and values one of 58 ri = reference implementation on host (default for m1) 59 hint = Art interpreter on host 60 hopt = Art optimizing on host (default for m2) 61 tint = Art interpreter on target 62 topt = Art optimizing on target 63 --report_script : path to script called for each divergence 64 --jfuzz_arg : argument for jfuzz 65 --true_divergence : don't bisect timeout divergences 66 67How to start JFuzz nightly testing 68================================== 69 70 run_jfuzz_test_nightly.py 71 [--num_proc NUM_PROC] 72 73where 74 75 --num_proc : number of run_jfuzz_test.py instances to run (8 by default) 76 77Remaining arguments are passed to run\_jfuzz_test.py. 78 79How to start J/DexFuzz testing (multi-layered) 80============================================== 81 82 run_dex_fuzz_test.py 83 [--num_tests=NUM_TESTS] 84 [--num_inputs=NUM_INPUTS] 85 [--device=DEVICE] 86 87where 88 89 --num_tests : number of tests to run (10000 by default) 90 --num_inputs: number of JFuzz programs to generate 91 --device : target device serial number (passed to adb -s) 92 93Background 94========== 95 96Although test suites are extremely useful to validate the correctness of a 97system and to ensure that no regressions occur, any test suite is necessarily 98finite in size and scope. Tests typically focus on validating particular 99features by means of code sequences most programmers would expect. Regression 100tests often use slightly less idiomatic code sequences, since they reflect 101problems that were not anticipated originally, but occurred “in the field”. 102Still, any test suite leaves the developer wondering whether undetected bugs 103and flaws still linger in the system. 104 105Over the years, fuzz testing has gained popularity as a testing technique for 106discovering such lingering bugs, including bugs that can bring down a system 107in an unexpected way. Fuzzing refers to feeding a large amount of random data 108as input to a system in an attempt to find bugs or make it crash. Generation- 109based fuzz testing constructs random, but properly formatted input data. 110Mutation-based fuzz testing applies small random changes to existing inputs 111in order to detect shortcomings in a system. Profile-guided or coverage-guided 112fuzzing adds a direction to the way these random changes are applied. Multi- 113layered approaches generate random inputs that are subsequently mutated at 114various stages of execution. 115 116The randomness of fuzz testing implies that the size and scope of testing is no 117longer bounded. Every new run can potentially discover bugs and crashes that were 118hereto undetected. 119