• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1DexFuzz
2=======
3
4DexFuzz is primarily a tool for fuzzing DEX files. Fuzzing is the introduction of
5subtle changes ("mutations") to a file to produce a new test case. These test cases
6can be used to test the various modes of execution available to ART (Interpreter,
7Optimizing compiler) to check for bugs in these modes of execution.
8This is done by differential testing - each test file is executed with each mode of
9execution, and any differences between the resulting outputs may be an indication of
10a bug in one of the modes.
11
12For a wider overview of DexFuzz, see:
13
14http://community.arm.com/groups/android-community/blog/2014/11/26/the-art-of-fuzz-testing
15
16In typical operation, you provide DexFuzz with a set of DEX files that are the "seeds"
17for mutation - e.g. some tests taken from the ART test suite - and point it at an
18ADB-connected Android device, and it will fuzz these seed files, and execute the
19resulting new tests on the Android device.
20
21How to run DexFuzz
22==================
23
24DexFuzz can run its test programs on either an ADB-connected device, or a host-build of
25ART locally.
26
27Execution on an ADB-connected device
28------------------------------------
29
301. Build dexfuzz with mmma tools/dexfuzz from within art/.
312. Make sure you have an Android device connected via ADB, that is capable of
32   having DEX files pushed to it and executed with the dalvikvm command.
333. Make sure you're in the Android build environment!
34   (That is, . build/envsetup.sh && lunch)
354. Create a new directory, and place some DEX files in here. These are the seed files
36   that are mutated to form new tests.
375. Create a directory on your device that mutated test files can be pushed to and
38   executed in, using dalvikvm. For example, /data/art-test/
396. If you currently have multiple devices connected via ADB, find out the name of
40   your device using "adb devices -l".
417. Run this command:
42
43dexfuzz --inputs=<seeds dir> --execute --repeat=<attempts> \
44    --dump-output <combination of ISA(s) and and backend(s)>
45
46You MUST specify one of the following ISAs:
47  --arm
48  --arm64
49  --x86
50  --x86_64
51
52And also at least two of the following backends:
53  --interpreter
54  --optimizing
55
56Note that if you wanted to test both ARM and ARM64 on an ARM64 device, you can use
57--allarm. Also in this case only one backend is needed, if i.e., you wanted to test
58ARM Optimizing Backend vs. ARM64 Optimizing Backend.
59
60Some legal examples:
61  --arm --optimizing --interpreter
62  --x86 --optimizing --interpreter
63  --allarm --optimizing
64
65Add in --device=<device name, e.g. device:generic> if you want to specify a device.
66Add in --execute-dir=<dir on device> if you want to specify an execution directory.
67  (The default is /data/art-test/)
68
69Host Execution
70--------------
71
72DexFuzz now supports execution on your host machine.
73Follow steps 1, 3, 4, and 7 as above, but also observe the following:
74 - instead of specifying an ISA, use --host
75 - ANDROID_DATA must be set, pointing to a location where dex2oat will place
76   OAT files after compilation.
77 - Files will always be executed in the same directory where you are executing DexFuzz.
78
79Fuzzer Operation
80----------------
81
82As the fuzzer works, you'll see output like:
83
84|-----------------------------------------------------------------|
85|Iterations|VerifyFail|MutateFail|Timed Out |Successful|Divergence|
86|-----------------------------------------------------------------|
87| 48       | 37       | 4        | 0        | 6        | 1        |
88
89Iterations - number of attempts we've made to mutate DEX files.
90VerifyFail - the number of mutated files that ended up failing to verify, either
91             on the host, or the target.
92MutateFail - because mutation is a random process, and has attempt thresholds to
93             avoid attempting to mutate a file indefinitely, it is possible that
94             an attempt to mutate a file doesn't actually mutate it. This counts
95             those occurrences.
96Timed Out  - mutated files that timed out for one or more backends.
97             Current timeouts are:
98               Optimizing - 5 seconds
99               Interpreter - 30 seconds
100              (use --short-timeouts to set all backends to 2 seconds.)
101Successful - mutated files that executed and all backends agreed on the resulting
102             output. NB: if all backends crashed with the same output, this would
103             be considered a success - proper detection of crashes is still to come.
104Divergence - mutated files that executed and some backend disagreed about the
105             resulting output. Divergent programs are run multiple times with a
106             single backend, to check if they diverge from themselves, and these are
107             not included in the count. If multiple architectures are being used
108             (ARM/ARM64), and the divergences align with different architectures,
109             these are also not included in the count.
110
1118. Check report.log for the full report, including input file and RNG seed for each
112   test program. This allows you to recreate a bad program with, e.g.:
113
114dexfuzz --input=<input file> --seed=<seed value>
115
116Check dexfuzz --help for the full list of options.
117
118NOTE: DEX files with unicode strings are not fully supported yet, and DEX files with
119JNI elements are not supported at all currently.
120
121Mutation Likelihoods
122====================
123
124Each bytecode mutation has a chance out of 100% of firing. Following is the listing
125of each mutation's probability. If you wish to easily adjust these values, copy
126these values into a file called likelihoods.txt, and run dexfuzz with
127--likelihoods=likelihoods.txt.
128
129ArithOpChanger 75
130BranchShifter 30
131CmpBiasChanger 30
132ConstantValueChanger 70
133ConversionRepeater 50
134FieldFlagChanger 40
135InstructionDeleter 40
136InstructionDuplicator 80
137InstructionSwapper 80
138InvokeChanger 30
139NewArrayLengthChanger 50
140NewInstanceChanger 10
141NewMethodCaller 10
142NonsenseStringPrinter 10
143OppositeBranchChanger 40
144PoolIndexChanger 30
145RandomBranchChanger 30
146RandomInstructionGenerator 30
147RegisterClobber 10
148SwitchBranchShifter 30
149TryBlockShifter 40
150ValuePrinter 40
151VRegChanger 60
152