1# ART Chroot-Based On-Device Testing 2 3This file documents the use of a chroot environment in on-device testing of the 4Android Runtime (ART). Using a chroot allows tests to run a standalone ART from 5a locally built source tree on a device running (almost any) system image and 6does not interfere with the Runtime installed in the device's system partition. 7 8## Introduction 9 10The Android Runtime (ART) supports testing in a chroot-based environment, by 11setting up a chroot directory in a `ART_TEST_CHROOT` directory located under 12`/data/local` (e.g. `ART_TEST_CHROOT=/data/local/art-test-chroot`) on a device, 13installing ART and all other required artifacts there, and having tests use `adb 14shell chroot $ART_TEST_CHROOT <command>` to execute commands on the device 15within this environment. 16 17This way to run tests using a "standalone ART" ("guest system") only affects 18files in the data partition (the system partition and other partitions are left 19untouched) and is as independent as possible from the Android system ("host 20system") running on the device. This has some benefits: 21 22* no need to build and flash a whole device to do ART testing (or "overwriting" 23 an existing ART by syncing the system partition); 24* the possibility to use a smaller AOSP Android manifest 25 ([`master-art`](https://android.googlesource.com/platform/manifest/+/refs/heads/master-art/default.xml)) 26 to build ART and the required dependencies for testing; 27* no instability due to updating/replacing ART on the system partition (a 28 functional Android Runtime is necessary to properly boot a device); 29* the possibility to have several standalone ART instances (one per directory, 30 e.g. `/data/local/art-test-chroot1`, `/data/local/art-test-chroot2`, etc.). 31 32Note that using this chroot-based approach requires root access to the device 33(i.e. be able to run `adb root` successfully). 34 35## Quick User Guide 36 370. Unset variables which are not used with the chroot-based approach (if they 38 were set previously): 39 ```bash 40 unset ART_TEST_ANDROID_ROOT 41 unset CUSTOM_TARGET_LINKER 42 unset ART_TEST_ANDROID_ART_ROOT 43 unset ART_TEST_ANDROID_RUNTIME_ROOT 44 unset ART_TEST_ANDROID_I18N_ROOT 45 unset ART_TEST_ANDROID_TZDATA_ROOT 46 ``` 471. Set the chroot directory in `ART_TEST_CHROOT`: 48 ```bash 49 export ART_TEST_CHROOT=/data/local/art-test-chroot 50 ``` 512. Set lunch target and ADB: 52 * With a minimal `aosp/master-art` tree: 53 1. Initialize the environment: 54 ```bash 55 export SOONG_ALLOW_MISSING_DEPENDENCIES=true 56 export BUILD_BROKEN_DISABLE_BAZEL=true 57 . ./build/envsetup.sh 58 ``` 59 2. Select a lunch target corresponding to the architecture you want to 60 build and test: 61 * For (32-bit) Arm: 62 ```bash 63 lunch arm_krait-eng 64 ``` 65 * For (64-bit only) Arm64: 66 ```bash 67 lunch armv8-eng 68 ``` 69 * For (32- and 64-bit) Arm64: 70 ```bash 71 lunch arm_v7_v8-eng 72 ``` 73 * For (32-bit) Intel x86: 74 ```bash 75 lunch silvermont-eng 76 ``` 77 3. Set up the environment to use a pre-built ADB: 78 ```bash 79 export PATH="$(pwd)/prebuilts/runtime:$PATH" 80 export ADB="$ANDROID_BUILD_TOP/prebuilts/runtime/adb" 81 ``` 82 * With a full Android (AOSP) `aosp/master` tree: 83 1. Initialize the environment: 84 ```bash 85 . ./build/envsetup.sh 86 ``` 87 2. Select a lunch target corresponding to the architecture you want to 88 build and test: 89 * For (32-bit) Arm: 90 ```bash 91 lunch aosp_arm-eng 92 ``` 93 * For (32- and 64-bit) Arm64: 94 ```bash 95 lunch aosp_arm64-eng 96 ``` 97 * For (32-bit) Intel x86: 98 ```bash 99 lunch aosp_x86-eng 100 ``` 101 * For (32- and 64-bit) Intel x86-64: 102 ```bash 103 lunch aosp_x86_64-eng 104 ``` 105 3. Build ADB: 106 ```bash 107 m adb 108 ``` 1093. Build ART and required dependencies: 110 ```bash 111 art/tools/buildbot-build.sh --target 112 ``` 113 After building it is fine to see it finish with an error like: 114 ``` 115 linkerconfig E [...] variableloader.cc:83] Unable to access VNDK APEX at path: <path>: No such file or directory 116 ``` 1174. Clean up the device: 118 ```bash 119 art/tools/buildbot-cleanup-device.sh 120 ``` 1215. Setup the device (including setting up mount points and files in the chroot 122 directory): 123 ```bash 124 art/tools/buildbot-setup-device.sh 125 ``` 1266. Populate the chroot tree on the device (including "activating" APEX packages 127 in the chroot environment): 128 ```bash 129 art/tools/buildbot-sync.sh 130 ``` 1317. Run ART gtests: 132 ```bash 133 art/tools/run-gtests.sh -j4 134 ``` 135 * Specific tests to run can be passed on the command line, specified by 136 their absolute paths beginning with `/apex/`: 137 ```bash 138 art/tools/run-gtests.sh \ 139 /apex/com.android.art/bin/art/arm64/art_cmdline_tests \ 140 /apex/com.android.art/bin/art/arm64/art_dexdump_tests 141 ``` 142 * Gtest options can be passed to each gtest by passing them after `--`; see 143 the following examples. 144 * To print the list of all test cases of a given gtest, use option 145 `--gtest_list_tests`: 146 ```bash 147 art/tools/run-gtests.sh \ 148 /apex/com.android.art/bin/art/arm64/art_cmdline_tests \ 149 -- --gtest_list_tests 150 ``` 151 * To filter the test cases to execute, use option `--gtest_filter`: 152 ```bash 153 art/tools/run-gtests.sh \ 154 /apex/com.android.art/bin/art/arm64/art_cmdline_tests \ 155 -- --gtest_filter="*TestJdwp*" 156 ``` 157 * To see all the options supported by a gtest, use option `--help`: 158 ```bash 159 art/tools/run-gtests.sh \ 160 /apex/com.android.art/bin/art/arm64/art_cmdline_tests \ 161 -- --help 162 ``` 163 * Note: Some test cases of `art_runtime_tests` defined in 164 `art/runtime/gc/space/image_space_test.cc` may fail when using the full AOSP 165 tree (b/119815008). 166 * Workaround: Run `m clean-oat-host` before the build step 167 (`art/tools/buildbot-build.sh --target`) above. 168 * Note: The `-j` option of script `art/tools/run-gtests.sh` is not honored 169 yet (b/129930445). However, gtests themselves support parallel execution, 170 which can be specified via the gtest option `-j`: 171 ```bash 172 art/tools/run-gtests.sh -- -j4 173 ``` 1748. Run ART run-tests: 175 * On a 64-bit target: 176 ```bash 177 art/test/testrunner/testrunner.py --target --64 178 ``` 179 * On a 32-bit target: 180 ```bash 181 art/test/testrunner/testrunner.py --target --32 182 ``` 1839. Run Libcore tests: 184 * On a 64-bit target: 185 ```bash 186 art/tools/run-libcore-tests.sh --mode=device --variant=X64 187 ``` 188 * On a 32-bit target: 189 ```bash 190 art/tools/run-libcore-tests.sh --mode=device --variant=X32 191 ``` 19210. Run JDWP tests: 193 * On a 64-bit target: 194 ```bash 195 art/tools/run-libjdwp-tests.sh --mode=device --variant=X64 196 ``` 197 * On a 32-bit target: 198 ```bash 199 art/tools/run-libjdwp-tests.sh --mode=device --variant=X32 200 ``` 20111. Tear down device setup: 202 ```bash 203 art/tools/buildbot-teardown-device.sh 204 ``` 20512. Clean up the device: 206 ```bash 207 art/tools/buildbot-cleanup-device.sh 208 ``` 209