1# ART Chroot-Based On-Device Testing 2 3This file documents the use of a chroot environment in on-device testing of the 4Android Runtime (ART). Using a chroot allows tests to run a standalone ART from 5a locally built source tree on a device running (almost any) system image and 6does not interfere with the Runtime installed in the device's system partition. 7 8## Introduction 9 10The Android Runtime (ART) supports testing in a chroot-based environment, by 11setting up a chroot directory in a `ART_TEST_CHROOT` directory located under 12`/data/local` (e.g. `ART_TEST_CHROOT=/data/local/art-test-chroot`) on a device, 13installing ART and all other required artifacts there, and having tests use `adb 14shell chroot $ART_TEST_CHROOT <command>` to execute commands on the device 15within this environment. 16 17This way to run tests using a "standalone ART" ("guest system") only affects 18files in the data partition (the system partition and other partitions are left 19untouched) and is as independent as possible from the Android system ("host 20system") running on the device. This has some benefits: 21 22* no need to build and flash a whole device to do ART testing (or "overwriting" 23 an existing ART by syncing the system partition); 24* the possibility to use a smaller AOSP Android manifest 25 ([`master-art`](https://android.googlesource.com/platform/manifest/+/refs/heads/master-art/default.xml)) 26 to build ART and the required dependencies for testing; 27* no instability due to updating/replacing ART on the system partition (a 28 functional Android Runtime is necessary to properly boot a device); 29* the possibility to have several standalone ART instances (one per directory, 30 e.g. `/data/local/art-test-chroot1`, `/data/local/art-test-chroot2`, etc.). 31 32Note that using this chroot-based approach requires root access to the device 33(i.e. be able to run `adb root` successfully). 34 35## Quick User Guide 36 370. Unset variables which are not used with the chroot-based approach (if they 38 were set previously): 39 ```bash 40 unset ART_TEST_ANDROID_ROOT 41 unset CUSTOM_TARGET_LINKER 42 unset ART_TEST_ANDROID_ART_ROOT 43 unset ART_TEST_ANDROID_RUNTIME_ROOT 44 unset ART_TEST_ANDROID_I18N_ROOT 45 unset ART_TEST_ANDROID_TZDATA_ROOT 46 ``` 471. Set the chroot directory in `ART_TEST_CHROOT`: 48 ```bash 49 export ART_TEST_CHROOT=/data/local/art-test-chroot 50 ``` 512. Set lunch target and ADB: 52 * With a minimal `aosp/master-art` tree: 53 ```bash 54 export SOONG_ALLOW_MISSING_DEPENDENCIES=true 55 . ./build/envsetup.sh 56 lunch armv8-eng # or arm_krait-eng for 32-bit ARM 57 export PATH="$(pwd)/prebuilts/runtime:$PATH" 58 export ADB="$ANDROID_BUILD_TOP/prebuilts/runtime/adb" 59 ``` 60 * With a full Android (AOSP) `aosp/master` tree: 61 ```bash 62 . ./build/envsetup.sh 63 lunch aosp_arm64-eng # or aosp_arm-eng for 32-bit ARM 64 m adb 65 ``` 663. Build ART and required dependencies: 67 ```bash 68 art/tools/buildbot-build.sh --target 69 ``` 70 After building it is fine to see it finish with an error like: 71 ``` 72 linkerconfig E [...] variableloader.cc:83] Unable to access VNDK APEX at path: <path>: No such file or directory 73 ``` 744. Clean up the device: 75 ```bash 76 art/tools/buildbot-cleanup-device.sh 77 ``` 785. Setup the device (including setting up mount points and files in the chroot 79 directory): 80 ```bash 81 art/tools/buildbot-setup-device.sh 82 ``` 836. Populate the chroot tree on the device (including "activating" APEX packages 84 in the chroot environment): 85 ```bash 86 art/tools/buildbot-sync.sh 87 ``` 887. Run ART gtests: 89 ```bash 90 art/tools/run-gtests.sh -j4 91 ``` 92 * Specific tests to run can be passed on the command line, specified by 93 their absolute paths beginning with `/apex/`: 94 ```bash 95 art/tools/run-gtests.sh \ 96 /apex/com.android.art/bin/art/arm64/art_cmdline_tests \ 97 /apex/com.android.art/bin/art/arm64/art_dexdump_tests 98 ``` 99 * Gtest options can be passed to each gtest by passing them after `--`; see 100 the following examples. 101 * To print the list of all test cases of a given gtest, use option 102 `--gtest_list_tests`: 103 ```bash 104 art/tools/run-gtests.sh \ 105 /apex/com.android.art/bin/art/arm64/art_cmdline_tests \ 106 -- --gtest_list_tests 107 ``` 108 * To filter the test cases to execute, use option `--gtest_filter`: 109 ```bash 110 art/tools/run-gtests.sh \ 111 /apex/com.android.art/bin/art/arm64/art_cmdline_tests \ 112 -- --gtest_filter="*TestJdwp*" 113 ``` 114 * To see all the options supported by a gtest, use option `--help`: 115 ```bash 116 art/tools/run-gtests.sh \ 117 /apex/com.android.art/bin/art/arm64/art_cmdline_tests \ 118 -- --help 119 ``` 120 * Note: Some test cases of `art_runtime_tests` defined in 121 `art/runtime/gc/space/image_space_test.cc` may fail when using the full AOSP 122 tree (b/119815008). 123 * Workaround: Run `m clean-oat-host` before the build step 124 (`art/tools/buildbot-build.sh --target`) above. 125 * Note: The `-j` option of script `art/tools/run-gtests.sh` is not honored 126 yet (b/129930445). However, gtests themselves support parallel execution, 127 which can be specified via the gtest option `-j`: 128 ```bash 129 art/tools/run-gtests.sh -- -j4 130 ``` 1318. Run ART run-tests: 132 * On a 64-bit target: 133 ```bash 134 art/test/testrunner/testrunner.py --target --64 135 ``` 136 * On a 32-bit target: 137 ```bash 138 art/test/testrunner/testrunner.py --target --32 139 ``` 1409. Run Libcore tests: 141 * On a 64-bit target: 142 ```bash 143 art/tools/run-libcore-tests.sh --mode=device --variant=X64 144 ``` 145 * On a 32-bit target: 146 ```bash 147 art/tools/run-libcore-tests.sh --mode=device --variant=X32 148 ``` 14910. Run JDWP tests: 150 * On a 64-bit target: 151 ```bash 152 art/tools/run-libjdwp-tests.sh --mode=device --variant=X64 153 ``` 154 * On a 32-bit target: 155 ```bash 156 art/tools/run-libjdwp-tests.sh --mode=device --variant=X32 157 ``` 15811. Tear down device setup: 159 ```bash 160 art/tools/buildbot-teardown-device.sh 161 ``` 16212. Clean up the device: 163 ```bash 164 art/tools/buildbot-cleanup-device.sh 165 ``` 166