1# Testing 2 3Crosvm runs on a variety of platforms with a significant amount of platform-specific code. Testing 4on all the supported platforms is crucial to keep crosvm healthy. 5 6## Types of tests 7 8### Unit Tests 9 10Unit tests are your standard rust tests embedded with the rest of the code in `src/` and wrapped in 11a `#[cfg(test)]` attribute. 12 13Unit tests **cannot make any guarantees on the runtime environment**. Avoid doing the following in 14unit tests: 15 16- Avoid kernel features such as io_uring or userfaultfd, which may not be available on all kernels. 17- Avoid functionality that requires privileges (e.g. CAP_NET_ADMIN) 18- Avoid spawning threads or processes 19- Avoid accessing kernel devices 20- Avoid global state in unit tests 21 22This allows us to execute unit tests for any platform using emulators such as qemu-static or wine64. 23 24### Integration tests 25 26Cargo has native support for 27[integration testing](https://doc.rust-lang.org/rust-by-example/testing/integration_testing.html). 28Integration tests are written just like unit tests, but live in a separate directory at `tests/`. 29 30Integration tests **guarantee that the test has privileged access to the test environment**. They 31are only executed when a device-under-test (DUT) is specified when running tests: 32 33```sh 34./tools/run_tests --dut=vm|host 35``` 36 37### End To End (E2E) tests 38 39End to end tests live in the `e2e_tests` crate. The crate provides a framework to boot a guest with 40crosvm and execut commands in the guest to validate functionality at a high level. 41 42E2E tests are executed just like integration tests. 43 44### Downstream Product tests 45 46Each downstream product that uses crosvm is performing their own testing, e.g. ChromeOS is running 47high level testing of its VM features on ChromeOS hardware, while AOSP is running testing of their 48VM features on AOSP hardware. 49 50Upstream crosvm is not involved in these tests and they are not executed in crosvm CI. 51 52## Parallel test execution 53 54Crosvm tests are executed in parallel, each test case in it's own process via 55[cargo nextest](http://nexte.st). 56 57This requires tests to be cautious about global state, especially integration tests which interact 58with system devices. 59 60If you require exclusive access to a device or file, you have to use 61[file-based locking](https://docs.rs/namedlock/latest/namedlock/) to prevent access by other test 62processes. 63 64## Platorms tested 65 66The platforms below can all be tested using `tools/run_tests -p $platform`. The table indicates how 67these tests are executed: 68 69| Platform | Build | Unit Tests | Integration Tests | E2E Tests | 70| :-------------------------- | :---: | :---------------------------: | :---------------: | :-------: | 71| x86_64 (linux) | ✅ | ✅ | ✅ | ✅ | 72| aarch64 (linux) | ✅ | ✅ (qemu-static[^qemu-static]) | ✅ (qemu[^qemu]) | ❌ | 73| armhf (linux) | ✅ | ✅ (qemu-static[^qemu-static]) | ❌ | ❌ | 74| mingw64[^windows] (linux) | �� | �� (wine64) | ❌ | ❌ | 75| mingw64[^windows] (windows) | �� | �� | �� | ❌ | 76 77Crosvm CI will use the same configuration as `tools/run_tests`. 78 79[^qemu-static]: qemu-static-aarch64 or qemu-static-arm translate instructions into x86 and executes them on the 80 host kernel. This works well for unit tests, but will fail when interacting with platform 81 specific kernel features. 82 83[^qemu]: run_tests will launch a VM for testing in the background. This VM is using full system 84 emulation, which causes tests to be slow. Also not all aarch64 features are properly emulated, 85 which prevents us from running e2e tests. 86 87[^windows]: Windows builds of crosvm are a work in progress. Some tests are executed via wine64 on linux 88