1# ANGLE Test Harness 2 3The ANGLE test harness is a harness around GoogleTest that provides functionality similar to the 4[Chromium test harness][BaseTest]. It features: 5 6 * splitting a test set into shards 7 * catching and reporting crashes and timeouts 8 * outputting to the Chromium [JSON test results format][JSONFormat] 9 * multi-process execution 10 11## Command-Line Arguments 12 13The ANGLE test harness accepts all standard GoogleTest arguments. The harness also accepts the 14following additional command-line arguments: 15 16 * `--shard-count` and `--shard-index` control the test sharding 17 * `--bot-mode` enables multi-process execution and test batching 18 * `--batch-size` limits the number of tests to run in each batch 19 * `--batch-timeout` limits the amount of time spent in each batch 20 * `--max-processes` limits the number of simuntaneous processes 21 * `--test-timeout` limits the amount of time spent in each test 22 * `--results-file` specifies a location for the JSON test result output 23 * `--results-directory` specifies a directory to write test results to 24 * `--filter-file` allows passing a larget `gtest_filter` via a file 25 26As well as the custom command-line arguments we support a few standard GoogleTest arguments: 27 28 * `gtest_filter` works as it normally does with GoogleTest 29 * `gtest_also_run_disabled_tests` works as it normally does as well 30 31Other GoogleTest arguments are not supported although they may work. 32 33## Implementation Notes 34 35 * The test harness only requires `angle_common` and `angle_util`. 36 * It does not depend on any Chromium browser code. This allows us to compile on other non-Clang platforms. 37 * It uses rapidjson to read and write JSON files. 38 * Timeouts are detected via a watchdog thread. 39 * Crashes are handled via ANGLE's test crash handling code. 40 * Currently it does not entirely support Android or Fuchsia. 41 * Test execution is not currently deterministic in multi-process mode. 42 * We capture stdout to output test failure reasons. 43 44See the source code for more details: [TestSuite.h](TestSuite.h) and [TestSuite.cpp](TestSuite.cpp). 45 46## Potential Areas of Improvement 47 48 * Deterministic test execution. 49 * Using sockets to communicate with test children. Similar to dEQP's test harness. 50 * Closer integration with ANGLE's test expectations and system config libraries. 51 * Supporting a GoogleTest-free integration. 52 53[BaseTest]: https://chromium.googlesource.com/chromium/src/+/refs/heads/master/base/test/ 54[JSONFormat]: https://chromium.googlesource.com/chromium/src/+/master/docs/testing/json_test_results_format.md 55