1skqp 2==== 3 4`skqp <https://skia.org/docs/dev/testing/skqp/>`_ stands for SKIA Quality 5Program conformance tests. Basically, it has sets of rendering tests and unit 6tests to ensure that `SKIA <https://skia.org/>`_ is meeting its design specifications on a specific 7device. 8 9The rendering tests have support for GL, GLES and Vulkan backends and test some 10rendering scenarios. 11And the unit tests check the GPU behavior without rendering images. 12 13Tests 14----- 15 16Render tests design 17^^^^^^^^^^^^^^^^^^^ 18 19It is worth noting that `rendertests.txt` can bring some detail about each test 20expectation, so each test can have a max pixel error count, to tell skqp that it 21is OK to have at most that number of errors for that test. See also: 22https://github.com/google/skia/blob/c29454d1c9ebed4758a54a69798869fa2e7a36e0/tools/skqp/README_ALGORITHM.md 23 24.. _test-location: 25 26Location 27^^^^^^^^ 28 29Each `rendertests.txt` and `unittest.txt` file must be located inside a specific 30subdirectory inside skqp assets directory. 31 32+--------------+--------------------------------------------+ 33| Test type | Location | 34+==============+============================================+ 35| Render tests | `${SKQP_ASSETS_DIR}/skqp/rendertests.txt` | 36+--------------+--------------------------------------------+ 37| Unit tests | `${SKQP_ASSETS_DIR}/skqp/unittests.txt` | 38+--------------+--------------------------------------------+ 39 40The `skqp-runner.sh` script will make the necessary modifications to separate 41`rendertests.txt` for each backend-driver combination. As long as the test files are located in the expected place: 42 43+--------------+----------------------------------------------------------------------------------------------+ 44| Test type | Location | 45+==============+==============================================================================================+ 46| Render tests | `${MESA_REPOSITORY_DIR}/src/${GPU_DRIVER}/ci/${GPU_VERSION}-${SKQP_BACKEND}_rendertests.txt` | 47+--------------+----------------------------------------------------------------------------------------------+ 48| Unit tests | `${MESA_REPOSITORY_DIR}/src/${GPU_DRIVER}/ci/${GPU_VERSION}_unittests.txt` | 49+--------------+----------------------------------------------------------------------------------------------+ 50 51Where `SKQP_BACKEND` can be: 52 53- gl: for GL backend 54- gles: for GLES backend 55- vk: for Vulkan backend 56 57Example file 58"""""""""""" 59 60.. code-block:: console 61 62 src/freedreno/ci/freedreno-a630-skqp-gl_rendertests.txt 63 64- GPU_DRIVER: `freedreno` 65- GPU_VERSION: `freedreno-a630` 66- SKQP_BACKEND: `gl` 67 68.. _rendertests-design: 69 70skqp reports 71------------ 72 73skqp generates reports after finishing its execution, they are located at the job 74artifacts results directory and are divided in subdirectories by rendering tests 75backends and unit 76tests. The job log has links to every generated report in order to facilitate 77the skqp debugging. 78 79Maintaining skqp on Mesa CI 80--------------------------- 81 82skqp is built alongside with another binary, namely `list_gpu_unit_tests`, it is 83located in the same folder where `skqp` binary is. 84 85This binary will generate the expected `unittests.txt` for the target GPU, so 86ideally it should be executed on every skqp update and when a new device 87receives skqp CI jobs. 88 891. Generate target unit tests for the current GPU with :code:`./list_gpu_unit_tests > unittests.txt` 90 912. Run skqp job 92 933. If there is a failing or crashing unit test, remove it from the corresponding `unittests.txt` 94 954. If there is a crashing render test, remove it from the corresponding `rendertests.txt` 96 975. If there is a failing render test, visually inspect the result from the HTML report 98 - If the render result is OK, update the max error count for that test 99 - Otherwise, or put `-1` in the same threshold, as seen in :ref:`rendertests-design` 100 1016. Remember to put the new tests files to the locations cited in :ref:`test-location` 102