1.. _test-suite: 2 3============================================================================== 4libinput test suite 5============================================================================== 6 7libinput's primary test suite can be invoked with 8 9:: 10 11 $ sudo ./builddir/libinput-test-suite 12 13When developing libinput, the ``libinput-test-suite`` should always be 14run to check for behavior changes and/or regressions. For quick iteration, 15the number of tests to run can be filtered, see :ref:`test-filtering`. 16This allows for developers to verify a subset of tests (e.g. 17touchpad tap-to-click) while hacking on that specific feature and only run 18the full suite when development is done finished. 19 20.. note:: The test suite relies on udev and the kernel, specifically uinput. 21 It creates virtual input devices and replays the events. This may 22 interfere with your running session. The test suite is not suitable 23 for running inside containers. 24 25In addition, libinput ships with a set of (primarily janitorial) tests that 26must pass for any merge request. These tests are invoked by calling 27``meson test -C builddir`` (or ``ninja test``). The ``libinput-test-suite`` is 28part of that test set by default. 29 30The upstream CI runs all these tests but not the ``libinput-test-suite``. 31This CI is run for every merge request. 32 33.. _test-job-control: 34 35------------------------------------------------------------------------------ 36Job control in the test suite 37------------------------------------------------------------------------------ 38 39The test suite runner has a make-like job control enabled by the ``-j`` or 40``--jobs`` flag and will fork off as many parallel processes as given by this 41flag. The default if unspecified is 8. When debugging a specific test case 42failure it is recommended to employ test filtures (see :ref:`test-filtering`) 43and disable parallel tests. The test suite automatically disables parallel 44make when run in gdb. 45 46.. _test-config: 47 48------------------------------------------------------------------------------ 49X.Org config to avoid interference 50------------------------------------------------------------------------------ 51 52uinput devices created by the test suite are usually recognised by X as 53input devices. All events sent through these devices will generate X events 54and interfere with your desktop. 55 56Copy the file ``$srcdir/test/50-litest.conf`` into your ``/etc/X11/xorg.conf.d`` 57and restart X. This will ignore any litest devices and thus not interfere 58with your desktop. 59 60.. _test-root: 61 62------------------------------------------------------------------------------ 63Permissions required to run tests 64------------------------------------------------------------------------------ 65 66Most tests require the creation of uinput devices and access to the 67resulting ``/dev/input/eventX`` nodes. Some tests require temporary udev rules. 68**This usually requires the tests to be run as root**. If not run as 69root, the test suite runner will exit with status 77, an exit status 70interpreted as "skipped". 71 72.. _test-filtering: 73 74------------------------------------------------------------------------------ 75Selective running of tests 76------------------------------------------------------------------------------ 77 78litest's tests are grouped into test groups, test names and devices. A test 79group is e.g. "touchpad:tap" and incorporates all tapping-related tests for 80touchpads. Each test function is (usually) run with one or more specific 81devices. The ``--list`` commandline argument shows the list of suites and 82tests. This is useful when trying to figure out if a specific test is 83run for a device. 84 85 86:: 87 88 $ ./builddir/libinput-test-suite --list 89 ... 90 pointer:left-handed: 91 pointer_left_handed_during_click_multiple_buttons: 92 trackpoint 93 ms-surface-cover 94 mouse-wheelclickcount 95 mouse-wheelclickangle 96 low-dpi-mouse 97 mouse-roccat 98 mouse-wheel-tilt 99 mouse 100 logitech-trackball 101 cyborg-rat 102 magicmouse 103 pointer_left_handed_during_click: 104 trackpoint 105 ms-surface-cover 106 mouse-wheelclickcount 107 mouse-wheelclickangle 108 low-dpi-mouse 109 mouse-roccat 110 mouse-wheel-tilt 111 mouse 112 logitech-trackball 113 cyborg-rat 114 litest-magicmouse-device 115 pointer_left_handed: 116 trackpoint 117 ms-surface-cover 118 mouse-wheelclickcount 119 mouse-wheelclickangle 120 low-dpi-mouse 121 mouse-roccat 122 mouse-wheel-tilt 123 mouse 124 ... 125 126 127In the above example, the "pointer:left-handed" suite contains multiple 128tests, e.g. "pointer_left_handed_during_click" (this is also the function 129name of the test, making it easy to grep for). This particular test is run 130for various devices including the trackpoint device and the magic mouse 131device. 132 133The "no device" entry signals that litest does not instantiate a uinput 134device for a specific test (though the test itself may 135instantiate one). 136 137The ``--filter-test`` argument enables selective running of tests through 138basic shell-style function name matching. For example: 139 140 141:: 142 143 $ ./builddir/libinput-test-suite --filter-test="*1fg_tap*" 144 145 146The ``--filter-device`` argument enables selective running of tests through 147basic shell-style device name matching. The device names matched are the 148litest-specific shortnames, see the output of ``--list``. For example: 149 150 151:: 152 153 $ ./builddir/libinput-test-suite --filter-device="synaptics*" 154 155 156The ``--filter-group`` argument enables selective running of test groups 157through basic shell-style test group matching. The test groups matched are 158litest-specific test groups, see the output of ``--list``. For example: 159 160 161:: 162 163 $ ./builddir/libinput-test-suite --filter-group="touchpad:*hover*" 164 165 166The ``--filter-device`` and ``--filter-group`` arguments can be combined with 167``--list`` to show which groups and devices will be affected. 168 169.. _test-verbosity: 170 171------------------------------------------------------------------------------ 172Controlling test output 173------------------------------------------------------------------------------ 174 175Each test supports the ``--verbose`` commandline option to enable debugging 176output, see **libinput_log_set_priority()** for details. The ``LITEST_VERBOSE`` 177environment variable, if set, also enables verbose mode. 178 179 180:: 181 182 $ ./builddir/libinput-test-suite --verbose 183 $ LITEST_VERBOSE=1 meson test -C builddir 184 185.. _test-installed: 186 187------------------------------------------------------------------------------ 188Installing the test suite 189------------------------------------------------------------------------------ 190 191If libinput is configured to install the tests, the test suite is available 192as the ``libinput test-suite`` command. When run as installed binary, the 193behavior of the test suite changes: 194 195- the ``libinput.so`` used is the one in the library lookup paths 196- no system-wide quirks are installed by the test suite, only those specific 197 to the test devices 198- test device-specific quirks are installed in the system-wide quirks 199 directory, usually ``/usr/share/libinput/``. 200 201It is not advisable to run ``libinput test-suite`` on a production machine. 202Data loss may occur. The primary use-case for the installed test suite is 203verification of distribution composes. 204 205.. note:: The ``prefix`` is still used by the test suite. For verification 206 of a system package, the test suite must be configured with the same prefix. 207 208To configure libinput to install the tests, use the ``-Dinstall-tests=true`` 209meson option:: 210 211 $ meson builddir -Dtests=true -Dinstall-tests=true <other options> 212 213.. _test-meson-suites: 214 215------------------------------------------------------------------------------ 216Meson test suites 217------------------------------------------------------------------------------ 218 219This section is primarily of interest to distributors that want to run test 220or developers working on libinput's CI. 221 222Tests invoked by ``meson test`` are grouped into test suites, the test suite 223names identify when the respective test can be run: 224 225- ``valgrind``: tests that can be run under valgrind (in addition to a 226 normal run) 227- ``root``: tests that must be run as root 228- ``hardware``: tests that require a VM or physical machine 229- ``all``: all tests, only needed because of 230 `meson bug 5340 <https://github.com/mesonbuild/meson/issues/5340>`_ 231 232The suite names can be provided as filters to ``meson test 233--suite=<suitename>`` or ``meson test --no-suite=<suitename>``. 234For example, if running a container-based CI, you may specify the test 235suites as: 236 237:: 238 239 $ meson test --no-suite=machine # only run container-friendly tests 240 $ meson test --suite=valgrind --setup=valgrind # run all valgrind-compatible tests 241 $ meson test --no-suite=root # run all tests not requiring root 242 243These suites are subject to change at any time. 244