1LAVA CI 2======= 3 4`LAVA <https://lavasoftware.org/>`_ is a system for functional testing 5of boards including deploying custom bootloaders and kernels. This is 6particularly relevant to testing Mesa because we often need to change 7kernels for UAPI changes (and this lets us do full testing of a new 8kernel during development), and our workloads can easily take down 9boards when mistakes are made (kernel oopses, OOMs that take out 10critical system services). 11 12Mesa-LAVA software architecture 13------------------------------- 14 15The gitlab-runner will run on some host that has access to the LAVA 16lab, with tags like "lava-mesa-boardname" to control only taking in 17jobs for the hardware that the LAVA lab contains. The gitlab-runner 18spawns a Docker container with lava-cli in it, and connects to the 19LAVA lab using a predefined token to submit jobs under a specific 20device type. 21 22The LAVA instance manages scheduling those jobs to the boards present. 23For a job, it will deploy the kernel, device tree, and the ramdisk 24containing the CTS. 25 26Deploying a new Mesa-LAVA lab 27----------------------------- 28 29You'll want to start with setting up your LAVA instance and getting 30some boards booting using test jobs. Start with the stock QEMU 31examples to make sure your instance works at all. Then, you'll need 32to define your actual boards. 33 34The device type in lava-gitlab-ci.yml is the device type you create in 35your LAVA instance, which doesn't have to match the board's name in 36``/etc/lava-dispatcher/device-types``. You create your boards under 37that device type and the Mesa jobs will be scheduled to any of them. 38Instantiate your boards by creating them in the UI or at the command 39line attached to that device type, then populate their dictionary 40(using an "extends" line probably referencing the board's template in 41``/etc/lava-dispatcher/device-types``). Now, go find a relevant 42healthcheck job for your board as a test job definition, or cobble 43something together from a board that boots using the same boot_method 44and some public images, and figure out how to get your boards booting. 45 46Once you can boot your board using a custom job definition, it's time 47to connect Mesa CI to it. Install gitlab-runner and register as a 48shared runner (you'll need a GitLab admin for help with this). The 49runner *must* have a tag (like "mesa-lava-db410c") to restrict the 50jobs it takes or it will grab random jobs from tasks across fd.o, and 51your runner isn't ready for that. 52 53The runner will be running an ARM Docker image (we haven't done any 54x86 LAVA yet, so that isn't documented). If your host for the 55gitlab-runner is x86, then you'll need to install qemu-user-static and 56the binfmt support. 57 58The Docker image will need access to the lava instance. If it's on a 59public network it should be fine. If you're running the LAVA instance 60on localhost, you'll need to set ``network_mode="host"`` in 61``/etc/gitlab-runner/config.toml`` so it can access localhost. Create a 62gitlab-runner user in your LAVA instance, log in under that user on 63the web interface, and create an API token. Copy that into a 64``lavacli.yaml``: 65 66.. code-block:: yaml 67 68 default: 69 token: <token contents> 70 uri: <URL to the instance> 71 username: gitlab-runner 72 73Add a volume mount of that ``lavacli.yaml`` to 74``/etc/gitlab-runner/config.toml`` so that the Docker container can 75access it. You probably have a ``volumes = ["/cache"]`` already, so now it would be:: 76 77 volumes = ["/home/anholt/lava-config/lavacli.yaml:/root/.config/lavacli.yaml", "/cache"] 78 79Note that this token is visible to anybody that can submit MRs to 80Mesa! It is not an actual secret. We could just bake it into the 81GitLab CI yml, but this way the current method of connecting to the 82LAVA instance is separated from the Mesa branches (particularly 83relevant as we have many stable branches all using CI). 84 85Now it's time to define your test runner in 86``.gitlab-ci/lava-gitlab-ci.yml``. 87