1====================== 2How to add a unit test 3====================== 4 5Unit tests help to maintain higher bar for code quality. A new unit test which adds coverage to the code is always useful, 6no matter how small or large it is! Unit test is a valuable contribution, moreover it makes a good starter project, since 7you don't need specific hardware (apart from you development host machine). 8 9For more details on how to run unit tests and measure coverage, check the dev guide: :ref:`unit tests`. 10 11To see the examples of existing unit tests, check the ``/tests`` directory in the source tree. If it helps, you can also look 12at git history for examples of previous commits that add new tests. 13 14When is a good time to add a unit test? Any time is a good time. Test can go in its own separate patch, and also it can go 15together in a patch which introduces a new code. 16 17Unit tests are using `cmocka <https://cmocka.org/>`_ as a mocking framework, but we also have flashrom mock framework 18on the top of that. 19 20Mocking 21========= 22 23Unit tests mock all interactions with hardware, interactions with filesystem, syscalls, 3rd party libraries calls 24(e.g. libusb, libpci) etc. You can think of a flashrom unit test as a mini-emulator. The goal is to cover as much as possible 25flashrom code, but you don't need to go outside of that. 26 27See the list of all current mocks (which are called wraps in cmocka terminology) in ``/tests/wraps.h``. These might be enough for 28your new test, or you might need to add more wraps as a part of new test. 29 30New wrap needs to be added to ``/tests/wraps.h``, ``/tests/tests.c``, ``/tests/meson.build``. If it's fine for new wrap to 31do nothing, log invokation and return success, all good. 32 33If a wrap need to behave in a specific way for a test, and the behaviour can be different from one test to another, you need to 34extend the wrap into ``/tests/io_mock.h`` framework. 35 36Add corresponding member (a function pointer) to ``struct io_mock`` 37and redirect calls from a wrap function in ``/tests/tests.c`` into a member of ``io_mock``. The exact implementation 38of the member function needs to be defined in your new test. At the beginning of a test scenario, define function pointers that your 39test needs in your own ``struct io_mock`` and then register by calling ``io_mock_register``. At the end of a test, clean up 40by calling ``io_mock_register(NULL)``. 41 42Note that ``io_mock`` can support state (if needed). State is a piece of custom data which is carried around for the duration 43of the test scenario and is available in all functions in ``io_mock``. 44 45Adding a new test to a framework 46================================ 47 48To add new test you will either add a new test function in existing .c file, or add new .c file and new function(s) there. 49 50If you add new file, you need to add it into the list of test source files in ``/tests/meson.build``. 51 52Each new test function needs to be added into ``/tests/tests.h`` and ``/tests/tests.c``. Follow existing entries as examples 53how to do it. 54 55Types of tests 56============== 57 58Programmers tests 59----------------- 60 61The concept of a unit test for flashrom programmer is based on a programmer lifecycle. The options supported by the flashrom 62test framework are the following (but you are very welcome to try implement new ideas). 63 64The smallest possible lifecycle is called basic and it does initialisation -> shutdown of a programmer (nothing else). 65Another option is probing lifecycle, which does initialisation -> probing a chip -> shutdown. 66These two expect successful scenarious, the whole scenario is expected to run until the end with no errors and 67success return codes. 68 69One more option is to test expected failure for programmer initialisation. This is useful to test known invalid 70(and potentially dangerous) combination of programmer parameters, when such combination should be detected at init time. 71If invalid combination of parameters is detected, initialisation expected to fail early and programmer must not continue, 72and not send any opcodes to the chip. 73 74For more details, source code is in ``/tests/lifecycle.h`` and ``/tests/lifecycle.c``. 75 76If you want to add new test(s) for a programmer, first you look whether that programmer has any tests already, or none at all. 77Test source file has the same name as a programmer itself, for example programmer ``dummyflasher.c`` has its dedicated tests in 78``/tests/dummyflasher.c`` file. Either add your tests to an existing file, or create new file for a programmer that had no tests 79so far. The latter is more challenging, but it is very useful and highly appreciated. 80 81For programmers tests, the test scenario most likely won't be long: most likely it is one of the options to run lifecycle with 82given combination of programmer params as an input. Most time and effort is typically spent on mocking (see above), and this 83type of tests will indeed look like a mini-emulator. 84 85Chip operations tests 86--------------------- 87 88These tests are based on dummyflasher programmer and they are running operations of a chip: read, write, erase, verify. 89The test defines mock chip(s) with given properties, and all the operations of the chip are redirected to mock functions. 90Mock chip has its own mock memory (an array of bytes) and all operations are performed on this array of bytes. 91As all the others, these tests are completely independent of hardware, and are focused on testing core flashrom logic 92for read, write, erase, verify. 93 94Examples of chip operation tests are: ``tests/chip.c``, ``/tests/chip_wp.c`` (focused on write-protection logic), 95``/tests/erase_func_algo.c`` (focused on erasing and writing logic, especially the choice of erase blocks for given 96layout regions). 97 98Misc 99---- 100 101All other tests. You choose a function that you want to test, call it with given arguments, assert the results are as expected. 102