1LTP Test Writing Guidelines 2=========================== 3 4This document describes LTP guidelines and LTP test interface and is intended 5for anybody who want to write or modify a LTP testcase. It's not a definitive 6guide and it's not, by any means, a substitute for common sense. 7 81. General Rules 9---------------- 10 111.1 Simplicity 12~~~~~~~~~~~~~~ 13 14For all it's worth keep the testcases simple or better as simple as possible. 15The kernel and libc are tricky beasts and the complexity imposed by their 16interfaces is quite high. Concentrate on the interface you want to test and 17follow the UNIX philosophy. It's a good idea to make the test as 18self-contained as possible too (it should not depend on tools or libraries 19that are not widely available). 20 21Do not reinvent the wheel! 22 23* Use LTP standard interface 24* Do not add custom PASS/FAIL reporting functions 25* Do not write Makefiles from scratch, 26 use LTP build system instead, etc. 27* ... 28 291.2 Code duplication 30~~~~~~~~~~~~~~~~~~~~ 31 32Copy & paste is a good servant but very poor master. If you are about to copy a 33large part of the code from one testcase to another, think what would happen if 34you find bug in the code that has been copied all around the tree. What about 35moving it to a library instead? 36 37The same goes for short but complicated parts, whenever you are about to copy & 38paste a syscall wrapper that packs arguments accordingly to machine 39architecture or similarly complicated code, put it into a header instead. 40 411.3 Coding style 42~~~~~~~~~~~~~~~~ 43 441.3.1 C coding style 45^^^^^^^^^^^^^^^^^^^^ 46 47LTP adopted Linux kernel coding style. If you aren't familiar with its rules 48locate 'linux/Documentation/CodingStyle' in the kernel sources and read it, 49it's a well written introduction. 50 51There is also a checkpatch (see 'linux/scripts/checkpatch.pl') script that can 52be used to check your patches before the submission. 53 54NOTE: If checkpatch does not report any problems, the code still may be wrong 55 as the tool only looks for common mistakes. 56 571.3.2 Shell coding style 58^^^^^^^^^^^^^^^^^^^^^^^^ 59 60When writing testcases in shell write in *portable shell* only, it's a good 61idea to try to run the test using alternative shell (alternative to bash, for 62example dash) too. 63 64*Portable shell* means Shell Command Language as defined by POSIX with a 65exception of few widely used extensions, namely 'local' keyword used inside of 66functions and '-o' and '-a' test parameters (that are marked as obsolete in 67POSIX). 68 69You can either try to run the testcases on Debian which has '/bin/sh' pointing 70to 'dash' by default or install 'dash' on your favorite distribution and use 71it to run the tests. If your distribution lacks 'dash' package you can always 72compile it from http://gondor.apana.org.au/~herbert/dash/files/[source]. 73 74Debian also has nice devscript 75https://salsa.debian.org/debian/devscripts/raw/master/scripts/checkbashisms.pl[checkbashism.pl] 76that can be used to check for non-portable shell code. 77 78Here are some common sense style rules for shell 79 80* Keep lines under 80 chars 81 82* Use tabs for indentation 83 84* Keep things simple, avoid unnecessary subshells 85 86* Don't do confusing things (i.e. don't name your functions like common shell 87 commands, etc.) 88 89* Quote variables 90 91* Be consistent 92 931.4 Commenting code 94~~~~~~~~~~~~~~~~~~~ 95 96Comments can sometimes save you day but they can easily do more harm than 97good. There has been several cases where comments and actual implementation 98were drifting slowly apart which yielded into API misuses and hard to find 99bugs. Remember there is only one thing worse than no documentation, wrong 100documentation. 101 102Generally everybody should write code that is obvious (which unfortunately 103isn't always possible). If there is a code that needs to be commented keep it 104short and to the point. Never ever comment the obvious. 105 106In case of LTP testcases it's customary to add a paragraph with highlevel test 107description somewhere at the beginning of the file (usually right under the GPL 108header). This helps other people to understand the overall goal of the test 109before they dive into the technical details. 110 1111.5 Backwards compatibility 112~~~~~~~~~~~~~~~~~~~~~~~~~~~ 113 114LTP test should be as backward compatible as possible. Think of an enterprise 115distributions with long term support (more than five years since the initial 116release) or of an embedded platform that needs to use several years old 117toolchain supplied by the manufacturer. 118 119Therefore LTP test for more current features should be able to cope with older 120systems. It should at least compile fine and if it's not appropriate for the 121configuration it should return 'TCONF' (see test interface description below). 122 123There are several types of checks we use: 124 125The *configure script* is usually used to detect availability of a function 126declarations in system headers. It's used to disable tests at compile time. 127 128We also have runtime kernel version detection that can be used to disable 129tests at runtime. 130 131Checking the *errno* value is another type of runtime check. Most of the 132syscalls returns either 'EINVAL' or 'ENOSYS' when syscall was not implemented 133or was disabled upon kernel compilation. 134 135Sometimes it also makes sense to define a few macros instead of creating 136configure test. One example are Linux specific POSIX clock ids in 137'include/lapi/posix_clocks.h'. 138 1391.6 Dealing with messed up legacy code 140~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 141 142LTP contains a lot of old and messy code and we are cleaning it up as fast as 143we can but despite the efforts there is still a lot. If you start modifying 144old or a messed up testcase and your changes are more complicated than simple 145typo fixes you should do a cleanup first (in a separate patch). It's easier to 146review the changes if you separate the formatting fixes from the changes that 147affects the test behavior. 148 149The same goes for moving files. If you need a rename or move file do it in a 150separate patch. 151 1521.7 License 153~~~~~~~~~~~ 154 155Code contributed to LTP should be licensed under GPLv2+ (GNU GPL version 2 or 156any later version). Use `SPDX-License-Identifier: GPL-2.0-or-later`. 157 1582. Writing a testcase 159--------------------- 160 1612.1 LTP Structure 162~~~~~~~~~~~~~~~~~ 163 164The structure of LTP is quite simple. Each test is a binary written either in 165portable shell or C. The test gets a configuration via environment variables 166and/or command line parameters, it prints additional information into the 167stdout and reports overall success/failure via the exit value. 168 169Tests are generally placed under the 'testcases/' directory. Everything that 170is a syscall or (slightly confusingly) libc syscall wrapper goes under 171'testcases/kernel/syscalls/'. Then there is 'testcases/open_posix_testsuite' 172which is a well maintained fork of the upstream project that has been dead 173since 2005 and also a number of directories with tests for more specific 174features. 175 1762.1.1 Runtest Files 177^^^^^^^^^^^^^^^^^^^ 178 179The list of tests to be executed is stored in runtest files under the 180'runtest/' directory. The default set of runtest files to be executed is 181stored in 'scenario_groups/default'. When you add a test you should add 182corresponding entries into some runtest file(s) as well. 183 184For syscall tests (these placed under 'testcases/kernel/syscalls/') use 185'runtest/syscalls' file, for kernel related tests for memory management we 186have 'runtest/mm', etc. 187 188IMPORTANT: The runtest files should have one entry per a test. Creating a 189 wrapper that runs all your tests and adding it as a single test 190 into runtest file is strongly discouraged. 191 1922.1.2 Datafiles 193^^^^^^^^^^^^^^^ 194 195If your test needs datafiles to work, these should be put into a subdirectory 196named 'datafiles' and installed into the 'testcases/data/$TCID' directory (to 197do that you have to add 'INSTALL_DIR := testcases/data/TCID' into the 198'datafiles/Makefile'). 199 200You can obtain path to datafiles via $TST_DATAROOT provided by test.sh 201'$TST_DATAROOT/...' 202or via C function 'tst_dataroot()' provided by libltp: 203 204[source,c] 205------------------------------------------------------------------------------- 206const char *dataroot = tst_dataroot(); 207------------------------------------------------------------------------------- 208 209Datafiles can also be accessed as '$LTPROOT/testcases/data/$TCID/...', 210but '$TST_DATAROOT' and 'tst_dataroot()' are preferred as these can be used 211when running testcases directly in git tree as well as from install 212location. 213 214The path is constructed according to these rules: 215 2161. if '$LTPROOT' is set, return '$LTPROOT/testcases/data/$TCID' 2172. else if 'tst_tmpdir()' was called return '$STARTWD/datafiles' 218 (where '$STARTWD' is initial working directory as recorded by 'tst_tmpdir()') 2193. else return '$CWD/datafiles' 220 221See 'testcases/commands/file/' for example. 222 2232.1.3 Subexecutables 224^^^^^^^^^^^^^^^^^^^^ 225 226If you test needs to execute a binary, place it in the same directory as the 227testcase and name the file starting with '${test_binary_name}_'. Once the 228test is executed by the framework, the path to the directory with all LTP 229binaries is added to the '$PATH' and you can execute it just by its name. 230 231TIP: If you need to execute such test from the LTP tree, you can add path to 232 current directory to '$PATH' manually with: 'PATH="$PATH:$PWD" ./foo01'. 233 2342.2 Writing a test in C 235~~~~~~~~~~~~~~~~~~~~~~~ 236 2372.2.1 Basic test structure 238^^^^^^^^^^^^^^^^^^^^^^^^^^ 239 240Let's start with an example, following code is a simple test for a 'getenv()'. 241 242[source,c] 243------------------------------------------------------------------------------- 244/* 245 * This is test for basic functionality of getenv(). 246 * 247 * - create an env variable and verify that getenv() can get get it 248 * - call getenv() with nonexisting variable name, check that it returns NULL 249 */ 250 251#include "tst_test.h" 252 253#define ENV1 "LTP_TEST_ENV" 254#define ENV2 "LTP_TEST_THIS_DOES_NOT_EXIST" 255#define ENV_VAL "val" 256 257static void setup(void) 258{ 259 if (setenv(ENV1, ENV_VAL, 1)) 260 tst_brk(TBROK | TERRNO, "setenv() failed"); 261} 262 263static void test(void) 264{ 265 char *ret; 266 267 ret = getenv(ENV1); 268 269 if (!ret) { 270 tst_res(TFAIL, "getenv(" ENV1 ") = NULL"); 271 goto next; 272 } 273 274 if (!strcmp(ret, ENV_VAL)) { 275 tst_res(TPASS, "getenv(" ENV1 ") = '"ENV_VAL "'"); 276 } else { 277 tst_res(TFAIL, "getenv(" ENV1 ") = '%s', expected '" 278 ENV_VAL "'", ret); 279 } 280 281next: 282 ret = getenv(ENV2); 283 284 if (ret) 285 tst_res(TFAIL, "getenv(" ENV2 ") = '%s'", ret); 286 else 287 tst_res(TPASS, "getenv(" ENV2 ") = NULL"); 288} 289 290static struct tst_test test = { 291 .test_all = test, 292 .setup = setup, 293}; 294------------------------------------------------------------------------------- 295 296Each test includes the 'tst_test.h' header and must define the 'struct 297tst_test test' structure. 298 299The overall test initialization is done in the 'setup()' function. 300 301The overall cleanup is done in a 'cleanup()' function. Here 'cleanup()' is 302omitted as the test does not have anything to clean up. If cleanup is set in 303the test structure it's called on test exit just before the test library 304cleanup. That especially means that cleanup can be called at any point in a 305test execution. For example even when a test setup step has failed, therefore 306the 'cleanup()' function must be able to cope with unfinished initialization, 307and so on. 308 309The test itself is done in the 'test()' function. The test function must work 310fine if called in a loop. 311 312There are two types of a test function pointers in the test structure. The 313first one is a '.test_all' pointer that is used when test is implemented as a 314single function. Then there is a '.test' function along with the number of 315tests '.tcnt' that allows for more detailed result reporting. If the '.test' 316pointer is set the function is called '.tcnt' times with an integer parameter 317in range of [0, '.tcnt' - 1]. 318 319IMPORTANT: Only one of '.test' and '.test_all' can be set at a time. 320 321Each test has a default timeout set to 300s. The default timeout can be 322overridden by setting '.timeout' in the test structure or by calling 323'tst_set_timeout()' in the test 'setup()'. There are a few testcases whose run 324time may vary arbitrarily, for these timeout can be disabled by setting it to 325-1. 326 327Test can find out how much time (in seconds) is remaining to timeout, 328by calling 'tst_timeout_remaining()'. 329 330A word about the cleanup() callback 331+++++++++++++++++++++++++++++++++++ 332 333There are a few rules that needs to be followed in order to write correct 334cleanup() callback. 335 3361. Free only resources that were initialized. Keep in mind that callback can 337 be executed at any point in the test run. 338 3392. Make sure to free resources in the reverse order they were 340 initialized. (Some of the steps may not depend on others and everything 341 will work if there were swapped but let's keep it in order.) 342 343The first rule may seem complicated at first however, on the contrary, it's 344quite easy. All you have to do is to keep track of what was already 345initialized. For example file descriptors needs to be closed only if they were 346assigned a valid file descriptor. For most of the things you need to create 347extra flag that is set right after successful initialization though. Consider, 348for example, test setup below. 349 350We also prefer cleaning up resources that would otherwise be released on the 351program exit. There are two main reasons for this decision. Resources such as 352file descriptors and mmaped memory could block umounting a block device in 353cases where the test library has mounted a filesystem for the test temporary 354directory. Not freeing allocated memory would upset static analysis and tools 355such as valgrind and produce false-positives when checking for leaks in the 356libc and other low level libraries. 357 358[source,c] 359------------------------------------------------------------------------------- 360static int fd0, fd1, mount_flag; 361 362#define MNTPOINT "mntpoint" 363#define FILE1 "mntpoint/file1" 364#define FILE2 "mntpoint/file2" 365 366static void setup(void) 367{ 368 SAFE_MKDIR(MNTPOINT, 0777); 369 SAFE_MKFS(tst_device->dev, tst_device->fs_type, NULL, NULL); 370 SAFE_MOUNT(tst_device->dev, MNTPOINT, tst_device->fs_type, 0, 0); 371 mount_flag = 1; 372 373 fd0 = SAFE_OPEN(cleanup, FILE1, O_CREAT | O_RDWR, 0666); 374 fd1 = SAFE_OPEN(cleanup, FILE2, O_CREAT | O_RDWR, 0666); 375} 376------------------------------------------------------------------------------- 377 378In this case the 'cleanup()' function may be invoked when any of the 'SAFE_*' 379macros has failed and therefore must be able to work with unfinished 380initialization as well. Since global variables are initialized to zero we can 381just check that fd > 0 before we attempt to close it. The mount function 382requires extra flag to be set after device was successfully mounted. 383 384[source,c] 385------------------------------------------------------------------------------- 386static void cleanup(void) 387{ 388 if (fd1 > 0) 389 SAFE_CLOSE(fd1); 390 391 if (fd0 > 0) 392 SAFE_CLOSE(fd0); 393 394 if (mount_flag && tst_umouont(MNTPOINT)) 395 tst_res(TWARN | TERRNO, "umount(%s)", MNTPOINT); 396} 397------------------------------------------------------------------------------- 398 399IMPORTANT: 'SAFE_MACROS()' used in cleanup *do not* exit the test. Failure 400 only produces a warning and the 'cleanup()' carries on. This is 401 intentional as we want to execute as much 'cleanup()' as possible. 402 403WARNING: Calling tst_brk() in test 'cleanup()' does not exit the test as well 404 and 'TBROK' is converted to 'TWARN'. 405 406NOTE: Creation and removal of the test temporary directory is handled in 407 the test library and the directory is removed recursively. Therefore 408 we do not have to remove files and directories in the test cleanup. 409 4102.2.2 Basic test interface 411^^^^^^^^^^^^^^^^^^^^^^^^^^ 412 413[source,c] 414------------------------------------------------------------------------------- 415void tst_res(int ttype, char *arg_fmt, ...); 416------------------------------------------------------------------------------- 417 418Printf-like function to report test result, it's mostly used with ttype: 419 420|============================== 421| 'TPASS' | Test has passed. 422| 'TFAIL' | Test has failed. 423| 'TINFO' | General message. 424|============================== 425 426The 'ttype' can be combined bitwise with 'TERRNO' or 'TTERRNO' to print 427'errno', 'TST_ERR' respectively. 428 429[source,c] 430------------------------------------------------------------------------------- 431void tst_brk(int ttype, char *arg_fmt, ...); 432------------------------------------------------------------------------------- 433 434Printf-like function to report error and exit the test, it can be used with ttype: 435 436|============================================================ 437| 'TBROK' | Something has failed in test preparation phase. 438| 'TCONF' | Test is not appropriate for current configuration 439 (syscall not implemented, unsupported arch, ...) 440|============================================================ 441 442The 'ttype' can be combined bitwise with 'TERRNO' or 'TTERRNO' to print 443'errno', 'TST_ERR' respectively. 444 445[source,c] 446------------------------------------------------------------------------------- 447const char *tst_strsig(int sig); 448------------------------------------------------------------------------------- 449 450Return the given signal number's corresponding string. 451 452[source,c] 453------------------------------------------------------------------------------- 454const char *tst_strerrno(int err); 455------------------------------------------------------------------------------- 456 457Return the given errno number's corresponding string. Using this function to 458translate 'errno' values to strings is preferred. You should not use the 459'strerror()' function in the testcases. 460 461[source,c] 462------------------------------------------------------------------------------- 463const char *tst_strstatus(int status); 464------------------------------------------------------------------------------- 465 466Returns string describing the status as returned by 'wait()'. 467 468WARNING: This function is not thread safe. 469 470[source,c] 471------------------------------------------------------------------------------- 472void tst_set_timeout(unsigned int timeout); 473------------------------------------------------------------------------------- 474 475Allows for setting timeout per test iteration dynamically in the test setup(), 476the timeout is specified in seconds. There are a few testcases whose runtime 477can vary arbitrarily, these can disable timeouts by setting it to -1. 478 479[source,c] 480------------------------------------------------------------------------------- 481void tst_flush(void); 482------------------------------------------------------------------------------- 483 484Flush output streams, handling errors appropriately. 485 486This function is rarely needed when you have to flush the output streams 487before calling 'fork()' or 'clone()'. Note that the 'SAFE_FORK()' calls this 488function automatically. See 3.4 FILE buffers and fork() for explanation why is 489this needed. 490 4912.2.3 Test temporary directory 492^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 493 494If '.needs_tmpdir' is set to '1' in the 'struct tst_test' unique test 495temporary is created and it's set as the test working directory. Tests *MUST 496NOT* create temporary files outside that directory. The flag is not needed to 497be set when use these flags: '.all_filesystems', '.format_device', '.mntpoint', 498'.mount_device' '.needs_checkpoints', '.needs_device', '.resource_file' 499(these flags imply creating temporary directory). 500 501IMPORTANT: Close all file descriptors (that point to files in test temporary 502 directory, even the unlinked ones) either in the 'test()' function 503 or in the test 'cleanup()' otherwise the test may break temporary 504 directory removal on NFS (look for "NFS silly rename"). 505 5062.2.4 Safe macros 507^^^^^^^^^^^^^^^^^ 508 509Safe macros aim to simplify error checking in test preparation. Instead of 510calling system API functions, checking for their return value and aborting the 511test if the operation has failed, you just use corresponding safe macro. 512 513Use them whenever it's possible. 514 515Instead of writing: 516 517[source,c] 518------------------------------------------------------------------------------- 519 fd = open("/dev/null", O_RDONLY); 520 if (fd < 0) 521 tst_brk(TBROK | TERRNO, "opening /dev/null failed"); 522------------------------------------------------------------------------------- 523 524You write just: 525 526[source,c] 527------------------------------------------------------------------------------- 528 fd = SAFE_OPEN("/dev/null", O_RDONLY); 529------------------------------------------------------------------------------- 530 531IMPORTANT: The SAFE_CLOSE() function also sets the passed file descriptor to -1 532 after it's successfully closed. 533 534They can also simplify reading and writing of sysfs files, you can, for 535example, do: 536 537[source,c] 538------------------------------------------------------------------------------- 539 SAFE_FILE_SCANF("/proc/sys/kernel/pid_max", "%lu", &pid_max); 540------------------------------------------------------------------------------- 541 542See 'include/tst_safe_macros.h', 'include/tst_safe_stdio.h' and 543'include/tst_safe_file_ops.h' and 'include/tst_safe_net.h' for a complete list. 544 5452.2.5 Test specific command line options 546^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 547 548[source,c] 549------------------------------------------------------------------------------- 550struct tst_option { 551 char *optstr; 552 char **arg; 553 char *help; 554}; 555------------------------------------------------------------------------------- 556 557Test specific command line parameters can be passed with the 'NULL'-terminated 558array of 'struct tst_option'. The 'optstr' is the command line option i.e. "o" 559or "o:" if option has a parameter. Only short options are supported. The 'arg' 560is where 'optarg' is stored upon match. If option has no parameter it's set to 561non-'NULL' value if option was present. The 'help' is a short help string. 562 563NOTE: The test parameters must not collide with common test parameters defined 564 in the library the currently used ones are +-i+, +-I+, +-C+, and +-h+. 565 566[source,c] 567------------------------------------------------------------------------------- 568int tst_parse_int(const char *str, int *val, int min, int max); 569int tst_parse_float(const char *str, float *val, float min, float max); 570------------------------------------------------------------------------------- 571 572Helpers for parsing the strings returned in the 'struct tst_option'. 573 574Both return zero on success and 'errno', mostly 'EINVAL' or 'ERANGE', on 575failure. 576 577Both functions are no-op if 'str' is 'NULL'. 578 579The valid range for result includes both 'min' and 'max'. 580 581.Example Usage 582[source,c] 583------------------------------------------------------------------------------- 584#include <limits.h> 585#include "tst_test.h" 586 587static char *str_threads; 588static int threads = 10; 589 590static struct tst_option options[] = { 591 {"t:", &str_threads, "Number of threads (default 10)"}, 592 ... 593 {NULL, NULL, NULL} 594}; 595 596static void setup(void) 597{ 598 if (tst_parse_int(str_threads, &threads, 1, INT_MAX)) 599 tst_brk(TBROK, "Invalid number of threads '%s'", str_threads); 600 601 ... 602} 603 604static void test_threads(void) 605{ 606 ... 607 608 for (i = 0; i < threads; i++) { 609 ... 610 } 611 612 ... 613} 614 615static struct tst_test test = { 616 ... 617 .options = options, 618 ... 619}; 620------------------------------------------------------------------------------- 621 622 6232.2.6 Runtime kernel version detection 624^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 625 626Testcases for newly added kernel functionality require kernel newer than a 627certain version to run. All you need to skip a test on older kernels is to 628set the '.min_kver' string in the 'struct tst_test' to a minimal required 629kernel version, e.g. '.min_kver = "2.6.30"'. 630 631For more complicated operations such as skipping a test for a certain range 632of kernel versions, following functions could be used: 633 634[source,c] 635------------------------------------------------------------------------------- 636int tst_kvercmp(int r1, int r2, int r3); 637 638struct tst_kern_exv { 639 char *dist_name; 640 char *extra_ver; 641}; 642 643int tst_kvercmp2(int r1, int r2, int r3, struct tst_kern_exv *vers); 644------------------------------------------------------------------------------- 645 646These two functions are intended for runtime kernel version detection. They 647parse the output from 'uname()' and compare it to the passed values. 648 649The return value is similar to the 'strcmp()' function, i.e. zero means equal, 650negative value means that the kernel is older than than the expected value and 651positive means that it's newer. 652 653The second function 'tst_kvercmp2()' allows for specifying per-vendor table of 654kernel versions as vendors typically backport fixes to their kernels and the 655test may be relevant even if the kernel version does not suggests so. See 656'testcases/kernel/syscalls/inotify/inotify04.c' for example usage. 657 658WARNING: The shell 'tst_kvercmp' maps the result into unsigned integer - the 659 process exit value. 660 6612.2.7 Fork()-ing 662^^^^^^^^^^^^^^^^ 663 664Be wary that if the test forks and there were messages printed by the 665'tst_*()' interfaces, the data may still be in libc/kernel buffers and these 666*ARE NOT* flushed automatically. 667 668This happens when 'stdout' gets redirected to a file. In this case, the 669'stdout' is not line buffered, but block buffered. Hence after a fork content 670of the buffers will be printed by the parent and each of the children. 671 672To avoid that you should use 'SAFE_FORK()'. 673 674IMPORTANT: You have to set the '.forks_child' flag in the test structure 675 if your testcase forks. 676 6772.2.8 Doing the test in the child process 678^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 679 680Results reported by 'tst_res()' are propagated to the parent test process via 681block of shared memory. 682 683Calling 'tst_brk()' causes child process to exit with non-zero exit value. 684Which means that it's safe to use 'SAFE_*()' macros in the child processes as 685well. 686 687Children that outlive the 'test()' function execution are waited for in the 688test library. Unclean child exit (killed by signal, non-zero exit value, etc.) 689will cause the main test process to exit with 'tst_brk()', which especially 690means that 'TBROK' propagated from a child process will cause the whole test 691to exit with 'TBROK'. 692 693If a test needs a child that segfaults or does anything else that cause it to 694exit uncleanly all you need to do is to wait for such children from the 695'test()' function so that it's reaped before the main test exits the 'test()' 696function. 697 698[source,c] 699------------------------------------------------------------------------------- 700#include "tst_test.h" 701 702void tst_reap_children(void); 703------------------------------------------------------------------------------- 704 705The 'tst_reap_children()' function makes the process wait for all of its 706children and exits with 'tst_brk(TBROK, ...)' if any of them returned 707a non zero exit code. 708 709.Using 'tst_res()' from binaries started by 'exec()' 710[source,c] 711------------------------------------------------------------------------------- 712/* test.c */ 713#define _GNU_SOURCE 714#include <unistd.h> 715#include "tst_test.h" 716 717static void do_test(void) 718{ 719 char *const argv[] = {"test_exec_child", NULL}; 720 char path[4096]; 721 722 if (tst_get_path("test_exec_child", path, sizeof(path))) 723 tst_brk(TCONF, "Couldn't find test_exec_child in $PATH"); 724 725 execve(path, argv, environ); 726 727 tst_res(TBROK | TERRNO, "EXEC!"); 728} 729 730static struct tst_test test = { 731 .test_all = do_test, 732 .child_needs_reinit = 1, 733}; 734 735/* test_exec_child.c */ 736#define TST_NO_DEFAULT_MAIN 737#include "tst_test.h" 738 739int main(void) 740{ 741 tst_reinit(); 742 tst_res(TPASS, "Child passed!"); 743 return 0; 744} 745------------------------------------------------------------------------------- 746 747The 'tst_res()' function can be also used from binaries started by 'exec()', 748the parent test process has to set the '.child_needs_reinit' flag so that the 749library prepares for it and has to make sure the 'LTP_IPC_PATH' environment 750variable is passed down, then the very fist thing the program has to call in 751'main()' is 'tst_reinit()' that sets up the IPC. 752 7532.2.9 Fork() and Parent-child synchronization 754^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 755 756As LTP tests are written for Linux, most of the tests involve fork()-ing and 757parent-child process synchronization. LTP includes a checkpoint library that 758provides wait/wake futex based functions. 759 760In order to use checkpoints the '.needs_checkpoints' flag in the 'struct 761tst_test' must be set to '1', this causes the test library to initialize 762checkpoints before the 'test()' function is called. 763 764[source,c] 765------------------------------------------------------------------------------- 766#include "tst_test.h" 767 768TST_CHECKPOINT_WAIT(id) 769 770TST_CHECKPOINT_WAIT2(id, msec_timeout) 771 772TST_CHECKPOINT_WAKE(id) 773 774TST_CHECKPOINT_WAKE2(id, nr_wake) 775 776TST_CHECKPOINT_WAKE_AND_WAIT(id) 777------------------------------------------------------------------------------- 778 779The checkpoint interface provides pair of wake and wait functions. The 'id' is 780unsigned integer which specifies checkpoint to wake/wait for. As a matter of 781fact it's an index to an array stored in a shared memory, so it starts on 782'0' and there should be enough room for at least of hundred of them. 783 784The 'TST_CHECKPOINT_WAIT()' and 'TST_CHECKPOINT_WAIT2()' suspends process 785execution until it's woken up or until timeout is reached. 786 787The 'TST_CHECKPOINT_WAKE()' wakes one process waiting on the checkpoint. 788If no process is waiting the function retries until it success or until 789timeout is reached. 790 791If timeout has been reached process exits with appropriate error message (uses 792'tst_brk()'). 793 794The 'TST_CHECKPOINT_WAKE2()' does the same as 'TST_CHECKPOINT_WAKE()' but can 795be used to wake precisely 'nr_wake' processes. 796 797The 'TST_CHECKPOINT_WAKE_AND_WAIT()' is a shorthand for doing wake and then 798immediately waiting on the same checkpoint. 799 800Child processes created via 'SAFE_FORK()' are ready to use the checkpoint 801synchronization functions, as they inherited the mapped page automatically. 802 803Child processes started via 'exec()', or any other processes not forked from 804the test process must initialize the checkpoint by calling 'tst_reinit()'. 805 806For the details of the interface, look into the 'include/tst_checkpoint.h'. 807 808[source,c] 809------------------------------------------------------------------------------- 810#include "tst_test.h" 811 812/* 813 * Waits for process state change. 814 * 815 * The state is one of the following: 816 * 817 * R - process is running 818 * S - process is sleeping 819 * D - process sleeping uninterruptibly 820 * Z - zombie process 821 * T - process is traced 822 */ 823TST_PROCESS_STATE_WAIT(pid, state) 824------------------------------------------------------------------------------- 825 826The 'TST_PROCESS_STATE_WAIT()' waits until process 'pid' is in requested 827'state'. The call polls +/proc/pid/stat+ to get this information. 828 829It's mostly used with state 'S' which means that process is sleeping in kernel 830for example in 'pause()' or any other blocking syscall. 831 8322.2.10 Signals and signal handlers 833^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 834 835If you need to use signal handlers, keep the code short and simple. Don't 836forget that the signal handler is called asynchronously and can interrupt the 837code execution at any place. 838 839This means that problems arise when global state is changed both from the test 840code and signal handler, which will occasionally lead to: 841 842* Data corruption (data gets into inconsistent state), this may happen, for 843 example, for any operations on 'FILE' objects. 844 845* Deadlock, this happens, for example, if you call 'malloc(2)', 'free(2)', 846 etc. from both the test code and the signal handler at the same time since 847 'malloc' has global lock for it's internal data structures. (Be wary that 848 'malloc(2)' is used by the libc functions internally too.) 849 850* Any other unreproducible and unexpected behavior. 851 852Quite common mistake is to call 'exit(3)' from a signal handler. Note that this 853function is not signal-async-safe as it flushes buffers, etc. If you need to 854exit a test immediately from a signal handler use '_exit(2)' instead. 855 856TIP: See 'man 7 signal' for the list of signal-async-safe functions. 857 858If a signal handler sets a variable, its declaration must be 'volatile', 859otherwise compiler may misoptimize the code. This is because the variable may 860not be changed in the compiler code flow analysis. There is 'sig_atomic_t' 861type defined in C99 but this one *DOES NOT* imply 'volatile' (it's just a 862'typedef' to 'int'). So the correct type for a flag that is changed from a 863signal handler is either 'volatile int' or 'volatile sig_atomic_t'. 864 865If a crash (e.g. triggered by signal SIGSEGV) is expected in testing, you 866can avoid creation of core files by calling tst_no_corefile() function. 867This takes effect for process (and its children) which invoked it, unless 868they subsequently modify RLIMIT_CORE. 869 870Note that LTP library will reap any processes that test didn't reap itself, 871and report any non-zero exit code as failure. 872 8732.2.11 Kernel Modules 874^^^^^^^^^^^^^^^^^^^^^ 875 876There are certain cases where the test needs a kernel part and userspace part, 877happily, LTP can build a kernel module and then insert it to the kernel on test 878start for you. See 'testcases/kernel/device-drivers/block' for details. 879 8802.2.12 Useful macros 881^^^^^^^^^^^^^^^^^^^^^ 882 883[source,c] 884------------------------------------------------------------------------------- 885ARRAY_SIZE(arr) 886------------------------------------------------------------------------------- 887 888Returns the size of statically defined array, i.e. 889'(sizeof(arr) / sizeof(*arr))' 890 891[source,c] 892------------------------------------------------------------------------------- 893LTP_ALIGN(x, a) 894------------------------------------------------------------------------------- 895 896Aligns the x to be next multiple of a. The a must be power of 2. 897 8982.2.13 Filesystem type detection 899^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 900 901Some tests are known to fail on certain filesystems (you cannot swap on TMPFS, 902there are unimplemented 'fcntl()' etc.). 903 904If your test needs to be skipped on certain filesystems, use the interface 905below: 906 907[source,c] 908------------------------------------------------------------------------------- 909#include "tst_test.h" 910 911 /* 912 * Unsupported only on NFS. 913 */ 914 if (tst_fs_type(".") == TST_NFS_MAGIC) 915 tst_brk(TCONF, "Test not supported on NFS filesystem"); 916 917 918 /* 919 * Unsupported on NFS, TMPFS and RAMFS 920 */ 921 long type; 922 923 switch ((type = tst_fs_type("."))) { 924 case TST_NFS_MAGIC: 925 case TST_TMPFS_MAGIC: 926 case TST_RAMFS_MAGIC: 927 tst_brk(TCONF, "Test not supported on %s filesystem", 928 tst_fs_type_name(type)); 929 break; 930 } 931------------------------------------------------------------------------------- 932 9332.2.14 Thread-safety in the LTP library 934^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 935 936It is safe to use library 'tst_res()' function in multi-threaded tests. 937 938Only the main thread must return from the 'test()' function to the test 939library and that must be done only after all threads that may call any library 940function has been terminated. That especially means that threads that may call 941'tst_brk()' must terminate before the execution of the 'test()' function 942returns to the library. This is usually done by the main thread joining all 943worker threads at the end of the 'test()' function. Note that the main thread 944will never get to the library code in a case that 'tst_brk()' was called from 945one of the threads since it will sleep at least in 'pthread_join()' on the 946thread that called the 'tst_brk()' till 'exit()' is called by 'tst_brk()'. 947 948The test-supplied cleanup function runs *concurrently* to the rest of the 949threads in a case that cleanup was entered from 'tst_brk()'. Subsequent 950threads entering 'tst_brk()' must be suspended or terminated at the start of 951the user supplied cleanup function. It may be necessary to stop or exit 952the rest of the threads before the test cleans up as well. For example threads 953that create new files should be stopped before temporary directory is be 954removed. 955 956Following code example shows thread safe cleanup function example using atomic 957increment as a guard. The library calls its cleanup after the execution returns 958from the user supplied cleanup and expects that only one thread returns from 959the user supplied cleanup to the test library. 960 961[source,c] 962------------------------------------------------------------------------------- 963#include "tst_test.h" 964 965static void cleanup(void) 966{ 967 static int flag; 968 969 if (tst_atomic_inc(&flag) != 1) 970 pthread_exit(NULL); 971 972 /* if needed stop the rest of the threads here */ 973 974 ... 975 976 /* then do cleanup work */ 977 978 ... 979 980 /* only one thread returns to the library */ 981} 982------------------------------------------------------------------------------- 983 984 9852.2.15 Testing with a block device 986^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 987 988Some tests needs a block device (inotify tests, syscall 'EROFS' failures, 989etc.). LTP library contains a code to prepare a testing device. 990 991If '.needs_device' flag in the 'struct tst_test' is set the 'tst_device' 992structure is initialized with a path to a test device and default filesystem 993to be used. 994 995You can also request minimal device size in megabytes by setting 996'.dev_min_size' the device is guaranteed to have at least the requested size 997then. 998 999If '.format_device' flag is set the device is formatted with a filesystem as 1000well. You can use '.dev_fs_type' to override the default filesystem type if 1001needed and pass additional options to mkfs via '.dev_fs_opts' and 1002'.dev_extra_opts' pointers. Note that '.format_device' implies '.needs_device' 1003there is no need to set both. 1004 1005If '.mount_device' is set, the device is mounted at '.mntpoint' which is used 1006to pass a directory name that will be created and used as mount destination. 1007You can pass additional flags and data to the mount command via '.mnt_flags' 1008and '.mnt_data' pointers. Note that '.mount_device' implies '.needs_device' 1009and '.format_device' so there is no need to set the later two. 1010 1011If '.needs_rofs' is set, read-only filesystem is mounted at '.mntpoint' this 1012one is supposed to be used for 'EROFS' tests. 1013 1014If '.all_filesystems' is set the test function is executed for all supported 1015filesystems. Supported filesystems are detected based on existence of the 1016'mkfs.$fs' helper and on kernel support to mount it. For each supported 1017filesystem the 'tst_device.fs_type' is set to the currently tested fs type, if 1018'.format_device' is set the device is formatted as well, if '.mount_device' is 1019set it's mounted at '.mntpoint'. Also the test timeout is reset for each 1020execution of the test function. This flag is expected to be used for filesystem 1021related syscalls that are at least partly implemented in the filesystem 1022specific code e.g. fallocate(). 1023 1024[source,c] 1025------------------------------------------------------------------------------- 1026#include "tst_test.h" 1027 1028struct tst_device { 1029 const char *dev; 1030 const char *fs_type; 1031}; 1032 1033extern struct tst_device *tst_device; 1034 1035int tst_umount(const char *path); 1036------------------------------------------------------------------------------- 1037 1038In case that 'LTP_DEV' is passed to the test in an environment, the library 1039checks that the file exists and that it's a block device, if 1040'.device_min_size' is set the device size is checked as well. If 'LTP_DEV' 1041wasn't set or if size requirements were not met a temporary file is created 1042and attached to a free loop device. 1043 1044If there is no usable device and loop device couldn't be initialized the test 1045exits with 'TCONF'. 1046 1047The 'tst_umount()' function works exactly as 'umount(2)' but retries several 1048times on 'EBUSY'. This is because various desktop daemons (gvfsd-trash is known 1049for that) may be stupid enough to probe all newly mounted filesystem which 1050results in 'umount(2)' failing with 'EBUSY'. 1051 1052IMPORTANT: All testcases should use 'tst_umount()' instead of 'umount(2)' to 1053 umount filesystems. 1054 1055[source,c] 1056------------------------------------------------------------------------------- 1057#include "tst_test.h" 1058 1059int tst_find_free_loopdev(const char *path, size_t path_len); 1060------------------------------------------------------------------------------- 1061 1062This function finds a free loopdev and returns the free loopdev minor (-1 for no 1063free loopdev). If path is non-NULL, it will be filled with free loopdev path. 1064If you want to use a customized loop device, we can call tst_find_free_loopdev 1065(NULL, 0) in tests to get a free minor number and then mknod. 1066 1067[source,c] 1068------------------------------------------------------------------------------- 1069#include "tst_test.h" 1070 1071unsigned long tst_dev_bytes_written(const char *dev); 1072------------------------------------------------------------------------------- 1073 1074This function reads test block device stat file (/sys/block/<device>/stat) and 1075returns the bytes written since the last invocation of this function. To avoid 1076FS deferred IO metadata/cache interference, we suggest doing "syncfs" before the 1077tst_dev_bytes_written first invocation. And an inline function named tst_dev_sync 1078is created for that intention. 1079 10802.2.16 Formatting a device with a filesystem 1081^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1082 1083[source,c] 1084------------------------------------------------------------------------------- 1085#include "tst_test.h" 1086 1087static void setup(void) 1088{ 1089 ... 1090 SAFE_MKFS(tst_device->dev, tst_device->fs_type, NULL, NULL); 1091 ... 1092} 1093------------------------------------------------------------------------------- 1094 1095This function takes a path to a device, filesystem type and an array of extra 1096options passed to mkfs. 1097 1098The fs options 'fs_opts' should either be 'NULL' if there are none, or a 1099'NULL' terminated array of strings such as: 1100+const char *const opts[] = {"-b", "1024", NULL}+. 1101 1102The extra options 'extra_opts' should either be 'NULL' if there are none, or a 1103'NULL' terminated array of strings such as +{"102400", NULL}+; 'extra_opts' 1104will be passed after device name. e.g: +mkfs -t ext4 -b 1024 /dev/sda1 102400+ 1105in this case. 1106 11072.2.17 Verifying a filesystem's free space 1108^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1109 1110Some tests have size requirements for the filesystem's free space. If these 1111requirements are not satisfied, the tests should be skipped. 1112 1113[source,c] 1114------------------------------------------------------------------------------- 1115#include "tst_test.h" 1116 1117int tst_fs_has_free(const char *path, unsigned int size, unsigned int mult); 1118------------------------------------------------------------------------------- 1119 1120The 'tst_fs_has_free()' function returns 1 if there is enough space and 0 if 1121there is not. 1122 1123The 'path' is the pathname of any directory/file within a filesystem. 1124 1125The 'mult' is a multiplier, one of 'TST_BYTES', 'TST_KB', 'TST_MB' or 'TST_GB'. 1126 1127The required free space is calculated by 'size * mult', e.g. 1128'tst_fs_has_free("/tmp/testfile", 64, TST_MB)' will return 1 if the 1129filesystem, which '"/tmp/testfile"' is in, has 64MB free space at least, and 0 1130if not. 1131 11322.2.18 Files, directories and fs limits 1133^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1134 1135Some tests need to know the maximum count of links to a regular file or 1136directory, such as 'rename(2)' or 'linkat(2)' to test 'EMLINK' error. 1137 1138[source,c] 1139------------------------------------------------------------------------------- 1140#include "tst_test.h" 1141 1142int tst_fs_fill_hardlinks(const char *dir); 1143------------------------------------------------------------------------------- 1144 1145Try to get maximum count of hard links to a regular file inside the 'dir'. 1146 1147NOTE: This number depends on the filesystem 'dir' is on. 1148 1149This function uses 'link(2)' to create hard links to a single file until it 1150gets 'EMLINK' or creates 65535 links. If the limit is hit, the maximum number of 1151hardlinks is returned and the 'dir' is filled with hardlinks in format 1152"testfile%i", where i belongs to [0, limit) interval. If no limit is hit or if 1153'link(2)' failed with 'ENOSPC' or 'EDQUOT', zero is returned and previously 1154created files are removed. 1155 1156[source,c] 1157------------------------------------------------------------------------------- 1158#include "tst_test.h" 1159 1160int tst_fs_fill_subdirs(const char *dir); 1161------------------------------------------------------------------------------- 1162 1163Try to get maximum number of subdirectories in directory. 1164 1165NOTE: This number depends on the filesystem 'dir' is on. For current kernel, 1166subdir limit is not available for all filesystems (available for ext2, ext3, 1167minix, sysv and more). If the test runs on some other filesystems, like ramfs, 1168tmpfs, it will not even try to reach the limit and return 0. 1169 1170This function uses 'mkdir(2)' to create directories in 'dir' until it gets 1171'EMLINK' or creates 65535 directories. If the limit is hit, the maximum number 1172of subdirectories is returned and the 'dir' is filled with subdirectories in 1173format "testdir%i", where i belongs to [0, limit - 2) interval (because each 1174newly created dir has two links already - the '.' and the link from parent 1175dir). If no limit is hit or if 'mkdir(2)' failed with 'ENOSPC' or 'EDQUOT', 1176zero is returned and previously created directories are removed. 1177 1178[source,c] 1179------------------------------------------------------------------------------- 1180#include "tst_test.h" 1181 1182int tst_dir_is_empty(const char *dir, int verbose); 1183------------------------------------------------------------------------------- 1184 1185Returns non-zero if directory is empty and zero otherwise. 1186 1187Directory is considered empty if it contains only '.' and '..'. 1188 1189[source,c] 1190------------------------------------------------------------------------------- 1191#include "tst_test.h" 1192 1193int tst_fill_fd(int fd, char pattern, size_t bs, size_t bcount); 1194------------------------------------------------------------------------------- 1195 1196Fill a file with specified pattern using file descriptor. 1197 1198[source,c] 1199------------------------------------------------------------------------------- 1200#include "tst_test.h" 1201 1202int tst_fill_file(const char *path, char pattern, size_t bs, size_t bcount); 1203------------------------------------------------------------------------------- 1204 1205Creates/overwrites a file with specified pattern using file path. 1206 12072.2.19 Getting an unused PID number 1208^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1209 1210Some tests require a 'PID', which is not used by the OS (does not belong to 1211any process within it). For example, kill(2) should set errno to 'ESRCH' if 1212it's passed such 'PID'. 1213 1214[source,c] 1215------------------------------------------------------------------------------- 1216#include "tst_test.h" 1217 1218pid_t tst_get_unused_pid(void); 1219------------------------------------------------------------------------------- 1220 1221Return a 'PID' value not used by the OS or any process within it. 1222 1223[source,c] 1224------------------------------------------------------------------------------- 1225#include "tst_test.h" 1226 1227int tst_get_free_pids(void); 1228------------------------------------------------------------------------------- 1229 1230Returns number of unused pids in the system. Note that this number may be 1231different once the call returns and should be used only for rough estimates. 1232 12332.2.20 Running executables 1234^^^^^^^^^^^^^^^^^^^^^^^^^^ 1235 1236[source,c] 1237------------------------------------------------------------------------------- 1238#include "tst_test.h" 1239 1240int tst_run_cmd(const char *const argv[], 1241 const char *stdout_path, 1242 const char *stderr_path, 1243 int pass_exit_val); 1244------------------------------------------------------------------------------- 1245 1246'tst_run_cmd' is a wrapper for 'vfork() + execvp()' which provides a way 1247to execute an external program. 1248 1249'argv[]' is a NULL-terminated array of strings starting with the program name 1250which is followed by optional arguments. 1251 1252A non-zero 'pass_exit_val' makes 'tst_run_cmd' return the program exit code to 1253the caller. A zero for 'pass_exit_val' makes 'tst_run_cmd' exit the tests 1254on failure. 1255 1256In case that 'execvp()' has failed and the 'pass_exit_val' flag was set, the 1257return value is '255' if 'execvp()' failed with 'ENOENT' and '254' otherwise. 1258 1259'stdout_path' and 'stderr_path' determine where to redirect the program 1260stdout and stderr I/O streams. 1261 1262.Example 1263[source,c] 1264------------------------------------------------------------------------------- 1265#include "tst_test.h" 1266 1267const char *const cmd[] = { "ls", "-l", NULL }; 1268 1269... 1270 /* Store output of 'ls -l' into log.txt */ 1271 tst_run_cmd(cmd, "log.txt", NULL, 0); 1272... 1273------------------------------------------------------------------------------- 1274 12752.2.21 Measuring elapsed time and helper functions 1276^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1277 1278[source,c] 1279------------------------------------------------------------------------------- 1280#include "tst_timer.h" 1281 1282void tst_timer_check(clockid_t clk_id); 1283 1284void tst_timer_start(clockid_t clk_id); 1285 1286void tst_timer_stop(void); 1287 1288struct timespec tst_timer_elapsed(void); 1289 1290long long tst_timer_elapsed_ms(void); 1291 1292long long tst_timer_elapsed_us(void); 1293 1294int tst_timer_expired_ms(long long ms); 1295------------------------------------------------------------------------------- 1296 1297The 'tst_timer_check()' function checks if specified 'clk_id' is suppored and 1298exits the test with 'TCONF' otherwise. It's expected to be used in test 1299'setup()' before any resources that needs to be cleaned up are initialized, 1300hence it does not include a cleanup function parameter. 1301 1302The 'tst_timer_start()' marks start time and stores the 'clk_id' for further 1303use. 1304 1305The 'tst_timer_stop()' marks the stop time using the same 'clk_id' as last 1306call to 'tst_timer_start()'. 1307 1308The 'tst_timer_elapsed*()' returns time difference between the timer start and 1309last timer stop in several formats and units. 1310 1311The 'tst_timer_expired_ms()' function checks if the timer started by 1312'tst_timer_start()' has been running longer than ms milliseconds. The function 1313returns non-zero if timer has expired and zero otherwise. 1314 1315IMPORTANT: The timer functions use 'clock_gettime()' internally which needs to 1316 be linked with '-lrt' on older glibc. Please do not forget to add 1317 'LDLIBS+=-lrt' in Makefile. 1318 1319[source,c] 1320------------------------------------------------------------------------------- 1321#include "tst_test.h" 1322#include "tst_timer.h" 1323 1324static void setup(void) 1325{ 1326 ... 1327 tst_timer_check(CLOCK_MONOTONIC); 1328 ... 1329} 1330 1331static void run(void) 1332{ 1333 ... 1334 tst_timer_start(CLOCK_MONOTONIC); 1335 ... 1336 while (!tst_timer_expired_ms(5000)) { 1337 ... 1338 } 1339 ... 1340} 1341 1342struct tst_test test = { 1343 ... 1344 .setup = setup, 1345 .test_all = run, 1346 ... 1347}; 1348------------------------------------------------------------------------------- 1349 1350Expiration timer example usage. 1351 1352[source,c] 1353------------------------------------------------------------------------------- 1354long long tst_timespec_to_us(struct timespec t); 1355long long tst_timespec_to_ms(struct timespec t); 1356 1357struct timeval tst_us_to_timeval(long long us); 1358struct timeval tst_ms_to_timeval(long long ms); 1359 1360int tst_timespec_lt(struct timespec t1, struct timespec t2); 1361 1362struct timespec tst_timespec_add_us(struct timespec t, long long us); 1363 1364struct timespec tst_timespec_diff(struct timespec t1, struct timespec t2); 1365long long tst_timespec_diff_us(struct timespec t1, struct timespec t2); 1366long long tst_timespec_diff_ms(struct timespec t1, struct timespec t2); 1367 1368struct timespec tst_timespec_abs_diff(struct timespec t1, struct timespec t2); 1369long long tst_timespec_abs_diff_us(struct timespec t1, struct timespec t2); 1370long long tst_timespec_abs_diff_ms(struct timespec t1, struct timespec t2); 1371------------------------------------------------------------------------------- 1372 1373The first four functions are simple inline conversion functions. 1374 1375The 'tst_timespec_lt()' function returns non-zero if 't1' is earlier than 1376't2'. 1377 1378The 'tst_timespec_add_us()' function adds 'us' microseconds to the timespec 1379't'. The 'us' is expected to be positive. 1380 1381The 'tst_timespec_diff*()' functions returns difference between two times, the 1382't1' is expected to be later than 't2'. 1383 1384The 'tst_timespec_abs_diff*()' functions returns absolute value of difference 1385between two times. 1386 1387NOTE: All conversions to ms and us rounds the value. 1388 13892.2.22 Datafiles 1390^^^^^^^^^^^^^^^^ 1391 1392[source,c] 1393------------------------------------------------------------------------------- 1394#include "tst_test.h" 1395 1396static const char *const res_files[] = { 1397 "foo", 1398 "bar", 1399 NULL 1400}; 1401 1402static struct tst_test test = { 1403 ... 1404 .resource_files = res_files, 1405 ... 1406} 1407------------------------------------------------------------------------------- 1408 1409If the test needs additional files to be copied to the test temporary 1410directory all you need to do is to list their filenames in the 1411'NULL'-terminated array '.resource_files' in the tst_test structure. 1412 1413When resource files is set test temporary directory is created automatically, 1414there is need to set '.needs_tmpdir' as well. 1415 1416The test library looks for datafiles first, these are either stored in a 1417directory called +datafiles+ in the +$PWD+ at the start of the test or in 1418+$LTPROOT/testcases/data/${test_binary_name}+. If the file is not found the 1419library looks into +$LTPROOT/testcases/bin/+ and to +$PWD+ at the start of the 1420test. This ensures that the testcases can copy the file(s) effortlessly both 1421when test is started from the directory it was compiled in as well as when LTP 1422was installed. 1423 1424The file(s) are copied to the newly created test temporary directory which is 1425set as the test working directory when the 'test()' functions is executed. 1426 14272.2.23 Code path tracing 1428^^^^^^^^^^^^^^^^^^^^^^^^ 1429 1430'tst_res' is a macro, so on when you define a function in one file: 1431 1432[source,c] 1433------------------------------------------------------------------------------- 1434int do_action(int arg) 1435{ 1436 ... 1437 1438 if (ok) { 1439 tst_res(TPASS, "check passed"); 1440 return 0; 1441 } else { 1442 tst_res(TFAIL, "check failed"); 1443 return -1; 1444 } 1445} 1446------------------------------------------------------------------------------- 1447 1448and call it from another file, the file and line reported by 'tst_res' in this 1449function will be from the former file. 1450 1451'TST_TRACE' can make the analysis of such situations easier. It's a macro which 1452inserts a call to 'tst_res(TINFO, ...)' in case its argument evaluates to 1453non-zero. In this call to 'tst_res(TINFO, ...)' the file and line will be 1454expanded using the actual location of 'TST_TRACE'. 1455 1456For example, if this another file contains: 1457 1458[source,c] 1459------------------------------------------------------------------------------- 1460#include "tst_test.h" 1461 1462if (TST_TRACE(do_action(arg))) { 1463 ... 1464} 1465------------------------------------------------------------------------------- 1466 1467the generated output may look similar to: 1468 1469------------------------------------------------------------------------------- 1470common.h:9: FAIL: check failed 1471test.c:8: INFO: do_action(arg) failed 1472------------------------------------------------------------------------------- 1473 14742.2.24 Tainted kernels 1475^^^^^^^^^^^^^^^^^^^^^^ 1476 1477If you need to detect, if a testcase triggers a kernel warning, bug or oops, 1478the following can be used to detect TAINT_W or TAINT_D: 1479 1480[source,c] 1481------------------------------------------------------------------------------- 1482#include "tst_test.h" 1483#include "tst_taint.h" 1484 1485void setup(void) 1486{ 1487 ... 1488 tst_taint_init(TST_TAINT_W | TST_TAINT_D); 1489 ... 1490} 1491... 1492void run(void) 1493{ 1494 ... 1495 if (tst_taint_check() == 0) 1496 tst_res(TPASS, "kernel is not tainted"); 1497 else 1498 tst_res(TFAIL, "kernel is tainted"); 1499} 1500------------------------------------------------------------------------------- 1501 1502You have to call tst_taint_init() with non-zero flags first, preferably during 1503setup(). The function will generate a TCONF if the requested flags are not 1504fully supported on the running kernel, and TBROK if either a zero mask was 1505supplied or if the kernel is already tainted before executing the test. 1506 1507Then you can call tst_taint_check() during run(), which returns 0 or the 1508tainted flags set in /proc/sys/kernel/tainted as specified earlier. 1509 1510Depending on your kernel version, not all tainted-flags will be supported. 1511 1512For reference to tainted kernels, see kernel documentation: 1513Documentation/admin-guide/tainted-kernels.rst or 1514https://www.kernel.org/doc/html/latest/admin-guide/tainted-kernels.html 1515 15162.2.25 Checksums 1517^^^^^^^^^^^^^^^^ 1518 1519CRC32c checksum generation is supported by LTP. In order to use it, the 1520test should include "tst_checksum.h" header, then can call tst_crc32c(). 1521 15222.2.26 Checking kernel for the driver support 1523^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1524 1525Some tests may need specific kernel drivers, either compiled in, or built 1526as a module. If .need_drivers points to a NULL-terminated array of kernel 1527module names these are all checked and the test exits with TCONF on the 1528first missing driver. 1529 1530Since it relies on modprobe command, the check will be skipped if the command 1531itself is not available on the system. 1532 15332.2.27 Saving & restoring /proc|sys values 1534^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1535 1536LTP library can be instructed to save and restore value of specified 1537(/proc|sys) files. This is achieved by initialized tst_test struct 1538field 'save_restore'. It is a NULL terminated array of strings where 1539each string represents a file, whose value is saved at the beginning 1540and restored at the end of the test. Only first line of a specified 1541file is saved and restored. 1542 1543Pathnames can be optionally prefixed to specify how strictly (during 1544'store') are handled errors: 1545 1546* (no prefix) - test ends with TCONF, if file doesn't exist 1547* '?' - test prints info message and continues, 1548 if file doesn't exist or open/read fails 1549* '!' - test ends with TBROK, if file doesn't exist 1550 1551'restore' is always strict and will TWARN if it encounters any error. 1552 1553[source,c] 1554------------------------------------------------------------------------------- 1555static const char *save_restore[] = { 1556 "/proc/sys/kernel/core_pattern", 1557 NULL, 1558}; 1559 1560static void setup(void) 1561{ 1562 FILE_PRINTF("/proc/sys/kernel/core_pattern", "/mypath"); 1563} 1564 1565static struct tst_test test = { 1566 ... 1567 .setup = setup, 1568 .save_restore = save_restore, 1569}; 1570------------------------------------------------------------------------------- 1571 15722.2.28 Parsing kernel .config 1573^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1574 1575Generally testcases should attempt to autodetect as much kernel features as 1576possible based on the currently running kernel. We do have tst_check_driver() 1577to check if functionality that could be compiled as kernel module is present 1578on the system, disabled syscalls can be detected by checking for 'ENOSYS' 1579errno etc. 1580 1581However in rare cases core kernel features couldn't be detected based on the 1582kernel userspace API and we have to resort to kernel .config parsing. 1583 1584For this cases the test should set the 'NULL' terminated needs_kconfig array 1585of kernel config options required for the test. The config option can be 1586specified either as plain "CONFIG_FOO" in which case it's sufficient for the 1587test continue if it's set to any value (typically =y or =m). Or with a value 1588as "CONFIG_FOO=bar" in which case the value has to match as well. The test is 1589aborted with 'TCONF' if any of the required options were not set. 1590 1591[source,c] 1592------------------------------------------------------------------------------- 1593#include "tst_test.h" 1594 1595static const char *kconfigs[] = { 1596 "CONFIG_X86_INTEL_UMIP", 1597 NULL 1598}; 1599 1600static struct tst_test test = { 1601 ... 1602 .needs_kconfigs = kconfigs, 1603 ... 1604}; 1605------------------------------------------------------------------------------- 1606 16072.2.29 Changing the Wall Clock Time during test execution 1608^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1609 1610There are some tests that, for different reasons, might need to change the 1611system-wide clock time. Whenever this happens, it is imperative that the clock 1612is restored, at the end of test's execution, taking in consideration the amount 1613of time elapsed during that test. 1614 1615In order for that to happen, struct tst_test has a variable called 1616"restore_wallclock" that should be set to "1" so LTP knows it should: (1) 1617initialize a monotonic clock during test setup phase and (2) use that monotonic 1618clock to fix the system-wide clock time at the test cleanup phase. 1619 1620[source,c] 1621------------------------------------------------------------------------------- 1622#include "tst_test.h" 1623 1624static void setup(void) 1625{ 1626 ... 1627} 1628 1629static void run(void) 1630{ 1631 ... 1632} 1633 1634struct tst_test test = { 1635 ... 1636 .setup = setup, 1637 .test_all = run, 1638 .restore_wallclock = 1, 1639 ... 1640}; 1641------------------------------------------------------------------------------- 1642 16432.2.30 Testing similar syscalls in one test 1644^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1645 1646In some cases kernel has several very similar syscalls that do either the same 1647or very similar job. This is most noticeable on i386 where we commonly have 1648two or three syscall versions. That is because i386 was first platform that 1649Linux was developed on and because of that most mistakes in API happened there 1650as well. However this is not limited to i386 at all, it's quite common that 1651version two syscall has added missing flags parameters or so. 1652 1653In such cases it does not make much sense to copy&paste the test code over and 1654over, rather than that the test library provides support for test variants. 1655The idea behind test variants is simple, we run the test several times each 1656time with different syscall variant. 1657 1658The implementation consist of test_variant integer that, if set, denotes number 1659of test variants. The test is then forked and executed test_variant times each 1660time with different value in global tst_variant variable. 1661 1662[source,c] 1663------------------------------------------------------------------------------- 1664#include "tst_test.h" 1665 1666static int do_foo(void) 1667{ 1668 switch (tst_variant) { 1669 case 0: 1670 return foo(); 1671 case 1: 1672 return syscall(__NR_foo); 1673 } 1674 1675 return -1; 1676} 1677 1678static void run(void) 1679{ 1680 ... 1681 1682 TEST(do_foo); 1683 1684 ... 1685} 1686 1687static void setup(void) 1688{ 1689 switch (tst_variant) { 1690 case 0: 1691 tst_res(TINFO, "Testing foo variant 1"); 1692 break; 1693 case 1: 1694 tst_res(TINFO, "Testing foo variant 2"); 1695 break; 1696 } 1697} 1698 1699struct tst_test test = { 1700 ... 1701 .setup = setup, 1702 .test_all = run, 1703 .test_variants = 2, 1704 ... 1705}; 1706------------------------------------------------------------------------------- 1707 17082.2.31 Guarded buffers 1709^^^^^^^^^^^^^^^^^^^^^^ 1710 1711The test library supports guarded buffers, which are buffers allocated so 1712that: 1713 1714* The end of the buffer is followed by a PROT_NONE page 1715 1716* The remainder of the page before the buffer is filled with random canary 1717 data 1718 1719Which means that the any access after the buffer will yield a Segmentation 1720fault or EFAULT depending on if the access happened in userspace or the kernel 1721respectively. The canary before the buffer will also catch any write access 1722outside of the buffer. 1723 1724The purpose of the patch is to catch off-by-one bugs which happens when 1725buffers and structures are passed to syscalls. New tests should allocate 1726guarded buffers for all data passed to the tested syscall which are passed by 1727a pointer. 1728 1729[source,c] 1730------------------------------------------------------------------------------- 1731#include "tst_test.h" 1732 1733static struct foo *foo_ptr; 1734static struct iovec *iov; 1735static void *buf_ptr; 1736static char *id; 1737... 1738 1739static void run(void) 1740{ 1741 ... 1742 1743 foo_ptr->bar = 1; 1744 foo_ptr->buf = buf_ptr; 1745 1746 ... 1747} 1748 1749static void setup(void) 1750{ 1751 ... 1752 1753 id = tst_strdup(string); 1754 1755 ... 1756} 1757 1758static struct tst_test test = { 1759 ... 1760 .bufs = (struct tst_buffers []) { 1761 {&foo_ptr, .size = sizeof(*foo_ptr)}, 1762 {&buf_ptr, .size = BUF_SIZE}, 1763 {&iov, .iov_sizes = (int[]){128, 32, -1}, 1764 {} 1765 } 1766}; 1767------------------------------------------------------------------------------- 1768 1769Guarded buffers can be allocated on runtime in a test setup() by a 1770'tst_alloc()' or by 'tst_strdup()' as well as by filling up the .bufs array in 1771the tst_test structure. 1772 1773So far the tst_test structure supports allocating either a plain buffer by 1774setting up the size or struct iovec, which is allocated recursively including 1775the individual buffers as described by an '-1' terminated array of buffer 1776sizes. 1777 17782.2.32 Adding and removing capabilities 1779^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1780 1781Some tests may require the presence or absence of particular 1782capabilities. Using the API provided by 'tst_capability.h' the test author can 1783try to ensure that some capabilities are either present or absent during the 1784test. 1785 1786For example; below we try to create a raw socket, which requires 1787CAP_NET_ADMIN. During setup we should be able to do it, then during run it 1788should be impossible. The LTP capability library will check before setup that 1789we have this capability, then after setup it will drop it. 1790 1791[source,c] 1792-------------------------------------------------------------------------------- 1793#include "tst_test.h" 1794#include "tst_capability.h" 1795#include "tst_safe_net.h" 1796 1797#include "lapi/socket.h" 1798 1799static void run(void) 1800{ 1801 TEST(socket(AF_INET, SOCK_RAW, 1)); 1802 if (TST_RET > -1) { 1803 tst_res(TFAIL, "Created raw socket"); 1804 } else if (TST_ERR != EPERM) { 1805 tst_res(TBROK | TTERRNO, 1806 "Failed to create socket for wrong reason"); 1807 } else { 1808 tst_res(TPASS | TTERRNO, "Didn't create raw socket"); 1809 } 1810} 1811 1812static void setup(void) 1813{ 1814 TEST(socket(AF_INET, SOCK_RAW, 1)); 1815 if (TST_RET < 0) 1816 tst_brk(TCONF | TTERRNO, "We don't have CAP_NET_RAW to begin with"); 1817 1818 SAFE_CLOSE(TST_RET); 1819} 1820 1821static struct tst_test test = { 1822 .setup = setup, 1823 .test_all = run, 1824 .caps = (struct tst_cap []) { 1825 TST_CAP(TST_CAP_REQ, CAP_NET_RAW), 1826 TST_CAP(TST_CAP_DROP, CAP_NET_RAW), 1827 {} 1828 }, 1829}; 1830-------------------------------------------------------------------------------- 1831 1832Look at the test struct at the bottom. We have filled in the 'caps' field with 1833a NULL terminated array containing two 'tst_cap' structs. 'TST_CAP_REQ' 1834actions are executed before setup and 'TST_CAP_DROP' are executed after 1835setup. This means it is possible to both request and drop a capability. 1836 1837[source,c] 1838-------------------------------------------------------------------------------- 1839static struct tst_test test = { 1840 .test_all = run, 1841 .caps = (struct tst_cap []) { 1842 TST_CAP(TST_CAP_REQ, CAP_NET_RAW), 1843 TST_CAP(TST_CAP_DROP, CAP_SYS_ADMIN), 1844 {} 1845 }, 1846}; 1847-------------------------------------------------------------------------------- 1848 1849Here we request 'CAP_NET_RAW', but drop 'CAP_SYS_ADMIN'. If the capability is 1850in the permitted set, but not the effective set, the library will try to 1851permit it. If it is not in the permitted set, then it will fail with 'TCONF'. 1852 1853This API does not require 'libcap' to be installed. However it has limited 1854features relative to 'libcap'. It only tries to add or remove capabilities 1855from the effective set. This means that tests which need to spawn child 1856processes may have difficulties ensuring the correct capabilities are 1857available to the children (see the capabilities (7) manual pages). 1858 1859However a lot of problems can be solved by using 'tst_cap_action(struct 1860tst_cap *cap)' directly which can be called at any time. This also helps if 1861you wish to drop a capability at the begining of setup. 1862 18632.2.33 Reproducing race-conditions 1864^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1865 1866If a bug is caused by two tasks in the kernel racing and you wish to create a 1867regression test (or bug-fix validation test) then the 'tst_fuzzy_sync.h' 1868library should be used. 1869 1870It allows you to specify, in your code, two race windows. One window in each 1871thread's loop (triggering a race usually requires many iterations). These 1872windows show fuzzy-sync where the race can happen. They don't need to be 1873exact, hence the 'fuzzy' part. If the race condition is not immediately 1874triggered then the library will begin experimenting with different timings. 1875 1876[source,c] 1877-------------------------------------------------------------------------------- 1878#include "tst_fuzzy_sync.h" 1879 1880static struct tst_fzsync_pair fzsync_pair; 1881 1882static void setup(void) 1883{ 1884 tst_fzsync_pair_init(&fzsync_pair); 1885} 1886 1887static void cleanup(void) 1888{ 1889 tst_fzsync_pair_cleanup(&fzsync_pair); 1890} 1891 1892static void *thread_b(void *arg) 1893{ 1894 while (tst_fzsync_run_b(&fzsync_pair)) { 1895 1896 tst_fzsync_start_race_b(&fzsync_pair); 1897 1898 /* This is the race window for thread B */ 1899 1900 tst_fzsync_end_race_b(&fzsync_pair); 1901 } 1902 1903 return arg; 1904} 1905 1906static void thread_a(void) 1907{ 1908 tst_fzsync_pair_reset(&fzsync_pair, thread_b); 1909 1910 while (tst_fzsync_run_a(&fzsync_pair)) { 1911 1912 tst_fzsync_start_race_a(&fzsync_pair); 1913 1914 /* This is the race window for thread A */ 1915 1916 tst_fzsync_end_race_a(&fzsync_pair); 1917 } 1918} 1919 1920static struct tst_test test = { 1921 .test_all = thread_a, 1922 .setup = setup, 1923 .cleanup = cleanup, 1924}; 1925-------------------------------------------------------------------------------- 1926 1927Above is a minimal template for a test using fuzzy-sync. In a simple case, you 1928just need to put the bits you want to race inbetween 'start_race' and 1929'end_race'. Meanwhile, any setup you need to do per-iteration goes outside the 1930windows. 1931 1932Fuzzy sync synchronises 'run_a' and 'run_b', which act as barriers, so that 1933neither thread can progress until the other has caught up with it. There is 1934also the 'pair_wait' function which can be used to add barriers in other 1935locations. Of course 'start/end_race_a/b' are also a barriers. 1936 1937The library decides how long the test should run for based on the timeout 1938specified by the user plus some other heuristics. 1939 1940For full documentation see the comments in 'include/tst_fuzzy_sync.h'. 1941 19422.3 Writing a testcase in shell 1943~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1944 1945LTP supports testcases to be written in a portable shell too. 1946 1947There is a shell library modeled closely to the C interface at 1948'testcases/lib/tst_test.sh'. 1949 1950WARNING: All identifiers starting with TST_ or tst_ are reserved for the 1951 test library. 1952 19532.3.1 Basic test interface 1954^^^^^^^^^^^^^^^^^^^^^^^^^^ 1955 1956[source,sh] 1957------------------------------------------------------------------------------- 1958#!/bin/sh 1959# SPDX-License-Identifier: GPL-2.0-or-later 1960# This is a basic test for true shell builtin 1961 1962TST_TESTFUNC=do_test 1963. tst_test.sh 1964 1965do_test() 1966{ 1967 true 1968 ret=$? 1969 1970 if [ $ret -eq 0 ]; then 1971 tst_res TPASS "true returned 0" 1972 else 1973 tst_res TFAIL "true returned $ret" 1974 fi 1975} 1976 1977tst_run 1978------------------------------------------------------------------------------- 1979 1980TIP: To execute this test the 'tst_test.sh' library must be in '$PATH'. If you 1981 are executing the test from a git checkout you can run it as 1982 'PATH="$PATH:../../lib" ./foo01.sh' 1983 1984The shell library expects test setup, cleanup and the test function executing 1985the test in the '$TST_SETUP', '$TST_CLEANUP' and '$TST_TESTFUNC' variables. 1986 1987Both '$TST_SETUP' and '$TST_CLEANUP' are optional. 1988 1989The '$TST_TESTFUNC' may be called several times if more than one test 1990iteration was requested by passing right command line options to the test. 1991 1992The '$TST_CLEANUP' may be called even in the middle of the setup and must be 1993able to clean up correctly even in this situation. The easiest solution for 1994this is to keep track of what was initialized and act accordingly in the 1995cleanup. 1996 1997WARNING: Similar to the C library, calling tst_brk() in the $TST_CLEANUP does 1998 not exit the test and 'TBROK' is converted to 'TWARN'. 1999 2000Notice also the 'tst_run' function called at the end of the test that actually 2001starts the test. 2002 2003[source,sh] 2004------------------------------------------------------------------------------- 2005#!/bin/sh 2006# SPDX-License-Identifier: GPL-2.0-or-later 2007# Example test with tests in separate functions 2008 2009TST_TESTFUNC=test 2010TST_CNT=2 2011. tst_test.sh 2012 2013test1() 2014{ 2015 tst_res TPASS "Test $1 passed" 2016} 2017 2018test2() 2019{ 2020 tst_res TPASS "Test $1 passed" 2021} 2022 2023tst_run 2024# output: 2025# foo 1 TPASS: Test 1 passed 2026# foo 2 TPASS: Test 2 passed 2027------------------------------------------------------------------------------- 2028 2029If '$TST_CNT' is set, the test library looks if there are functions named 2030'$\{TST_TESTFUNC\}1', ..., '$\{TST_TESTFUNC\}$\{TST_CNT\}' and if these are 2031found they are executed one by one. The test number is passed to it in the '$1'. 2032 2033[source,sh] 2034------------------------------------------------------------------------------- 2035#!/bin/sh 2036# SPDX-License-Identifier: GPL-2.0-or-later 2037# Example test with tests in a single function 2038 2039TST_TESTFUNC=do_test 2040TST_CNT=2 2041. tst_test.sh 2042 2043do_test() 2044{ 2045 case $1 in 2046 1) tst_res TPASS "Test $1 passed";; 2047 2) tst_res TPASS "Test $1 passed";; 2048 esac 2049} 2050 2051tst_run 2052# output: 2053# foo 1 TPASS: Test 1 passed 2054# foo 2 TPASS: Test 2 passed 2055------------------------------------------------------------------------------- 2056 2057Otherwise, if '$TST_CNT' is set but there is no '$\{TST_TESTFUNC\}1', etc., 2058the '$TST_TESTFUNC' is executed '$TST_CNT' times and the test number is passed 2059to it in the '$1'. 2060 2061[source,sh] 2062------------------------------------------------------------------------------- 2063#!/bin/sh 2064# SPDX-License-Identifier: GPL-2.0-or-later 2065# Example test with tests in a single function, using $TST_TEST_DATA and 2066# $TST_TEST_DATA_IFS 2067 2068TST_TESTFUNC=do_test 2069TST_TEST_DATA="foo:bar:d dd" 2070TST_TEST_DATA_IFS=":" 2071. tst_test.sh 2072 2073do_test() 2074{ 2075 tst_res TPASS "Test $1 passed with data '$2'" 2076} 2077 2078tst_run 2079# output: 2080# foo 1 TPASS: Test 1 passed with data 'foo' 2081# foo 2 TPASS: Test 1 passed with data 'bar' 2082# foo 3 TPASS: Test 1 passed with data 'd dd' 2083------------------------------------------------------------------------------- 2084 2085It's possible to pass data for function with '$TST_TEST_DATA'. Optional 2086'$TST_TEST_DATA_IFS' is used for splitting, default value is space. 2087 2088[source,sh] 2089------------------------------------------------------------------------------- 2090#!/bin/sh 2091# SPDX-License-Identifier: GPL-2.0-or-later 2092# Example test with tests in a single function, using $TST_TEST_DATA and $TST_CNT 2093 2094TST_TESTFUNC=do_test 2095TST_CNT=2 2096TST_TEST_DATA="foo bar" 2097. tst_test.sh 2098 2099do_test() 2100{ 2101 case $1 in 2102 1) tst_res TPASS "Test $1 passed with data '$2'";; 2103 2) tst_res TPASS "Test $1 passed with data '$2'";; 2104 esac 2105} 2106 2107tst_run 2108# output: 2109# foo 1 TPASS: Test 1 passed with data 'foo' 2110# foo 2 TPASS: Test 2 passed with data 'foo' 2111# foo 3 TPASS: Test 1 passed with data 'bar' 2112# foo 4 TPASS: Test 2 passed with data 'bar' 2113------------------------------------------------------------------------------- 2114 2115'$TST_TEST_DATA' can be used with '$TST_CNT'. If '$TST_TEST_DATA_IFS' not specified, 2116space as default value is used. Of course, it's possible to use separate functions. 2117 21182.3.2 Library environment variables for shell 2119^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2120 2121Similarily to the C library various checks and preparations can be requested 2122simply by setting right '$TST_NEEDS_FOO'. 2123 2124[options="header"] 2125|============================================================================= 2126| Variable name | Action done 2127| 'TST_NEEDS_ROOT' | Exit the test with 'TCONF' unless executed under root 2128| 'TST_NEEDS_TMPDIR' | Create test temporary directory and cd into it. 2129| 'TST_NEEDS_DEVICE' | Prepare test temporary device, the path to testing 2130 device is stored in '$TST_DEVICE' variable. 2131 The option implies 'TST_NEEDS_TMPDIR'. 2132| 'TST_NEEDS_CMDS' | String with command names that has to be present for 2133 the test (see below). 2134| 'TST_NEEDS_MODULE' | Test module name needed for the test (see below). 2135| 'TST_NEEDS_DRIVERS'| Checks kernel drivers support for the test. 2136| 'TST_TIMEOUT' | Maximum timeout set for the test in sec. Must be int >= 1, 2137 or -1 (special value to disable timeout), default is 300. 2138 Variable is meant be set in tests, not by user. 2139 It's equivalent of `tst_test.timeout` in C. 2140|============================================================================= 2141 2142NOTE: Network tests (see testcases/network/README.md) use additional variables 2143in 'tst_net.sh'. 2144 2145Checking for presence of commands 2146+++++++++++++++++++++++++++++++++ 2147 2148[source,sh] 2149------------------------------------------------------------------------------- 2150#!/bin/sh 2151 2152... 2153 2154TST_NEEDS_CMDS="modinfo modprobe" 2155. tst_test.sh 2156 2157... 2158 2159------------------------------------------------------------------------------- 2160 2161Setting '$TST_NEEDS_CMDS' to a string listing required commands will check for 2162existence each of them and exits the test with 'TCONF' on first missing. 2163 2164Alternatively the 'tst_require_cmds()' function can be used to do the same on 2165runtime, since sometimes we need to the check at runtime too. 2166 2167'tst_check_cmds()' can be used for requirements just for a particular test 2168as it doesn't exit (it issues 'tst_res TCONF'). Expected usage is: 2169... 2170 2171TST_TESTFUNC=do_test 2172. tst_test.sh 2173 2174do_test() 2175{ 2176 tst_check_cmds cmd || return 2177 cmd --foo 2178 ... 2179} 2180 2181tst_run 2182... 2183 2184Locating kernel modules 2185+++++++++++++++++++++++ 2186 2187The LTP build system can build kernel modules as well, setting 2188'$TST_NEEDS_MODULE' to module name will cause the library to look for the 2189module in a few possible paths. 2190 2191If module was found the path to it will be stored into '$TST_MODPATH' 2192variable, if module wasn't found the test will exit with 'TCONF'. 2193 21942.3.3 Optional command line parameters 2195^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2196 2197[source,sh] 2198------------------------------------------------------------------------------- 2199#!/bin/sh 2200# SPDX-License-Identifier: GPL-2.0-or-later 2201# Optional test command line parameters 2202 2203TST_OPTS="af:" 2204TST_USAGE=usage 2205TST_PARSE_ARGS=parse_args 2206TST_TESTFUNC=do_test 2207 2208. tst_test.sh 2209 2210ALTERNATIVE=0 2211MODE="foo" 2212 2213usage() 2214{ 2215 cat << EOF 2216usage: $0 [-a] [-f <foo|bar>] 2217 2218OPTIONS 2219-a Enable support for alternative foo 2220-f Specify foo or bar mode 2221EOF 2222} 2223 2224parse_args() 2225{ 2226 case $1 in 2227 a) ALTERNATIVE=1;; 2228 f) MODE="$2";; 2229 esac 2230} 2231 2232do_test() 2233{ 2234 ... 2235} 2236 2237tst_run 2238------------------------------------------------------------------------------- 2239 2240The 'getopts' string for optional parameters is passed in the '$TST_OPTS' 2241variable. There are a few default parameters that cannot be used by a test, 2242these can be listed with passing help '-h' option to any test. 2243 2244The function that prints the usage is passed in '$TST_USAGE', the help for 2245the options implemented in the library is appended when usage is printed. 2246 2247Lastly the function '$PARSE_ARGS' is called with the option name in the '$1' 2248and, if option has argument, its value in the '$2'. 2249 2250[source,sh] 2251------------------------------------------------------------------------------- 2252#!/bin/sh 2253# SPDX-License-Identifier: GPL-2.0-or-later 2254# Optional test positional parameters 2255 2256TST_POS_ARGS=3 2257TST_USAGE=usage 2258TST_TESTFUNC=do_test 2259 2260. tst_test.sh 2261 2262usage() 2263{ 2264 cat << EOF 2265usage: $0 [min] [max] [size] 2266 2267EOF 2268} 2269 2270min="$1" 2271max="$2" 2272size="$3" 2273 2274do_test() 2275{ 2276 ... 2277} 2278 2279tst_run 2280------------------------------------------------------------------------------- 2281 2282You can also request a number of positional parameters by setting the 2283'$TST_POS_ARGS' variable. If you do, these will be available as they were 2284passed directly to the script in '$1', '$2', ..., '$n'. 2285 22862.3.4 Useful library functions 2287^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2288 2289Retrieving configuration variables 2290++++++++++++++++++++++++++++++++++ 2291 2292You may need to retrieve configuration values such as PAGESIZE, there is 2293'getconf' but as some system may not have it, you are advised to use 2294'tst_getconf' instead. Note that it implements subset of 'getconf' 2295system variables used by the testcases only. 2296 2297[source,sh] 2298------------------------------------------------------------------------------- 2299# retrieve PAGESIZE 2300pagesize=`tst_getconf PAGESIZE` 2301------------------------------------------------------------------------------- 2302 2303Sleeping for subsecond intervals 2304++++++++++++++++++++++++++++++++ 2305 2306Albeit there is a sleep command available basically everywhere not all 2307implementations can support sleeping for less than one second. And most of the 2308time sleeping for a second is too much. Therefore LTP includes 'tst_sleep' 2309that can sleep for defined amount of seconds, milliseconds or microseconds. 2310 2311[source,sh] 2312------------------------------------------------------------------------------- 2313# sleep for 100 milliseconds 2314tst_sleep 100ms 2315------------------------------------------------------------------------------- 2316 2317Retry a function in limited time 2318++++++++++++++++++++++++++++++++ 2319 2320Sometimes LTP test needs retrying a function for many times to get success. 2321This achievement makes that possible via keeping it retrying if the return 2322value of the function is NOT as we expected. After exceeding a limited time, 2323test will break from the retries immediately. The time limit is multiplied 2324with LTP_TIMEOUT_MUL. 2325 2326[source,c] 2327------------------------------------------------------------------------------- 2328# retry function in 1 second 2329TST_RETRY_FUNC(FUNC, EXPECTED_RET) 2330 2331# retry function in N second 2332TST_RETRY_FN_EXP_BACKOFF(FUNC, EXPECTED_RET, N) 2333------------------------------------------------------------------------------- 2334 2335[source,sh] 2336------------------------------------------------------------------------------- 2337# retry function in 1 second 2338TST_RETRY_FUNC "FUNC arg1 arg2 ..." "EXPECTED_RET" 2339 2340# retry function in N second 2341TST_RETRY_FN_EXP_BACKOFF "FUNC arg1 arg2 ..." "EXPECTED_RET" "N" 2342------------------------------------------------------------------------------- 2343 2344Checking for integers 2345+++++++++++++++++++++ 2346 2347[source,sh] 2348------------------------------------------------------------------------------- 2349# returns zero if passed an integer parameter, non-zero otherwise 2350tst_is_int "$FOO" 2351------------------------------------------------------------------------------- 2352 2353Checking for integers and floating point numbers 2354++++++++++++++++++++++++++++++++++++++++++++++++ 2355 2356[source,sh] 2357------------------------------------------------------------------------------- 2358# returns zero if passed an integer or floating point number parameter, 2359# non-zero otherwise 2360tst_is_num "$FOO" 2361------------------------------------------------------------------------------- 2362 2363Obtaining random numbers 2364++++++++++++++++++++++++ 2365 2366There is no '$RANDOM' in portable shell, use 'tst_random' instead. 2367 2368[source,sh] 2369------------------------------------------------------------------------------- 2370# get random integer between 0 and 1000 (including 0 and 1000) 2371tst_random 0 1000 2372------------------------------------------------------------------------------- 2373 2374Formatting device with a filesystem 2375+++++++++++++++++++++++++++++++++++ 2376 2377The 'tst_mkfs' helper will format device with the filesystem. 2378 2379[source,sh] 2380------------------------------------------------------------------------------- 2381# format test device with ext2 2382tst_mkfs ext2 $TST_DEVICE 2383# default params are $TST_FS_TYPE $TST_DEVICE 2384tst_mkfs 2385# optional parameters 2386tst_mkfs ext4 /dev/device -T largefile 2387------------------------------------------------------------------------------- 2388 2389Mounting and unmounting filesystems 2390+++++++++++++++++++++++++++++++++++ 2391 2392The 'tst_mount' and 'tst_umount' helpers are a safe way to mount/umount 2393a filesystem. 2394 2395The 'tst_mount' mounts '$TST_DEVICE' of '$TST_FS_TYPE' (optional) to 2396'$TST_MNTPOINT' (defaults to mntpoint), optionally using the 2397'$TST_MNT_PARAMS'. The '$TST_MNTPOINT' directory is created if it didn't 2398exist prior to the function call. 2399 2400If the path passed (optional, defaults to '$TST_DEVICE') to the 'tst_umount' is 2401not mounted (present in '/proc/mounts') it's noop. 2402Otherwise it retries to umount the filesystem a few times on a failure, which 2403is a workaround since there are a daemons dumb enough to probe all newly 2404mounted filesystems, which prevents them from umounting shortly after they 2405were mounted. 2406 2407ROD and ROD_SILENT 2408++++++++++++++++++ 2409 2410These functions supply the 'SAFE_MACROS' used in C although they work and are 2411named differently. 2412 2413[source,sh] 2414------------------------------------------------------------------------------- 2415ROD_SILENT command arg1 arg2 ... 2416 2417# is shorthand for: 2418 2419command arg1 arg2 ... > /dev/null 2>&1 2420if [ $? -ne 0 ]; then 2421 tst_brkm TBROK "..." 2422fi 2423 2424 2425ROD command arg1 arg2 ... 2426 2427# is shorthand for: 2428 2429ROD arg1 arg2 ... 2430if [ $? -ne 0 ]; then 2431 tst_brkm TBROK "..." 2432fi 2433------------------------------------------------------------------------------- 2434 2435WARNING: Keep in mind that output redirection (to a file) happens in the 2436 caller rather than in the ROD function and cannot be checked for 2437 write errors by the ROD function. 2438 2439As a matter of a fact doing +ROD echo a > /proc/cpuinfo+ would work just fine 2440since the 'ROD' function will only get the +echo a+ part that will run just 2441fine. 2442 2443[source,sh] 2444------------------------------------------------------------------------------- 2445# Redirect output to a file with ROD 2446ROD echo foo \> bar 2447------------------------------------------------------------------------------- 2448 2449Note the '>' is escaped with '\', this causes that the '>' and filename are 2450passed to the 'ROD' function as parameters and the 'ROD' function contains 2451code to split '$@' on '>' and redirects the output to the file. 2452 2453EXPECT_PASS{,_BRK} and EXPECT_FAIL{,_BRK} 2454+++++++++++++++++++++++++++++++++++++++++ 2455 2456[source,sh] 2457------------------------------------------------------------------------------- 2458EXPECT_PASS command arg1 arg2 ... [ \> file ] 2459EXPECT_FAIL command arg1 arg2 ... [ \> file ] 2460------------------------------------------------------------------------------- 2461 2462'EXPECT_PASS' calls 'tst_resm TPASS' if the command exited with 0 exit code, 2463and 'tst_resm TFAIL' otherwise. 'EXPECT_FAIL' does vice versa. 2464 2465Output redirection rules are the same as for the 'ROD' function. In addition 2466to that, 'EXPECT_FAIL' always redirects the command's stderr to '/dev/null'. 2467 2468There are also 'EXPECT_PASS_BRK' and 'EXPECT_FAIL_BRK', which works the same way 2469except breaking a test when unexpected action happen. 2470 2471It's possible to detect whether expected value happened: 2472[source,sh] 2473------------------------------------------------------------------------------- 2474if ! EXPECT_PASS command arg1 2\> /dev/null; then 2475 continue 2476fi 2477------------------------------------------------------------------------------- 2478 2479tst_kvcmp 2480+++++++++ 2481 2482This command compares the currently running kernel version given conditions 2483with syntax similar to the shell test command. 2484 2485[source,sh] 2486------------------------------------------------------------------------------- 2487# Exit the test if kernel version is older or equal to 2.6.8 2488if tst_kvcmp -le 2.6.8; then 2489 tst_brk TCONF "Kernel newer than 2.6.8 is needed" 2490fi 2491 2492# Exit the test if kernel is newer than 3.8 and older than 4.0.1 2493if tst_kvcmp -gt 3.8 -a -lt 4.0.1; then 2494 tst_brk TCONF "Kernel must be older than 3.8 or newer than 4.0.1" 2495fi 2496------------------------------------------------------------------------------- 2497 2498[options="header"] 2499|======================================================================= 2500| expression | description 2501| -eq kver | Returns true if kernel version is equal 2502| -ne kver | Returns true if kernel version is not equal 2503| -gt kver | Returns true if kernel version is greater 2504| -ge kver | Returns true if kernel version is greater or equal 2505| -lt kver | Returns true if kernel version is lesser 2506| -le kver | Returns true if kernel version is lesser or equal 2507| -a | Does logical and between two expressions 2508| -o | Does logical or between two expressions 2509|======================================================================= 2510 2511The format for kernel version has to either be with one dot e.g. '2.6' or with 2512two dots e.g. '4.8.1'. 2513 2514.tst_fs_has_free 2515[source,sh] 2516------------------------------------------------------------------------------- 2517#!/bin/sh 2518 2519... 2520 2521# whether current directory has 100MB free space at least. 2522if ! tst_fs_has_free . 100MB; then 2523 tst_brkm TCONF "Not enough free space" 2524fi 2525 2526... 2527------------------------------------------------------------------------------- 2528 2529The 'tst_fs_has_free' shell interface returns 0 if the specified free space is 2530satisfied, 1 if not, and 2 on error. 2531 2532The second argument supports suffixes kB, MB and GB, the default unit is Byte. 2533 2534.tst_retry 2535[source,sh] 2536------------------------------------------------------------------------------- 2537#!/bin/sh 2538 2539... 2540 2541# Retry ping command three times 2542tst_retry "ping -c 1 127.0.0.1" 2543 2544if [ $? -ne 0 ]; then 2545 tst_resm TFAIL "Failed to ping 127.0.0.1" 2546else 2547 tst_resm TPASS "Successfully pinged 127.0.0.1" 2548fi 2549 2550... 2551------------------------------------------------------------------------------- 2552 2553The 'tst_retry' function allows you to retry a command after waiting small 2554amount of time until it succeeds or until given amount of retries has been 2555reached (default is three attempts). 2556 25572.3.5 Restarting daemons 2558^^^^^^^^^^^^^^^^^^^^^^^^ 2559 2560Restarting system daemons is a complicated task for two reasons. 2561 2562* There are different init systems 2563 (SysV init, systemd, etc...) 2564 2565* Daemon names are not unified between distributions 2566 (apache vs httpd, cron vs crond, various syslog variations) 2567 2568To solve these problems LTP has 'testcases/lib/daemonlib.sh' library that 2569provides functions to start/stop/query daemons as well as variables that store 2570correct daemon name. 2571 2572.Supported operations 2573|============================================================================== 2574| start_daemon() | Starts daemon, name is passed as first parameter. 2575| stop_daemon() | Stops daemon, name is passed as first parameter. 2576| restart_daemon() | Restarts daemon, name is passed as first parameter. 2577| status_daemon() | Detect daemon status (exit code: 0: running, 1: not running). 2578|============================================================================== 2579 2580.Variables with detected names 2581|============================================================================== 2582| CROND_DAEMON | Cron daemon name (cron, crond). 2583| SYSLOG_DAEMON | Syslog daemon name (syslog, syslog-ng, rsyslog). 2584|============================================================================== 2585 2586.Cron daemon restart example 2587[source,sh] 2588------------------------------------------------------------------------------- 2589#!/bin/sh 2590# SPDX-License-Identifier: GPL-2.0-or-later 2591# Cron daemon restart example 2592 2593TCID=cron01 2594TST_COUNT=1 2595. test.sh 2596. daemonlib.sh 2597 2598... 2599 2600restart_daemon $CROND_DAEMON 2601 2602... 2603 2604tst_exit 2605------------------------------------------------------------------------------- 2606 26072.3.6 Access to the checkpoint interface 2608~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2609 2610The shell library provides an implementation of the checkpoint interface 2611compatible with the C version. All 'TST_CHECKPOINT_*' functions are available. 2612 2613In order to initialize checkpoints '$TST_NEEDS_CHECKPOINTS' must be set to '1' 2614before the inclusion of 'test.sh': 2615 2616[source,sh] 2617------------------------------------------------------------------------------- 2618#!/bin/sh 2619 2620TST_NEEDS_CHECKPOINTS=1 2621. test.sh 2622------------------------------------------------------------------------------- 2623 2624Since both the implementations are compatible, it's also possible to start 2625a child binary process from a shell test and synchronize with it. This process 2626must have checkpoints initialized by calling 'tst_reinit()'. 2627 26283. Common problems 2629------------------ 2630 2631This chapter describes common problems/misuses and less obvious design patters 2632(quirks) in UNIX interfaces. Read it carefully :) 2633 26343.1 umask() 2635~~~~~~~~~~~ 2636 2637I've been hit by this one several times already... When you create files 2638with 'open()' or 'creat()' etc, the mode specified as the last parameter *is 2639not* the mode the file is created with. The mode depends on current 'umask()' 2640settings which may clear some of the bits. If your test depends on specific 2641file permissions you need either to change umask to 0 or 'chmod()' the file 2642afterwards or use 'SAFE_TOUCH()' that does the 'chmod()' for you. 2643 26443.2 access() 2645~~~~~~~~~~~ 2646 2647If 'access(some_file, W_OK)' is executed by root, it will return success even 2648if the file doesn't have write permission bits set (the same holds for R_OK 2649too). For sysfs files you can use 'open()' as a workaround to check file 2650read/write permissions. It might not work for other filesystems, for these you 2651have to use 'stat()', 'lstat()' or 'fstat()'. 2652 26533.3 umount() EBUSY 2654~~~~~~~~~~~~~~~~~~ 2655 2656Various desktop daemons (gvfsd-trash is known for that) may be stupid enough 2657to probe all newly mounted filesystem which results in 'umount(2)' failing 2658with 'EBUSY'; use 'tst_umount()' described in 2.2.19 that retries in this case 2659instead of plain 'umount(2)'. 2660 26613.4 FILE buffers and fork() 2662~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2663 2664Be vary that if a process calls 'fork(2)' the child process inherits open 2665descriptors as well as copy of the parent memory so especially if there are 2666any open 'FILE' buffers with a data in them they may be written both by the 2667parent and children resulting in corrupted/duplicated data in the resulting 2668files. 2669 2670Also open 'FILE' streams are flushed and closed at 'exit(3)' so if your 2671program works with 'FILE' streams, does 'fork(2)', and the child may end up 2672calling 'exit(3)' you will likely end up with corrupted files. 2673 2674The solution to this problem is either simply call 'fflush(NULL)' that flushes 2675all open output 'FILE' streams just before doing 'fork(2)'. You may also use 2676'_exit(2)' in child processes which does not flush 'FILE' buffers and also 2677skips 'atexit(3)' callbacks. 2678 26794. Test Contribution Checklist 2680------------------------------ 2681 26821. Test compiles and runs fine (check with -i 10 too) 26832. Checkpatch does not report any errors 26843. The runtest entires are in place 26854. Test files are added into corresponding .gitignore files 26865. Patches apply over the latest git 2687 2688 26894.1 About .gitignore files 2690~~~~~~~~~~~~~~~~~~~~~~~~~~ 2691 2692There are numerous '.gitignore' files in the LTP tree. Usually there is a 2693'.gitignore' file per a group of tests. The reason for this setup is simple. 2694It's easier to maintain a '.gitignore' file per directory with tests, rather 2695than having single file in the project root directory. This way, we don't have 2696to update all the gitignore files when moving directories, and they get deleted 2697automatically when a directory with tests is removed. 2698