1LTP C Test API 2============== 3 4NOTE: See also 5 https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines[Test Writing Guidelines], 6 https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial[C Test Case Tutorial], 7 https://github.com/linux-test-project/ltp/wiki/Shell-Test-API[Shell Test API]. 8 91 Writing a test in C 10--------------------- 11 121.1 Basic test structure 13~~~~~~~~~~~~~~~~~~~~~~~~ 14 15Let's start with an example, following code is a simple test for a 'getenv()'. 16 17[source,c] 18------------------------------------------------------------------------------- 19/*\ 20 * [Description] 21 * Tests basic functionality of getenv(). 22 * 23 * - create an env variable and verify that getenv() can get it 24 * - call getenv() with nonexisting variable name, check that it returns NULL 25 */ 26 27#include "tst_test.h" 28 29#define ENV1 "LTP_TEST_ENV" 30#define ENV2 "LTP_TEST_THIS_DOES_NOT_EXIST" 31#define ENV_VAL "val" 32 33static void setup(void) 34{ 35 if (setenv(ENV1, ENV_VAL, 1)) 36 tst_brk(TBROK | TERRNO, "setenv() failed"); 37} 38 39static void run(void) 40{ 41 char *ret; 42 43 ret = getenv(ENV1); 44 45 if (!ret) { 46 tst_res(TFAIL, "getenv(" ENV1 ") = NULL"); 47 goto next; 48 } 49 50 if (!strcmp(ret, ENV_VAL)) { 51 tst_res(TPASS, "getenv(" ENV1 ") = '"ENV_VAL "'"); 52 } else { 53 tst_res(TFAIL, "getenv(" ENV1 ") = '%s', expected '" 54 ENV_VAL "'", ret); 55 } 56 57next: 58 ret = getenv(ENV2); 59 60 if (ret) 61 tst_res(TFAIL, "getenv(" ENV2 ") = '%s'", ret); 62 else 63 tst_res(TPASS, "getenv(" ENV2 ") = NULL"); 64} 65 66static struct tst_test test = { 67 .test_all = run, 68 .setup = setup, 69}; 70------------------------------------------------------------------------------- 71 72Each test includes the 'tst_test.h' header and must define the 'struct 73tst_test test' structure. 74 75The overall test initialization is done in the 'setup()' function. 76 77The overall cleanup is done in a 'cleanup()' function. Here 'cleanup()' is 78omitted as the test does not have anything to clean up. If cleanup is set in 79the test structure it's called on test exit just before the test library 80cleanup. That especially means that cleanup can be called at any point in a 81test execution. For example even when a test setup step has failed, therefore 82the 'cleanup()' function must be able to cope with unfinished initialization, 83and so on. 84 85The test itself is done in the 'test()' function. The test function must work 86fine if called in a loop. 87 88There are two types of a test function pointers in the test structure. The 89first one is a '.test_all' pointer that is used when test is implemented as a 90single function. Then there is a '.test' function along with the number of 91tests '.tcnt' that allows for more detailed result reporting. If the '.test' 92pointer is set the function is called '.tcnt' times with an integer parameter 93in range of [0, '.tcnt' - 1]. 94 95IMPORTANT: Only one of '.test' and '.test_all' can be set at a time. 96 97Each test has a limit on how long it can run and the limit composes of two 98parts max_runtime and timeout. The max_runtime is a limit for how long can the 99'.test_all' or a set of '.test' functions take and the timeout is static part 100that should cover the duration of test setup and cleanup plus some safety. 101 102Any test that runs for more than a second or two has to make sure to: 103 104- set the runtime either by setting the '.max_runtime' in tst_test or by 105 calling 'tst_set_max_runtime()' in the test setup 106 107- monitor remaning runtime by regular calls to 'tst_remaining_runtime()' and 108 exit when runtime has been used up 109 110Test is free to exit before max_runtime has been used up for example when 111minimal number of iteration was finished. 112 113The limit is applied to a single call of the '.test_all' function that means 114that for example when '.test_variants' or '.all_filesystems' is set the whole 115test will be limited by 'variants * (max_runtime + timeout)' seconds and the 116test runtime will be likely close to 'variants * max_runtime' seconds. 117 118[source,c] 119------------------------------------------------------------------------------- 120/* 121 * Returns number of seconds or zero in case that runtime has been used up. 122 */ 123 124int tst_remaining_runtime(void); 125------------------------------------------------------------------------------- 126 127LAPI headers 128++++++++++++ 129 130Use our LAPI headers ('include "lapi/foo.h"') to keep compatibility with old 131distributions. LAPI header should always include original header. Older linux 132headers were problematic, therefore we preferred to use libc headers. There are 133still some bugs when combining certain glibc headers with linux headers, see 134https://sourceware.org/glibc/wiki/Synchronizing_Headers. 135 136A word about the cleanup() callback 137+++++++++++++++++++++++++++++++++++ 138 139There are a few rules that needs to be followed in order to write correct 140cleanup() callback. 141 1421. Free only resources that were initialized. Keep in mind that callback can 143 be executed at any point in the test run. 144 1452. Make sure to free resources in the reverse order they were 146 initialized. (Some of the steps may not depend on others and everything 147 will work if there were swapped but let's keep it in order.) 148 149The first rule may seem complicated at first however, on the contrary, it's 150quite easy. All you have to do is to keep track of what was already 151initialized. For example file descriptors needs to be closed only if they were 152assigned a valid file descriptor. For most of the things you need to create 153extra flag that is set right after successful initialization though. Consider, 154for example, test setup below. 155 156We also prefer cleaning up resources that would otherwise be released on the 157program exit. There are two main reasons for this decision. Resources such as 158file descriptors and mmaped memory could block umounting a block device in 159cases where the test library has mounted a filesystem for the test temporary 160directory. Not freeing allocated memory would upset static analysis and tools 161such as valgrind and produce false-positives when checking for leaks in the 162libc and other low level libraries. 163 164[source,c] 165------------------------------------------------------------------------------- 166static int fd0, fd1, mount_flag; 167 168#define MNTPOINT "mntpoint" 169#define FILE1 "mntpoint/file1" 170#define FILE2 "mntpoint/file2" 171 172static void setup(void) 173{ 174 SAFE_MKDIR(MNTPOINT, 0777); 175 SAFE_MKFS(tst_device->dev, tst_device->fs_type, NULL, NULL); 176 SAFE_MOUNT(tst_device->dev, MNTPOINT, tst_device->fs_type, 0, 0); 177 mount_flag = 1; 178 179 fd0 = SAFE_OPEN(cleanup, FILE1, O_CREAT | O_RDWR, 0666); 180 fd1 = SAFE_OPEN(cleanup, FILE2, O_CREAT | O_RDWR, 0666); 181} 182------------------------------------------------------------------------------- 183 184In this case the 'cleanup()' function may be invoked when any of the 'SAFE_*' 185macros has failed and therefore must be able to work with unfinished 186initialization as well. Since global variables are initialized to zero we can 187just check that fd > 0 before we attempt to close it. The mount function 188requires extra flag to be set after device was successfully mounted. 189 190[source,c] 191------------------------------------------------------------------------------- 192static void cleanup(void) 193{ 194 if (fd1 > 0) 195 SAFE_CLOSE(fd1); 196 197 if (fd0 > 0) 198 SAFE_CLOSE(fd0); 199 200 if (mount_flag && tst_umouont(MNTPOINT)) 201 tst_res(TWARN | TERRNO, "umount(%s)", MNTPOINT); 202} 203------------------------------------------------------------------------------- 204 205IMPORTANT: 'SAFE_MACROS()' used in cleanup *do not* exit the test. Failure 206 only produces a warning and the 'cleanup()' carries on. This is 207 intentional as we want to execute as much 'cleanup()' as possible. 208 209WARNING: Calling tst_brk() in test 'cleanup()' does not exit the test as well 210 and 'TBROK' is converted to 'TWARN'. 211 212NOTE: Creation and removal of the test temporary directory is handled in 213 the test library and the directory is removed recursively. Therefore 214 we do not have to remove files and directories in the test cleanup. 215 2161.2 Basic test interface 217~~~~~~~~~~~~~~~~~~~~~~~~ 218 219[source,c] 220------------------------------------------------------------------------------- 221void tst_res(int ttype, char *arg_fmt, ...); 222------------------------------------------------------------------------------- 223 224Printf-like function to report test result, it's mostly used with ttype: 225 226|============================== 227| 'TPASS' | Test has passed. 228| 'TFAIL' | Test has failed. 229| 'TINFO' | General message. 230| 'TWARN' | Something went wrong but we decided to continue. Mostly used in cleanup functions. 231|============================== 232 233The 'ttype' can be combined bitwise with 'TERRNO' or 'TTERRNO' to print 234'errno', 'TST_ERR' respectively. 235 236[source,c] 237------------------------------------------------------------------------------- 238void tst_brk(int ttype, char *arg_fmt, ...); 239------------------------------------------------------------------------------- 240 241Printf-like function to report error and exit the test, it can be used with ttype: 242 243|============================================================ 244| 'TBROK' | Something has failed in test preparation phase. 245| 'TCONF' | Test is not appropriate for current configuration 246 (syscall not implemented, unsupported arch, ...) 247|============================================================ 248 249The 'ttype' can be combined bitwise with 'TERRNO' or 'TTERRNO' to print 250'errno', 'TST_ERR' respectively. 251 252There are also 'TST_EXP_*()' macros that can simplify syscall unit tests to a 253single line, use them whenever possible. These macros take a function call as 254the first parameter and a printf-like format string and parameters as well. 255These test macros then expand to a code that runs the call, checks the return 256value and errno and reports the test result. 257 258[source,c] 259------------------------------------------------------------------------------- 260static void run(void) 261{ 262 ... 263 TST_EXP_PASS(stat(fname, &statbuf), "stat(%s, ...)", fname); 264 265 if (!TST_PASS) 266 return; 267 ... 268} 269------------------------------------------------------------------------------- 270 271The 'TST_EXP_PASS()' can be used for calls that return -1 on failure and 0 on 272success. It will check for the return value and reports failure if the return 273value is not equal to 0. The call also sets the 'TST_PASS' variable to 1 if 274the call succeeeded. 275 276As seen above, this and similar macros take optional variadic arguments. These 277begin with a format string and then appropriate values to be formatted. 278 279[source,c] 280------------------------------------------------------------------------------- 281static void run(void) 282{ 283 ... 284 TST_EXP_FD(open(fname, O_RDONLY), "open(%s, O_RDONLY)", fname); 285 286 SAFE_CLOSE(TST_RET); 287 ... 288} 289------------------------------------------------------------------------------- 290 291The 'TST_EXP_FD()' is the same as 'TST_EXP_PASS()' the only difference is that 292the return value is expected to be a file descriptor so the call passes if 293positive integer is returned. 294 295[source,c] 296------------------------------------------------------------------------------- 297static void run(void) 298{ 299 ... 300 TST_EXP_FAIL(stat(fname, &statbuf), ENOENT, "stat(%s, ...)", fname); 301 ... 302} 303------------------------------------------------------------------------------- 304 305The 'TST_EXP_FAIL()' is similar to 'TST_EXP_PASS()' but it fails the test if 306the call haven't failed with -1 and 'errno' wasn't set to the expected one 307passed as the second argument. 308 309[source,c] 310------------------------------------------------------------------------------- 311static void run(void) 312{ 313 ... 314 TST_EXP_FAIL2(msgget(key, flags), EINVAL, "msgget(%i, %i)", key, flags); 315 ... 316} 317------------------------------------------------------------------------------- 318 319The 'TST_EXP_FAIL2()' is the same as 'TST_EXP_FAIL()' except the return value is 320expected to be non-negative integer if call passes. These macros build upon the 321+TEST()+ macro and associated variables. 322 323'TST_EXP_FAIL_SILENT()' and 'TST_EXP_FAIL2_SILENT()' variants are less verbose 324and do not print TPASS messages when SCALL fails as expected. 325 326[source,c] 327------------------------------------------------------------------------------- 328TEST(socket(AF_INET, SOCK_RAW, 1)); 329if (TST_RET > -1) { 330 tst_res(TFAIL, "Created raw socket"); 331 SAFE_CLOSE(TST_RET); 332} else if (TST_ERR != EPERM) { 333 tst_res(TFAIL | TTERRNO, 334 "Failed to create socket for wrong reason"); 335} else { 336 tst_res(TPASS | TTERRNO, "Didn't create raw socket"); 337} 338------------------------------------------------------------------------------- 339 340The +TEST+ macro sets +TST_RET+ to its argument's return value and +TST_ERR+ to 341+errno+. The +TTERNO+ flag can be used to print the error number's symbolic 342value. 343 344No LTP library function or macro, except those in 'tst_test_macros.h', will 345write to these variables (rule 'LTP-002'). So their values will not be changed 346unexpectedly. 347 348[source,c] 349------------------------------------------------------------------------------- 350TST_EXP_POSITIVE(wait(&status)); 351 352if (!TST_PASS) 353 return; 354------------------------------------------------------------------------------- 355 356If the return value of 'wait' is positive or zero, this macro will print a pass 357result and set +TST_PASS+ appropriately. If the return value is negative, then 358it will print fail. There are many similar macros to those shown here, please 359see 'tst_test_macros.h'. 360 361[source,c] 362------------------------------------------------------------------------------- 363TST_EXP_EQ_LI(val1, val2); 364TST_EXP_EQ_UI(val1, val2); 365TST_EXP_EQ_SZ(val1, val2); 366TST_EXP_EQ_SSZ(val1, val2); 367 368/* Use as */ 369TST_EXP_EQ_LI(sig_caught, SIGCHLD); 370------------------------------------------------------------------------------- 371 372Set of macros for different integer type comparsions. These macros print the 373variable names as well as values in both pass and fail scenarios. 374 375[source,c] 376------------------------------------------------------------------------------- 377const char *tst_strsig(int sig); 378------------------------------------------------------------------------------- 379 380Return the given signal number's corresponding string. 381 382[source,c] 383------------------------------------------------------------------------------- 384const char *tst_strerrno(int err); 385------------------------------------------------------------------------------- 386 387Return the given errno number's corresponding string. Using this function to 388translate 'errno' values to strings is preferred. You should not use the 389'strerror()' function in the testcases. 390 391[source,c] 392------------------------------------------------------------------------------- 393const char *tst_strstatus(int status); 394------------------------------------------------------------------------------- 395 396Returns string describing the status as returned by 'wait()'. 397 398WARNING: This function is not thread safe. 399 400[source,c] 401------------------------------------------------------------------------------- 402void tst_set_max_runtime(int max_runtime); 403------------------------------------------------------------------------------- 404 405Allows for setting max_runtime per test iteration dynamically in the test 'setup()', 406the timeout is specified in seconds. There are a few testcases whose runtime 407can vary arbitrarily, these can disable timeouts by setting it to 408TST_UNLIMITED_RUNTIME. 409 410[source,c] 411------------------------------------------------------------------------------- 412void tst_flush(void); 413------------------------------------------------------------------------------- 414 415Flush output streams, handling errors appropriately. 416 417This function is rarely needed when you have to flush the output streams 418before calling 'fork()' or 'clone()'. Note that the 'SAFE_FORK()' and 'SAFE_CLONE()' 419calls this function automatically. See 2.4 FILE buffers and fork() for explanation 420why is this needed. 421 4221.3 Test temporary directory 423~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 424 425If '.needs_tmpdir' is set to '1' in the 'struct tst_test' unique test 426temporary is created and it's set as the test working directory. Tests *MUST 427NOT* create temporary files outside that directory. The flag is not needed to 428be set when use these flags: '.all_filesystems', '.format_device', '.mntpoint', 429'.mount_device' '.needs_checkpoints', '.needs_device', '.resource_file' 430(these flags imply creating temporary directory). 431 432IMPORTANT: Close all file descriptors (that point to files in test temporary 433 directory, even the unlinked ones) either in the 'test()' function 434 or in the test 'cleanup()' otherwise the test may break temporary 435 directory removal on NFS (look for "NFS silly rename"). 436 4371.4 Safe macros 438~~~~~~~~~~~~~~~ 439 440Safe macros aim to simplify error checking in test preparation. Instead of 441calling system API functions, checking for their return value and aborting the 442test if the operation has failed, you just use corresponding safe macro. 443 444Use them whenever it's possible. 445 446Instead of writing: 447 448[source,c] 449------------------------------------------------------------------------------- 450fd = open("/dev/null", O_RDONLY); 451if (fd < 0) 452 tst_brk(TBROK | TERRNO, "opening /dev/null failed"); 453------------------------------------------------------------------------------- 454 455You write just: 456 457[source,c] 458------------------------------------------------------------------------------- 459fd = SAFE_OPEN("/dev/null", O_RDONLY); 460------------------------------------------------------------------------------- 461 462IMPORTANT: The 'SAFE_CLOSE()' function also sets the passed file descriptor to -1 463 after it's successfully closed. 464 465They can also simplify reading and writing of sysfs files, you can, for 466example, do: 467 468[source,c] 469------------------------------------------------------------------------------- 470SAFE_FILE_SCANF("/proc/sys/kernel/pid_max", "%lu", &pid_max); 471------------------------------------------------------------------------------- 472 473See 'include/tst_safe_macros.h', 'include/tst_safe_stdio.h' and 474'include/tst_safe_file_ops.h' and 'include/tst_safe_net.h' for a complete list. 475 4761.5 Test specific command line options 477~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 478 479[source,c] 480------------------------------------------------------------------------------- 481struct tst_option { 482 char *optstr; 483 char **arg; 484 char *help; 485}; 486------------------------------------------------------------------------------- 487 488Test specific command line parameters can be passed with the 'NULL' terminated 489array of 'struct tst_option'. The 'optstr' is the command line option i.e. "o" 490or "o:" if option has a parameter. Only short options are supported. The 'arg' 491is where 'optarg' is stored upon match. If option has no parameter it's set to 492non-'NULL' value if option was present. The 'help' is a short help string. 493 494NOTE: The test parameters must not collide with common test parameters defined 495 in the library the currently used ones are +-i+, +-I+, +-C+, and +-h+. 496 497[source,c] 498------------------------------------------------------------------------------- 499int tst_parse_int(const char *str, int *val, int min, int max); 500int tst_parse_long(const char *str, long *val, long min, long max); 501int tst_parse_float(const char *str, float *val, float min, float max); 502int tst_parse_filesize(const char *str, long long *val, long long min, long long max); 503------------------------------------------------------------------------------- 504 505Helpers for parsing the strings returned in the 'struct tst_option'. 506 507Helpers return zero on success and 'errno', mostly 'EINVAL' or 'ERANGE', on 508failure. 509 510Helpers functions are no-op if 'str' is 'NULL'. 511 512The valid range for result includes both 'min' and 'max'. 513 514In particular, 'tst_parse_filesize' function accepts prefix multiplies such as 515"k/K for kilobytes, "m/M" for megabytes and "g/G" for gigabytes. For example, 51610K are converted into 10240 bytes. 517 518[source,c] 519------------------------------------------------------------------------------- 520#include <limits.h> 521#include "tst_test.h" 522 523static char *str_threads; 524static int threads = 10; 525 526static void setup(void) 527{ 528 if (tst_parse_int(str_threads, &threads, 1, INT_MAX)) 529 tst_brk(TBROK, "Invalid number of threads '%s'", str_threads); 530 531 ... 532} 533 534static void test_threads(void) 535{ 536 ... 537 538 for (i = 0; i < threads; i++) { 539 ... 540 } 541 542 ... 543} 544 545static struct tst_test test = { 546 ... 547 .options = (struct tst_option[]) { 548 {"t:", &str_threads, "Number of threads (default 10)"}, 549 {}, 550 ... 551}; 552------------------------------------------------------------------------------- 553 554 5551.6 Runtime kernel version detection 556~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 557 558Testcases for newly added kernel functionality require kernel newer than a 559certain version to run. All you need to skip a test on older kernels is to 560set the '.min_kver' string in the 'struct tst_test' to a minimal required 561kernel version, e.g. '.min_kver = "4.10.0"'. 562 563For more complicated operations such as skipping a test for a certain range 564of kernel versions, following functions could be used: 565 566[source,c] 567------------------------------------------------------------------------------- 568int tst_kvercmp(int r1, int r2, int r3); 569 570struct tst_kern_exv { 571 char *dist_name; 572 char *extra_ver; 573}; 574 575int tst_kvercmp2(int r1, int r2, int r3, struct tst_kern_exv *vers); 576------------------------------------------------------------------------------- 577 578These two functions are intended for runtime kernel version detection. They 579parse the output from 'uname()' and compare it to the passed values. 580 581The return value is similar to the 'strcmp()' function, i.e. zero means equal, 582negative value means that the kernel is older than the expected value and 583positive means that it's newer. 584 585The second function 'tst_kvercmp2()' allows for specifying per-vendor table of 586kernel versions as vendors typically backport fixes to their kernels and the 587test may be relevant even if the kernel version does not suggests so. 588 589[source,c] 590------------------------------------------------------------------------------- 591if (tst_kvercmp(5, 19, 0) >= 0) 592 tst_res(TCONF, "Test valid only for kernel < 5.19"); 593 594static struct tst_kern_exv kvers[] = { 595 { "UBUNTU", "4.4.0-48.69" }, 596 { NULL, NULL}, 597}; 598 599if (tst_kvercmp2(4, 4, 27, kvers) < 0) 600 /* code for kernel < v4.4.27 or ubuntu kernel < 4.4.0-48.69 */ 601------------------------------------------------------------------------------- 602 603WARNING: The shell 'tst_kvercmp' maps the result into unsigned integer - the 604 process exit value. 605 606NOTE: See also LTP 607 https://github.com/linux-test-project/ltp/wiki/Supported-kernel,-libc,-toolchain-versions#13-minimal-supported-kernel-version[minimal supported kernel version]. 608 6091.7 Fork()-ing 610~~~~~~~~~~~~~~ 611 612Be wary that if the test forks and there were messages printed by the 613'tst_*()' interfaces, the data may still be in libc/kernel buffers and these 614*ARE NOT* flushed automatically. 615 616This happens when 'stdout' gets redirected to a file. In this case, the 617'stdout' is not line buffered, but block buffered. Hence after a fork content 618of the buffers will be printed by the parent and each of the children. 619 620To avoid that you should use 'SAFE_FORK()', 'SAFE_CLONE()' or 'tst_clone()'. 621 622IMPORTANT: You have to set the '.forks_child' flag in the test structure 623 if your testcase forks or calls 'SAFE_CLONE()'. 624 6251.8 Doing the test in the child process 626~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 627 628Results reported by 'tst_res()' are propagated to the parent test process via 629block of shared memory. 630 631Calling 'tst_brk()' causes child process to exit with non-zero exit value. 632Which means that it's safe to use 'SAFE_*()' macros in the child processes as 633well. 634 635Children that outlive the 'test()' function execution are waited for in the 636test library. Unclean child exit (killed by signal, non-zero exit value, etc.) 637will cause the main test process to exit with 'tst_brk()', which especially 638means that 'TBROK' propagated from a child process will cause the whole test 639to exit with 'TBROK'. 640 641If a test needs a child that segfaults or does anything else that cause it to 642exit uncleanly all you need to do is to wait for such children from the 643'test()' function so that it's reaped before the main test exits the 'test()' 644function. 645 646[source,c] 647------------------------------------------------------------------------------- 648#include "tst_test.h" 649 650void tst_reap_children(void); 651------------------------------------------------------------------------------- 652 653The 'tst_reap_children()' function makes the process wait for all of its 654children and exits with 'tst_brk(TBROK, ...)' if any of them returned 655a non zero exit code. 656 657When using 'SAFE_CLONE' or 'tst_clone', this may not work depending on 658the parameters passed to clone. The following call to 'SAFE_CLONE' is 659identical to 'fork()', so will work as expected. 660 661[source,c] 662-------------------------------------------------------------------------------- 663const struct tst_clone_args args = { 664 .exit_signal = SIGCHLD, 665}; 666 667SAFE_CLONE(&args); 668-------------------------------------------------------------------------------- 669 670If 'exit_signal' is set to something else, then this will break 671'tst_reap_children'. It's not expected that all parameters to clone will 672work with the LTP library unless specific action is taken by the test code. 673 674.Using 'tst_res()' from binaries started by 'exec()' 675[source,c] 676------------------------------------------------------------------------------- 677/* test.c */ 678#define _GNU_SOURCE 679#include <unistd.h> 680#include "tst_test.h" 681 682static void do_test(void) 683{ 684 char *const argv[] = {"test_exec_child", NULL}; 685 char path[4096]; 686 687 if (tst_get_path("test_exec_child", path, sizeof(path))) 688 tst_brk(TCONF, "Couldn't find test_exec_child in $PATH"); 689 690 execve(path, argv, environ); 691 692 tst_res(TFAIL | TERRNO, "EXEC!"); 693} 694 695static struct tst_test test = { 696 .test_all = do_test, 697 .child_needs_reinit = 1, 698}; 699 700/* test_exec_child.c */ 701#define TST_NO_DEFAULT_MAIN 702#include "tst_test.h" 703 704int main(void) 705{ 706 tst_reinit(); 707 tst_res(TPASS, "Child passed!"); 708 return 0; 709} 710------------------------------------------------------------------------------- 711 712The 'tst_res()' function can be also used from binaries started by 'exec()', 713the parent test process has to set the '.child_needs_reinit' flag so that the 714library prepares for it and has to make sure the 'LTP_IPC_PATH' environment 715variable is passed down, then the very first thing the program has to call in 716'main()' is 'tst_reinit()' that sets up the IPC. 717 7181.9 Fork() and Parent-child synchronization 719~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 720 721As LTP tests are written for Linux, most of the tests involve fork()-ing and 722parent-child process synchronization. LTP includes a checkpoint library that 723provides wait/wake futex based functions. 724 725In order to use checkpoints the '.needs_checkpoints' flag in the 'struct 726tst_test' must be set to '1', this causes the test library to initialize 727checkpoints before the 'test()' function is called. 728 729[source,c] 730------------------------------------------------------------------------------- 731#include "tst_test.h" 732 733TST_CHECKPOINT_WAIT(id) 734 735TST_CHECKPOINT_WAIT2(id, msec_timeout) 736 737TST_CHECKPOINT_WAKE(id) 738 739TST_CHECKPOINT_WAKE2(id, nr_wake) 740 741TST_CHECKPOINT_WAKE_AND_WAIT(id) 742------------------------------------------------------------------------------- 743 744The checkpoint interface provides pair of wake and wait functions. The 'id' is 745unsigned integer which specifies checkpoint to wake/wait for. As a matter of 746fact it's an index to an array stored in a shared memory, so it starts on 747'0' and there should be enough room for at least of hundred of them. 748 749The 'TST_CHECKPOINT_WAIT()' and 'TST_CHECKPOINT_WAIT2()' suspends process 750execution until it's woken up or until timeout is reached. 751 752The 'TST_CHECKPOINT_WAKE()' wakes one process waiting on the checkpoint. 753If no process is waiting the function retries until it success or until 754timeout is reached. 755 756If timeout has been reached process exits with appropriate error message (uses 757'tst_brk()'). 758 759The 'TST_CHECKPOINT_WAKE2()' does the same as 'TST_CHECKPOINT_WAKE()' but can 760be used to wake precisely 'nr_wake' processes. 761 762The 'TST_CHECKPOINT_WAKE_AND_WAIT()' is a shorthand for doing wake and then 763immediately waiting on the same checkpoint. 764 765Child processes created via 'SAFE_FORK()' are ready to use the checkpoint 766synchronization functions, as they inherited the mapped page automatically. 767 768Child processes started via 'exec()', or any other processes not forked from 769the test process must initialize the checkpoint by calling 'tst_reinit()'. 770 771For the details of the interface, look into the 'include/tst_checkpoint.h'. 772 773[source,c] 774------------------------------------------------------------------------------- 775#include "tst_test.h" 776 777/* 778 * Waits for process state change. 779 * 780 * The state is one of the following: 781 * 782 * R - process is running 783 * S - process is sleeping 784 * D - process sleeping uninterruptibly 785 * Z - zombie process 786 * T - process is traced 787 */ 788TST_PROCESS_STATE_WAIT(pid, state, msec_timeout) 789------------------------------------------------------------------------------- 790 791The 'TST_PROCESS_STATE_WAIT()' waits until process 'pid' is in requested 792'state' or timeout is reached. The call polls +/proc/pid/stat+ to get this 793information. A timeout of 0 will wait infinitely. 794 795On timeout -1 is returned and errno set to ETIMEDOUT. 796 797It's mostly used with state 'S' which means that process is sleeping in kernel 798for example in 'pause()' or any other blocking syscall. 799 8001.10 Signals and signal handlers 801~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 802 803If you need to use signal handlers, keep the code short and simple. Don't 804forget that the signal handler is called asynchronously and can interrupt the 805code execution at any place. 806 807This means that problems arise when global state is changed both from the test 808code and signal handler, which will occasionally lead to: 809 810* Data corruption (data gets into inconsistent state), this may happen, for 811 example, for any operations on 'FILE' objects. 812 813* Deadlock, this happens, for example, if you call 'malloc(2)', 'free(2)', 814 etc. from both the test code and the signal handler at the same time since 815 'malloc' has global lock for it's internal data structures. (Be wary that 816 'malloc(2)' is used by the libc functions internally too.) 817 818* Any other unreproducible and unexpected behavior. 819 820Quite common mistake is to call 'exit(3)' from a signal handler. Note that this 821function is not signal-async-safe as it flushes buffers, etc. If you need to 822exit a test immediately from a signal handler use '_exit(2)' instead. 823 824TIP: See 'man 7 signal' for the list of signal-async-safe functions. 825 826If a signal handler sets a variable, its declaration must be 'volatile', 827otherwise compiler may misoptimize the code. This is because the variable may 828not be changed in the compiler code flow analysis. There is 'sig_atomic_t' 829type defined in C99 but this one *DOES NOT* imply 'volatile' (it's just a 830'typedef' to 'int'). So the correct type for a flag that is changed from a 831signal handler is either 'volatile int' or 'volatile sig_atomic_t'. 832 833If a crash (e.g. triggered by signal SIGSEGV) is expected in testing, you 834can avoid creation of core files by calling 'tst_no_corefile()' function. 835This takes effect for process (and its children) which invoked it, unless 836they subsequently modify RLIMIT_CORE. 837 838Note that LTP library will reap any processes that test didn't reap itself, 839and report any non-zero exit code as failure. 840 8411.11 Kernel Modules 842~~~~~~~~~~~~~~~~~~~ 843 844There are certain cases where the test needs a kernel part and userspace part, 845happily, LTP can build a kernel module and then insert it to the kernel on test 846start for you. See 'testcases/kernel/device-drivers/block' for details. 847 8481.12 Useful macros 849~~~~~~~~~~~~~~~~~~ 850 851These macros are defined in 'include/tst_common.h'. 852 853[source,c] 854------------------------------------------------------------------------------- 855ARRAY_SIZE(arr) 856------------------------------------------------------------------------------- 857 858Returns the size of statically defined array, i.e. 859'(sizeof(arr) / sizeof(*arr))' 860 861[source,c] 862------------------------------------------------------------------------------- 863LTP_ALIGN(x, a) 864------------------------------------------------------------------------------- 865 866Aligns the x to be next multiple of a. The a must be power of 2. 867 868[source,c] 869------------------------------------------------------------------------------- 870TST_TO_STR(s) /* stringification */ 871TST_TO_STR_(s) /* macro expansion */ 872------------------------------------------------------------------------------- 873 874Macros for stringification. 875 8761.13 Filesystem type detection and skiplist 877~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 878 879Some tests are known to fail on certain filesystems (you cannot swap on TMPFS, 880there are unimplemented 'fcntl()' etc.). 881 882If your test needs to be skipped on certain filesystems use the 883'.skip_filesystems' field in the tst_test structure as follows: 884 885[source,c] 886------------------------------------------------------------------------------- 887#include "tst_test.h" 888 889static struct tst_test test = { 890 ... 891 .skip_filesystems = (const char *const []) { 892 "tmpfs", 893 "ramfs", 894 "nfs", 895 NULL 896 }, 897}; 898------------------------------------------------------------------------------- 899 900When the '.all_filesystems' flag is set the '.skip_filesystems' list is passed 901to the function that detects supported filesystems any listed filesystem is 902not included in the resulting list of supported filesystems. 903 904If test needs to adjust expectations based on filesystem type it's also 905possible to detect filesystem type at the runtime. This is preferably used 906when only subset of the test is not applicable for a given filesystem. 907 908NOTE: ext2, ext3 or ext4 in '.skip_filesystems' on tests which does *not* use 909 '.all_filesystems' needs to be defined as 'ext2/ext3/ext4'. The reason 910 is that it is hard to detect used filesystem due to overlapping the functionality. 911 OTOH tests which use '.skip_filesystems' *with* '.all_filesystems' can skip 912 only filesystems which are actually used in '.all_filesystems': ext2, ext3, 913 ext4, xfs, btrfs, vfat, exfat, ntfs, tmpfs (defined in 'fs_type_whitelist[]'). 914 It does not make sense to list other filesystems. 915 916 917[source,c] 918------------------------------------------------------------------------------- 919#include "tst_test.h" 920 921static void run(void) 922{ 923 ... 924 925 switch ((type = tst_fs_type("."))) { 926 case TST_NFS_MAGIC: 927 case TST_TMPFS_MAGIC: 928 case TST_RAMFS_MAGIC: 929 tst_brk(TCONF, "Subtest not supported on %s", 930 tst_fs_type_name(type)); 931 return; 932 break; 933 } 934 935 ... 936} 937------------------------------------------------------------------------------- 938 9391.14 Thread-safety in the LTP library 940~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 941 942It is safe to use library 'tst_res()' function in multi-threaded tests. 943 944Only the main thread must return from the 'test()' function to the test 945library and that must be done only after all threads that may call any library 946function has been terminated. That especially means that threads that may call 947'tst_brk()' must terminate before the execution of the 'test()' function 948returns to the library. This is usually done by the main thread joining all 949worker threads at the end of the 'test()' function. Note that the main thread 950will never get to the library code in a case that 'tst_brk()' was called from 951one of the threads since it will sleep at least in 'pthread_join()' on the 952thread that called the 'tst_brk()' till 'exit()' is called by 'tst_brk()'. 953 954The test-supplied cleanup function runs *concurrently* to the rest of the 955threads in a case that cleanup was entered from 'tst_brk()'. Subsequent 956threads entering 'tst_brk()' must be suspended or terminated at the start of 957the user supplied cleanup function. It may be necessary to stop or exit 958the rest of the threads before the test cleans up as well. For example threads 959that create new files should be stopped before temporary directory is be 960removed. 961 962Following code example shows thread safe cleanup function example using atomic 963increment as a guard. The library calls its cleanup after the execution returns 964from the user supplied cleanup and expects that only one thread returns from 965the user supplied cleanup to the test library. 966 967[source,c] 968------------------------------------------------------------------------------- 969#include "tst_test.h" 970 971static void cleanup(void) 972{ 973 static int flag; 974 975 if (tst_atomic_inc(&flag) != 1) 976 pthread_exit(NULL); 977 978 /* if needed stop the rest of the threads here */ 979 980 ... 981 982 /* then do cleanup work */ 983 984 ... 985 986 /* only one thread returns to the library */ 987} 988------------------------------------------------------------------------------- 989 990 9911.15 Testing with a block device 992~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 993 994Some tests needs a block device (inotify tests, syscall 'EROFS' failures, 995etc.). LTP library contains a code to prepare a testing device. 996 997If '.needs_device' flag in the 'struct tst_test' is set the 'tst_device' 998structure is initialized with a path to a test device and default filesystem 999to be used. 1000 1001You can also request minimal device size in megabytes by setting 1002'.dev_min_size' the device is guaranteed to have at least the requested size 1003then. 1004 1005If '.format_device' flag is set the device is formatted with a filesystem as 1006well. You can use '.dev_fs_type' to override the default filesystem type if 1007needed and pass additional options to mkfs via '.dev_fs_opts' and 1008'.dev_extra_opts' pointers. Note that '.format_device' implies '.needs_device' 1009there is no need to set both. 1010 1011If '.mount_device' is set, the device is mounted at '.mntpoint' which is used 1012to pass a directory name that will be created and used as mount destination. 1013You can pass additional flags and data to the mount command via '.mnt_flags' 1014and '.mnt_data' pointers. Note that '.mount_device' implies '.needs_device' 1015and '.format_device' so there is no need to set the later two. 1016 1017If '.needs_rofs' is set, read-only filesystem is mounted at '.mntpoint' this 1018one is supposed to be used for 'EROFS' tests. 1019 1020If '.all_filesystems' is set the test function is executed for all supported 1021filesystems. Supported filesystems are detected based on existence of the 1022'mkfs.$fs' helper and on kernel support to mount it. For each supported 1023filesystem the 'tst_device.fs_type' is set to the currently tested fs type, if 1024'.format_device' is set the device is formatted as well, if '.mount_device' is 1025set it's mounted at '.mntpoint'. Also the test timeout is reset for each 1026execution of the test function. This flag is expected to be used for filesystem 1027related syscalls that are at least partly implemented in the filesystem 1028specific code e.g. 'fallocate()'. 1029 1030[source,c] 1031------------------------------------------------------------------------------- 1032#include "tst_test.h" 1033 1034struct tst_device { 1035 const char *dev; 1036 const char *fs_type; 1037}; 1038 1039extern struct tst_device *tst_device; 1040 1041int tst_umount(const char *path); 1042------------------------------------------------------------------------------- 1043 1044In case that 'LTP_DEV' is passed to the test in an environment, the library 1045checks that the file exists and that it's a block device, if 1046'.device_min_size' is set the device size is checked as well. If 'LTP_DEV' 1047wasn't set or if size requirements were not met a temporary file is created 1048and attached to a free loop device. 1049 1050If there is no usable device and loop device couldn't be initialized the test 1051exits with 'TCONF'. 1052 1053The 'tst_umount()' function works exactly as 'umount(2)' but retries several 1054times on 'EBUSY'. This is because various desktop daemons (gvfsd-trash is known 1055for that) may be stupid enough to probe all newly mounted filesystem which 1056results in 'umount(2)' failing with 'EBUSY'. 1057 1058IMPORTANT: All testcases should use 'tst_umount()' instead of 'umount(2)' to 1059 umount filesystems. 1060 1061[source,c] 1062------------------------------------------------------------------------------- 1063#include "tst_test.h" 1064 1065int tst_find_free_loopdev(const char *path, size_t path_len); 1066------------------------------------------------------------------------------- 1067 1068This function finds a free loopdev and returns the free loopdev minor (-1 for no 1069free loopdev). If path is non-NULL, it will be filled with free loopdev path. 1070If you want to use a customized loop device, we can call 'tst_find_free_loopdev(NULL, 0)' 1071in tests to get a free minor number and then mknod. 1072 1073[source,c] 1074------------------------------------------------------------------------------- 1075#include "tst_test.h" 1076 1077unsigned long tst_dev_bytes_written(const char *dev); 1078------------------------------------------------------------------------------- 1079 1080This function reads test block device stat file ('/sys/block/<device>/stat') and 1081returns the bytes written since the last invocation of this function. To avoid 1082FS deferred IO metadata/cache interference, we suggest doing "syncfs" before the 1083tst_dev_bytes_written first invocation. And an inline function named 'tst_dev_sync()' 1084is created for that intention. 1085 1086[source,c] 1087------------------------------------------------------------------------------- 1088#include "tst_test.h" 1089 1090void tst_find_backing_dev(const char *path, char *dev, size_t dev_size); 1091------------------------------------------------------------------------------- 1092 1093This function finds the block dev that this path belongs to, using uevent in sysfs. 1094For Btrfs it uses '/sys/fs/btrfs/UUID/devices/DEV_NAME/uevent'; for other 1095filesystems it uses '/sys/dev/block/MAJOR:MINOR/uevent'. 1096 1097[source,c] 1098------------------------------------------------------------------------------- 1099#include "tst_test.h" 1100 1101uint64_t tst_get_device_size(const char *dev_path); 1102------------------------------------------------------------------------------- 1103 1104This function gets size of the given block device, it checks the 'dev_path' is 1105valid first, if yes, return the size in MB, otherwise return -1. 1106 1107[source,c] 1108------------------------------------------------------------------------------- 1109#include "tst_test.h" 1110 1111int tst_dev_block_size(const char *path); 1112------------------------------------------------------------------------------- 1113 1114This function returns the physical device block size for the specific `path`. 1115It finds the device where `path` is located and then uses `ioctl` (BLKSSZGET) 1116to get a physical device block size. 1117 11181.16 Formatting a device with a filesystem 1119~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1120 1121[source,c] 1122------------------------------------------------------------------------------- 1123#include "tst_test.h" 1124 1125static void setup(void) 1126{ 1127 ... 1128 SAFE_MKFS(tst_device->dev, tst_device->fs_type, NULL, NULL); 1129 ... 1130} 1131------------------------------------------------------------------------------- 1132 1133This function takes a path to a device, filesystem type and an array of extra 1134options passed to mkfs. 1135 1136The fs options 'fs_opts' should either be 'NULL' if there are none, or a 1137'NULL' terminated array of strings such as: 1138+const char *const opts[] = {"-b", "1024", NULL}+. 1139 1140The extra options 'extra_opts' should either be 'NULL' if there are none, or a 1141'NULL' terminated array of strings such as +{"102400", NULL}+; 'extra_opts' 1142will be passed after device name. e.g: +mkfs -t ext4 -b 1024 /dev/sda1 102400+ 1143in this case. 1144 1145Note that perfer to store the options which can be passed before or after device 1146name by 'fs_opts' array. 1147 11481.17 Verifying a filesystem's free space 1149~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1150 1151Some tests have size requirements for the filesystem's free space. If these 1152requirements are not satisfied, the tests should be skipped. 1153 1154[source,c] 1155------------------------------------------------------------------------------- 1156#include "tst_test.h" 1157 1158int tst_fs_has_free(const char *path, unsigned int size, unsigned int mult); 1159------------------------------------------------------------------------------- 1160 1161The 'tst_fs_has_free()' function returns 1 if there is enough space and 0 if 1162there is not. 1163 1164The 'path' is the pathname of any directory/file within a filesystem. 1165 1166The 'mult' is a multiplier, one of 'TST_BYTES', 'TST_KB', 'TST_MB' or 'TST_GB'. 1167 1168The required free space is calculated by 'size * mult', e.g. 1169'tst_fs_has_free("/tmp/testfile", 64, TST_MB)' will return 1 if the 1170filesystem, which '"/tmp/testfile"' is in, has 64MB free space at least, and 0 1171if not. 1172 11731.18 Files, directories and fs limits 1174~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1175 1176Some tests need to know the maximum count of links to a regular file or 1177directory, such as 'rename(2)' or 'linkat(2)' to test 'EMLINK' error. 1178 1179[source,c] 1180------------------------------------------------------------------------------- 1181#include "tst_test.h" 1182 1183int tst_fs_fill_hardlinks(const char *dir); 1184------------------------------------------------------------------------------- 1185 1186Try to get maximum count of hard links to a regular file inside the 'dir'. 1187 1188NOTE: This number depends on the filesystem 'dir' is on. 1189 1190This function uses 'link(2)' to create hard links to a single file until it 1191gets 'EMLINK' or creates 65535 links. If the limit is hit, the maximum number of 1192hardlinks is returned and the 'dir' is filled with hardlinks in format 1193"testfile%i", where i belongs to [0, limit) interval. If no limit is hit or if 1194'link(2)' failed with 'ENOSPC' or 'EDQUOT', zero is returned and previously 1195created files are removed. 1196 1197[source,c] 1198------------------------------------------------------------------------------- 1199#include "tst_test.h" 1200 1201int tst_fs_fill_subdirs(const char *dir); 1202------------------------------------------------------------------------------- 1203 1204Try to get maximum number of subdirectories in directory. 1205 1206NOTE: This number depends on the filesystem 'dir' is on. For current kernel, 1207subdir limit is not available for all filesystems (available for ext2, ext3, 1208minix, sysv and more). If the test runs on some other filesystems, like ramfs, 1209tmpfs, it will not even try to reach the limit and return 0. 1210 1211This function uses 'mkdir(2)' to create directories in 'dir' until it gets 1212'EMLINK' or creates 65535 directories. If the limit is hit, the maximum number 1213of subdirectories is returned and the 'dir' is filled with subdirectories in 1214format "testdir%i", where i belongs to [0, limit - 2) interval (because each 1215newly created dir has two links already - the '.' and the link from parent 1216dir). If no limit is hit or if 'mkdir(2)' failed with 'ENOSPC' or 'EDQUOT', 1217zero is returned and previously created directories are removed. 1218 1219[source,c] 1220------------------------------------------------------------------------------- 1221#include "tst_test.h" 1222 1223int tst_dir_is_empty(const char *dir, int verbose); 1224------------------------------------------------------------------------------- 1225 1226Returns non-zero if directory is empty and zero otherwise. 1227 1228Directory is considered empty if it contains only '.' and '..'. 1229 1230[source,c] 1231------------------------------------------------------------------------------- 1232#include "tst_test.h" 1233 1234void tst_purge_dir(const char *path); 1235------------------------------------------------------------------------------- 1236 1237Deletes the contents of given directory but keeps the directory itself. Useful 1238for cleaning up the temporary directory and mount points between test cases or 1239test iterations. Terminates the program with 'TBROK' on error. 1240 1241[source,c] 1242------------------------------------------------------------------------------- 1243#include "tst_test.h" 1244 1245int tst_fill_fd(int fd, char pattern, size_t bs, size_t bcount); 1246------------------------------------------------------------------------------- 1247 1248Fill a file with specified pattern using file descriptor. 1249 1250[source,c] 1251------------------------------------------------------------------------------- 1252#include "tst_test.h" 1253 1254int tst_prealloc_size_fd(int fd, size_t bs, size_t bcount); 1255------------------------------------------------------------------------------- 1256 1257Preallocate the specified amount of space using 'fallocate()'. Falls back to 1258'tst_fill_fd()' if 'fallocate()' fails. 1259 1260[source,c] 1261------------------------------------------------------------------------------- 1262#include "tst_test.h" 1263 1264int tst_fill_file(const char *path, char pattern, size_t bs, size_t bcount); 1265------------------------------------------------------------------------------- 1266 1267Creates/overwrites a file with specified pattern using file path. 1268 1269[source,c] 1270------------------------------------------------------------------------------- 1271#include "tst_test.h" 1272 1273int tst_prealloc_file(const char *path, size_t bs, size_t bcount); 1274------------------------------------------------------------------------------- 1275 1276Create/overwrite a file and preallocate the specified amount of space for it. 1277The allocated space will not be initialized to any particular content. 1278 12791.19 Getting an unused PID number 1280~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1281 1282Some tests require a 'PID', which is not used by the OS (does not belong to 1283any process within it). For example, kill(2) should set errno to 'ESRCH' if 1284it's passed such 'PID'. 1285 1286[source,c] 1287------------------------------------------------------------------------------- 1288#include "tst_test.h" 1289 1290pid_t tst_get_unused_pid(void); 1291------------------------------------------------------------------------------- 1292 1293Return a 'PID' value not used by the OS or any process within it. 1294 1295[source,c] 1296------------------------------------------------------------------------------- 1297#include "tst_test.h" 1298 1299int tst_get_free_pids(void); 1300------------------------------------------------------------------------------- 1301 1302Returns number of unused pids in the system. Note that this number may be 1303different once the call returns and should be used only for rough estimates. 1304 13051.20 Running executables 1306~~~~~~~~~~~~~~~~~~~~~~~~ 1307 1308[source,c] 1309------------------------------------------------------------------------------- 1310#include "tst_test.h" 1311 1312int tst_cmd(const char *const argv[], 1313 const char *stdout_path, 1314 const char *stderr_path, 1315 enum tst_cmd_flags flags); 1316------------------------------------------------------------------------------- 1317 1318'tst_cmd()' is a wrapper for 'vfork() + execvp()' which provides a way 1319to execute an external program. 1320 1321'argv[]' is a 'NULL' terminated array of strings starting with the program name 1322which is followed by optional arguments. 1323 1324'TST_CMD_PASS_RETVAL' enum 'tst_cmd_flags' makes 'tst_cmd()' 1325return the program exit code to the caller, otherwise 'tst_cmd()' exit the 1326tests on failure. 'TST_CMD_TCONF_ON_MISSING' check for program in '$PATH' and exit 1327with 'TCONF' if not found. 1328 1329In case that 'execvp()' has failed and the enum 'TST_CMD_PASS_RETVAL' flag was set, the 1330return value is '255' if 'execvp()' failed with 'ENOENT' and '254' otherwise. 1331 1332'stdout_path' and 'stderr_path' determine where to redirect the program 1333stdout and stderr I/O streams. 1334 1335The 'SAFE_CMD()' macro can be used automatic handling non-zero exits (exits 1336with 'TBROK') and 'ENOENT' (exits with 'TCONF'). 1337 1338.Example 1339[source,c] 1340------------------------------------------------------------------------------- 1341#include "tst_test.h" 1342 1343const char *const cmd[] = { "ls", "-l", NULL }; 1344 1345... 1346 /* Store output of 'ls -l' into log.txt */ 1347 tst_cmd(cmd, "log.txt", NULL, 0); 1348... 1349------------------------------------------------------------------------------- 1350 13511.21 Measuring elapsed time and helper functions 1352~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1353 1354[source,c] 1355------------------------------------------------------------------------------- 1356#include "tst_timer.h" 1357 1358void tst_timer_check(clockid_t clk_id); 1359 1360void tst_timer_start(clockid_t clk_id); 1361 1362void tst_timer_stop(void); 1363 1364struct timespec tst_timer_elapsed(void); 1365 1366long long tst_timer_elapsed_ms(void); 1367 1368long long tst_timer_elapsed_us(void); 1369 1370int tst_timer_expired_ms(long long ms); 1371------------------------------------------------------------------------------- 1372 1373The 'tst_timer_check()' function checks if specified 'clk_id' is supported and 1374exits the test with 'TCONF' otherwise. It's expected to be used in test 1375'setup()' before any resources that needs to be cleaned up are initialized, 1376hence it does not include a cleanup function parameter. 1377 1378The 'tst_timer_start()' marks start time and stores the 'clk_id' for further 1379use. 1380 1381The 'tst_timer_stop()' marks the stop time using the same 'clk_id' as last 1382call to 'tst_timer_start()'. 1383 1384The 'tst_timer_elapsed*()' returns time difference between the timer start and 1385last timer stop in several formats and units. 1386 1387The 'tst_timer_expired_ms()' function checks if the timer started by 1388'tst_timer_start()' has been running longer than ms milliseconds. The function 1389returns non-zero if timer has expired and zero otherwise. 1390 1391IMPORTANT: The timer functions use 'clock_gettime()' internally which needs to 1392 be linked with '-lrt' on older glibc. Please do not forget to add 1393 'LDLIBS+=-lrt' in Makefile. 1394 1395[source,c] 1396------------------------------------------------------------------------------- 1397#include "tst_test.h" 1398#include "tst_timer.h" 1399 1400static void setup(void) 1401{ 1402 ... 1403 tst_timer_check(CLOCK_MONOTONIC); 1404 ... 1405} 1406 1407static void run(void) 1408{ 1409 ... 1410 tst_timer_start(CLOCK_MONOTONIC); 1411 ... 1412 while (!tst_timer_expired_ms(5000)) { 1413 ... 1414 } 1415 ... 1416} 1417 1418struct tst_test test = { 1419 ... 1420 .setup = setup, 1421 .test_all = run, 1422 ... 1423}; 1424------------------------------------------------------------------------------- 1425 1426Expiration timer example usage. 1427 1428[source,c] 1429------------------------------------------------------------------------------- 1430long long tst_timespec_to_us(struct timespec t); 1431long long tst_timespec_to_ms(struct timespec t); 1432 1433struct timeval tst_us_to_timeval(long long us); 1434struct timeval tst_ms_to_timeval(long long ms); 1435 1436int tst_timespec_lt(struct timespec t1, struct timespec t2); 1437 1438struct timespec tst_timespec_add_us(struct timespec t, long long us); 1439 1440struct timespec tst_timespec_diff(struct timespec t1, struct timespec t2); 1441long long tst_timespec_diff_us(struct timespec t1, struct timespec t2); 1442long long tst_timespec_diff_ms(struct timespec t1, struct timespec t2); 1443 1444struct timespec tst_timespec_abs_diff(struct timespec t1, struct timespec t2); 1445long long tst_timespec_abs_diff_us(struct timespec t1, struct timespec t2); 1446long long tst_timespec_abs_diff_ms(struct timespec t1, struct timespec t2); 1447------------------------------------------------------------------------------- 1448 1449The first four functions are simple inline conversion functions. 1450 1451The 'tst_timespec_lt()' function returns non-zero if 't1' is earlier than 1452't2'. 1453 1454The 'tst_timespec_add_us()' function adds 'us' microseconds to the timespec 1455't'. The 'us' is expected to be positive. 1456 1457The 'tst_timespec_diff*()' functions returns difference between two times, the 1458't1' is expected to be later than 't2'. 1459 1460The 'tst_timespec_abs_diff*()' functions returns absolute value of difference 1461between two times. 1462 1463NOTE: All conversions to ms and us rounds the value. 1464 14651.22 Datafiles 1466~~~~~~~~~~~~~~ 1467 1468[source,c] 1469------------------------------------------------------------------------------- 1470#include "tst_test.h" 1471 1472static const char *const res_files[] = { 1473 "foo", 1474 "bar", 1475 NULL 1476}; 1477 1478static struct tst_test test = { 1479 ... 1480 .resource_files = res_files, 1481 ... 1482} 1483------------------------------------------------------------------------------- 1484 1485If the test needs additional files to be copied to the test temporary 1486directory all you need to do is to list their filenames in the 1487'NULL' terminated array '.resource_files' in the tst_test structure. 1488 1489When resource files is set test temporary directory is created automatically, 1490there is need to set '.needs_tmpdir' as well. 1491 1492The test library looks for datafiles first, these are either stored in a 1493directory called +datafiles+ in the +$PWD+ at the start of the test or in 1494+$LTPROOT/testcases/data/${test_binary_name}+. If the file is not found the 1495library looks into +$LTPROOT/testcases/bin/+ and to +$PWD+ at the start of the 1496test. This ensures that the testcases can copy the file(s) effortlessly both 1497when test is started from the directory it was compiled in as well as when LTP 1498was installed. 1499 1500The file(s) are copied to the newly created test temporary directory which is 1501set as the test working directory when the 'test()' functions is executed. 1502 15031.23 Code path tracing 1504~~~~~~~~~~~~~~~~~~~~~~ 1505 1506'tst_res' is a macro, so on when you define a function in one file: 1507 1508[source,c] 1509------------------------------------------------------------------------------- 1510int do_action(int arg) 1511{ 1512 ... 1513 1514 if (ok) { 1515 tst_res(TPASS, "check passed"); 1516 return 0; 1517 } else { 1518 tst_res(TFAIL, "check failed"); 1519 return -1; 1520 } 1521} 1522------------------------------------------------------------------------------- 1523 1524and call it from another file, the file and line reported by 'tst_res' in this 1525function will be from the former file. 1526 1527'TST_TRACE' can make the analysis of such situations easier. It's a macro which 1528inserts a call to 'tst_res(TINFO, ...)' in case its argument evaluates to 1529non-zero. In this call to 'tst_res(TINFO, ...)' the file and line will be 1530expanded using the actual location of 'TST_TRACE'. 1531 1532For example, if this another file contains: 1533 1534[source,c] 1535------------------------------------------------------------------------------- 1536#include "tst_test.h" 1537 1538if (TST_TRACE(do_action(arg))) { 1539 ... 1540} 1541------------------------------------------------------------------------------- 1542 1543the generated output may look similar to: 1544 1545------------------------------------------------------------------------------- 1546common.h:9: FAIL: check failed 1547test.c:8: INFO: do_action(arg) failed 1548------------------------------------------------------------------------------- 1549 15501.24 Tainted kernels 1551~~~~~~~~~~~~~~~~~~~~ 1552 1553If you need to detect whether a testcase triggers a kernel warning, bug or 1554oops, the following can be used to detect TAINT_W or TAINT_D: 1555 1556[source,c] 1557------------------------------------------------------------------------------- 1558#include "tst_test.h" 1559 1560static struct tst_test test = { 1561 ... 1562 .taint_check = TST_TAINT_W | TST_TAINT_D, 1563 ... 1564}; 1565 1566void run(void) 1567{ 1568 ... 1569 if (tst_taint_check() != 0) 1570 tst_res(TFAIL, "kernel has issues"); 1571 else 1572 tst_res(TPASS, "kernel seems to be fine"); 1573} 1574------------------------------------------------------------------------------- 1575 1576To initialize taint checks, you have to set the taint flags you want to test 1577for in the 'taint_check' attribute of the tst_test struct. LTP library will 1578then automatically call 'tst_taint_init()' during test setup. The function 1579will generate a 'TCONF' if the requested flags are not fully supported on the 1580running kernel, and 'TBROK' if the kernel is already tainted before executing 1581the test. 1582 1583LTP library will then automatically check kernel taint at the end of testing. 1584If '.all_filesystems' is set in struct tst_test, taint check will be performed 1585after each file system and taint will abort testing early with 'TFAIL'. You 1586can optionally also call 'tst_taint_check()' during 'run()', which returns 0 1587or the tainted flags set in '/proc/sys/kernel/tainted' as specified earlier. 1588 1589Depending on your kernel version, not all tainted-flags will be supported. 1590 1591For reference to tainted kernels, see kernel documentation: 1592Documentation/admin-guide/tainted-kernels.rst or 1593https://www.kernel.org/doc/html/latest/admin-guide/tainted-kernels.html 1594 15951.25 Checksums 1596~~~~~~~~~~~~~~ 1597 1598CRC32c checksum generation is supported by LTP. In order to use it, the 1599test should include 'tst_checksum.h' header, then can call 'tst_crc32c()'. 1600 16011.26 Checking kernel for the driver support 1602~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1603 1604Some tests may need specific kernel drivers, either compiled in, or built 1605as a module. If '.needs_drivers' points to a 'NULL' terminated array of kernel 1606module names these are all checked and the test exits with 'TCONF' on the 1607first missing driver. 1608 1609The detection is based on reading 'modules.dep' and 'modules.builtin' files 1610generated by kmod. The check is skipped on Android. 1611 16121.27 Saving & restoring /proc|sys values 1613~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1614 1615LTP library can be instructed to save and restore value of specified 1616(/proc|sys) files. This is achieved by initialized tst_test struct 1617field 'save_restore'. It is a NULL-terminated array of struct 1618'tst_path_val' where each tst_path_val.path represents a file, whose 1619value is saved at the beginning and restored at the end of the test. 1620If non-NULL string is passed in tst_path_val.val, it is written 1621to the respective file at the beginning of the test. Only the first line 1622of a specified file is saved and restored. 1623 1624By default, the test will end with TCONF if the file is read-only or 1625does not exist. If the optional write of new value fails, the test will end 1626with 'TBROK'. This behavior can be changed using tst_path_val.flags: 1627 1628* 'TST_SR_TBROK_MISSING' – End test with 'TBROK' if the file does not exist 1629* 'TST_SR_TCONF_MISSING' – End test with 'TCONF' if the file does not exist 1630* 'TST_SR_SKIP_MISSING' – Continue without saving the file if it does not exist 1631* 'TST_SR_TBROK_RO' – End test with 'TBROK' if the file is read-only 1632* 'TST_SR_TCONF_RO' – End test with 'TCONF' if the file is read-only 1633* 'TST_SR_SKIP_RO' – Continue without saving the file if it is read-only 1634* 'TST_SR_IGNORE_ERR' – Ignore errors when writing new value into the file 1635 1636Common flag combinations also have shortcuts: 1637 1638* 'TST_SR_TCONF' – Equivalent to 'TST_SR_TCONF_MISSING | TST_SR_TCONF_RO' 1639* 'TST_SR_TBROK' – Equivalent to 'TST_SR_TBROK_MISSING | TST_SR_TBROK_RO' 1640* 'TST_SR_SKIP' – Equivalent to 'TST_SR_SKIP_MISSING | TST_SR_SKIP_RO' 1641 1642'restore' is always strict and will TWARN if it encounters any error. 1643 1644[source,c] 1645------------------------------------------------------------------------------- 1646static struct tst_test test = { 1647 ... 1648 .setup = setup, 1649 .save_restore = (const struct tst_path_val[]) { 1650 {"/proc/sys/kernel/core_pattern", NULL, TST_SR_TCONF}, 1651 {"/proc/sys/user/max_user_namespaces", NULL, TST_SR_SKIP}, 1652 {"/sys/kernel/mm/ksm/run", "1", TST_SR_TBROK}, 1653 {} 1654 }, 1655}; 1656------------------------------------------------------------------------------- 1657 16581.28 Parsing kernel .config 1659~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1660 1661Generally testcases should attempt to autodetect as much kernel features as 1662possible based on the currently running kernel. We do have tst_check_driver() 1663to check if functionality that could be compiled as kernel module is present 1664on the system, disabled syscalls can be detected by checking for 'ENOSYS' 1665errno etc. 1666 1667However in rare cases core kernel features couldn't be detected based on the 1668kernel userspace API and we have to resort to parse the kernel .config. 1669 1670For this cases the test should set the 'NULL' terminated '.needs_kconfigs' 1671array of boolean expressions with constraints on the kconfig variables. The 1672boolean expression consits of variables, two binary operations '&' and '|', 1673negation '!' and correct sequence of parentesis '()'. Variables are expected 1674to be in a form of "CONFIG_FOO[=bar]". 1675 1676The test will continue to run if all expressions are evaluated to 'True'. 1677Missing variable is mapped to 'False' as well as variable with different than 1678specified value, e.g. 'CONFIG_FOO=bar' will evaluate to 'False' if the value 1679is anything else but 'bar'. If config variable is specified as plain 1680'CONFIG_FOO' it's evaluated to true it's set to any value (typically =y or =m). 1681 1682[source,c] 1683------------------------------------------------------------------------------- 1684#include "tst_test.h" 1685 1686static const char *kconfigs[] = { 1687 "CONFIG_X86_INTEL_UMIP | CONFIG_X86_UMIP", 1688 NULL 1689}; 1690 1691static struct tst_test test = { 1692 ... 1693 .needs_kconfigs = kconfigs, 1694 ... 1695}; 1696------------------------------------------------------------------------------- 1697 16981.29 Changing the Wall Clock Time during test execution 1699~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1700 1701There are some tests that, for different reasons, might need to change the 1702system-wide clock time. Whenever this happens, it is imperative that the clock 1703is restored, at the end of test's execution, taking in consideration the amount 1704of time elapsed during that test. 1705 1706In order for that to happen, struct tst_test has a variable called 1707"restore_wallclock" that should be set to "1" so LTP knows it should: (1) 1708initialize a monotonic clock during test setup phase and (2) use that monotonic 1709clock to fix the system-wide clock time at the test cleanup phase. 1710 1711[source,c] 1712------------------------------------------------------------------------------- 1713#include "tst_test.h" 1714 1715static void setup(void) 1716{ 1717 ... 1718} 1719 1720static void run(void) 1721{ 1722 ... 1723} 1724 1725struct tst_test test = { 1726 ... 1727 .setup = setup, 1728 .test_all = run, 1729 .restore_wallclock = 1, 1730 ... 1731}; 1732------------------------------------------------------------------------------- 1733 17341.30 Testing similar syscalls in one test 1735~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1736 1737In some cases kernel has several very similar syscalls that do either the same 1738or very similar job. This is most noticeable on i386 where we commonly have 1739two or three syscall versions. That is because i386 was first platform that 1740Linux was developed on and because of that most mistakes in API happened there 1741as well. However this is not limited to i386 at all, it's quite common that 1742version two syscall has added missing flags parameters or so. 1743 1744In such cases it does not make much sense to copy&paste the test code over and 1745over, rather than that the test library provides support for test variants. 1746The idea behind test variants is simple, we run the test several times each 1747time with different syscall variant. 1748 1749The implementation consist of test_variants integer that, if set, denotes number 1750of test variants. The test is then forked and executed test_variants times each 1751time with different value in global tst_variant variable. 1752 1753[source,c] 1754------------------------------------------------------------------------------- 1755#include "tst_test.h" 1756 1757static int do_foo(void) 1758{ 1759 switch (tst_variant) { 1760 case 0: 1761 return foo(); 1762 case 1: 1763 return syscall(__NR_foo); 1764 } 1765 1766 return -1; 1767} 1768 1769static void run(void) 1770{ 1771 ... 1772 1773 TEST(do_foo); 1774 1775 ... 1776} 1777 1778static void setup(void) 1779{ 1780 switch (tst_variant) { 1781 case 0: 1782 tst_res(TINFO, "Testing foo variant 1"); 1783 break; 1784 case 1: 1785 tst_res(TINFO, "Testing foo variant 2"); 1786 break; 1787 } 1788} 1789 1790struct tst_test test = { 1791 ... 1792 .setup = setup, 1793 .test_all = run, 1794 .test_variants = 2, 1795 ... 1796}; 1797------------------------------------------------------------------------------- 1798 17991.31 Guarded buffers 1800~~~~~~~~~~~~~~~~~~~~ 1801 1802The test library supports guarded buffers, which are buffers allocated so 1803that: 1804 1805* The end of the buffer is followed by a PROT_NONE page 1806 1807* The remainder of the page before the buffer is filled with random canary 1808 data 1809 1810Which means that the any access after the buffer will yield a Segmentation 1811fault or EFAULT depending on if the access happened in userspace or the kernel 1812respectively. The canary before the buffer will also catch any write access 1813outside of the buffer. 1814 1815The purpose of the patch is to catch off-by-one bugs which happens when 1816buffers and structures are passed to syscalls. New tests should allocate 1817guarded buffers for all data passed to the tested syscall which are passed by 1818a pointer. 1819 1820[source,c] 1821------------------------------------------------------------------------------- 1822#include "tst_test.h" 1823 1824static struct foo *foo_ptr; 1825static struct iovec *iov; 1826static void *buf_ptr; 1827static char *id; 1828... 1829 1830static void run(void) 1831{ 1832 ... 1833 1834 foo_ptr->bar = 1; 1835 foo_ptr->buf = buf_ptr; 1836 1837 ... 1838} 1839 1840static void setup(void) 1841{ 1842 ... 1843 1844 id = tst_strdup(string); 1845 1846 ... 1847} 1848 1849static struct tst_test test = { 1850 ... 1851 .bufs = (struct tst_buffers []) { 1852 {&foo_ptr, .size = sizeof(*foo_ptr)}, 1853 {&buf_ptr, .size = BUF_SIZE}, 1854 {&iov, .iov_sizes = (int[]){128, 32, -1}, 1855 {} 1856 } 1857}; 1858------------------------------------------------------------------------------- 1859 1860Guarded buffers can be allocated on runtime in a test setup() by a 1861'tst_alloc()' or by 'tst_strdup()' as well as by filling up the .bufs array in 1862the tst_test structure. 1863 1864So far the tst_test structure supports allocating either a plain buffer by 1865setting up the size or struct iovec, which is allocated recursively including 1866the individual buffers as described by an '-1' terminated array of buffer 1867sizes. 1868 18691.32 Adding and removing capabilities 1870~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1871 1872Some tests may require the presence or absence of particular 1873capabilities. Using the API provided by 'tst_capability.h' the test author can 1874try to ensure that some capabilities are either present or absent during the 1875test. 1876 1877For example; below we try to create a raw socket, which requires 1878CAP_NET_ADMIN. During setup we should be able to do it, then during run it 1879should be impossible. The LTP capability library will check before setup that 1880we have this capability, then after setup it will drop it. 1881 1882[source,c] 1883-------------------------------------------------------------------------------- 1884#include "tst_test.h" 1885#include "tst_capability.h" 1886#include "tst_safe_net.h" 1887 1888#include "lapi/socket.h" 1889 1890static void run(void) 1891{ 1892 TEST(socket(AF_INET, SOCK_RAW, 1)); 1893 if (TST_RET > -1) { 1894 tst_res(TFAIL, "Created raw socket"); 1895 } else if (TST_ERR != EPERM) { 1896 tst_res(TFAIL | TTERRNO, 1897 "Failed to create socket for wrong reason"); 1898 } else { 1899 tst_res(TPASS | TTERRNO, "Didn't create raw socket"); 1900 } 1901} 1902 1903static void setup(void) 1904{ 1905 TEST(socket(AF_INET, SOCK_RAW, 1)); 1906 if (TST_RET < 0) 1907 tst_brk(TCONF | TTERRNO, "We don't have CAP_NET_RAW to begin with"); 1908 1909 SAFE_CLOSE(TST_RET); 1910} 1911 1912static struct tst_test test = { 1913 .setup = setup, 1914 .test_all = run, 1915 .caps = (struct tst_cap []) { 1916 TST_CAP(TST_CAP_REQ, CAP_NET_RAW), 1917 TST_CAP(TST_CAP_DROP, CAP_NET_RAW), 1918 {} 1919 }, 1920}; 1921-------------------------------------------------------------------------------- 1922 1923Look at the test struct at the bottom. We have filled in the 'caps' field with 1924a 'NULL' terminated array containing two 'tst_cap' structs. 'TST_CAP_REQ' 1925actions are executed before setup and 'TST_CAP_DROP' are executed after 1926setup. This means it is possible to both request and drop a capability. 1927 1928[source,c] 1929-------------------------------------------------------------------------------- 1930static struct tst_test test = { 1931 .test_all = run, 1932 .caps = (struct tst_cap []) { 1933 TST_CAP(TST_CAP_REQ, CAP_NET_RAW), 1934 TST_CAP(TST_CAP_DROP, CAP_SYS_ADMIN), 1935 {} 1936 }, 1937}; 1938-------------------------------------------------------------------------------- 1939 1940Here we request 'CAP_NET_RAW', but drop 'CAP_SYS_ADMIN'. If the capability is 1941in the permitted set, but not the effective set, the library will try to 1942permit it. If it is not in the permitted set, then it will fail with 'TCONF'. 1943 1944This API does not require 'libcap' to be installed. However it has limited 1945features relative to 'libcap'. It only tries to add or remove capabilities 1946from the effective set. This means that tests which need to spawn child 1947processes may have difficulties ensuring the correct capabilities are 1948available to the children (see the capabilities (7) manual pages). 1949 1950However a lot of problems can be solved by using 'tst_cap_action(struct 1951tst_cap *cap)' directly which can be called at any time. This also helps if 1952you wish to drop a capability at the beginning of setup. 1953 19541.33 Reproducing race-conditions 1955~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1956 1957If a bug is caused by two tasks in the kernel racing and you wish to create a 1958regression test (or bug-fix validation test) then the 'tst_fuzzy_sync.h' 1959library should be used. 1960 1961It allows you to specify, in your code, two race windows. One window in each 1962thread's loop (triggering a race usually requires many iterations). These 1963windows show fuzzy-sync where the race can happen. They don't need to be 1964exact, hence the 'fuzzy' part. If the race condition is not immediately 1965triggered then the library will begin experimenting with different timings. 1966 1967[source,c] 1968-------------------------------------------------------------------------------- 1969#include "tst_fuzzy_sync.h" 1970 1971static struct tst_fzsync_pair fzsync_pair; 1972 1973static void setup(void) 1974{ 1975 tst_fzsync_pair_init(&fzsync_pair); 1976} 1977 1978static void cleanup(void) 1979{ 1980 tst_fzsync_pair_cleanup(&fzsync_pair); 1981} 1982 1983static void *thread_b(void *arg) 1984{ 1985 while (tst_fzsync_run_b(&fzsync_pair)) { 1986 1987 tst_fzsync_start_race_b(&fzsync_pair); 1988 1989 /* This is the race window for thread B */ 1990 1991 tst_fzsync_end_race_b(&fzsync_pair); 1992 } 1993 1994 return arg; 1995} 1996 1997static void thread_a(void) 1998{ 1999 tst_fzsync_pair_reset(&fzsync_pair, thread_b); 2000 2001 while (tst_fzsync_run_a(&fzsync_pair)) { 2002 2003 tst_fzsync_start_race_a(&fzsync_pair); 2004 2005 /* This is the race window for thread A */ 2006 2007 tst_fzsync_end_race_a(&fzsync_pair); 2008 } 2009} 2010 2011static struct tst_test test = { 2012 .test_all = thread_a, 2013 .setup = setup, 2014 .cleanup = cleanup, 2015}; 2016-------------------------------------------------------------------------------- 2017 2018Above is a minimal template for a test using fuzzy-sync. In a simple case, you 2019just need to put the bits you want to race inbetween 'start_race' and 2020'end_race'. Meanwhile, any setup you need to do per-iteration goes outside the 2021windows. 2022 2023Fuzzy sync synchronises 'run_a' and 'run_b', which act as barriers, so that 2024neither thread can progress until the other has caught up with it. There is 2025also the 'pair_wait' function which can be used to add barriers in other 2026locations. Of course 'start/end_race_a/b' are also a barriers. 2027 2028The library decides how long the test should run for based on the timeout 2029specified by the user plus some other heuristics. 2030 2031For full documentation see the comments in 'include/tst_fuzzy_sync.h'. 2032 20331.34 Reserving hugepages 2034~~~~~~~~~~~~~~~~~~~~~~~~ 2035 2036Many of the LTP tests need to use hugepage in their testing, this allows the 2037test can reserve hugepages from system via '.hugepages = {xx, TST_REQUEST}'. 2038 2039We achieved two policies for reserving hugepages: 2040 2041TST_REQUEST: 2042 It will try the best to reserve available huge pages and return the number 2043 of available hugepages in tst_hugepages, which may be 0 if hugepages are 2044 not supported at all. 2045 2046TST_NEEDS: 2047 This is an enforced requirement, LTP should strictly do hpages applying and 2048 guarantee the 'HugePages_Free' no less than pages which makes that test can 2049 use these specified numbers correctly. Otherwise, test exits with TCONF if 2050 the attempt to reserve hugepages fails or reserves less than requested. 2051 2052With success test stores the reserved hugepage number in 'tst_hugepages'. For 2053system without hugetlb supporting, variable 'tst_hugepages' will be set to 0. 2054If the hugepage number needs to be set to 0 on supported hugetlb system, please 2055use '.hugepages = {TST_NO_HUGEPAGES}'. 2056 2057Also, we do cleanup and restore work for the hpages resetting automatically. 2058 2059[source,c] 2060------------------------------------------------------------------------------- 2061#include "tst_test.h" 2062 2063static void run(void) 2064{ 2065 ... 2066 2067 if (tst_hugepages == test.hugepages.number) 2068 TEST(do_hpage_test); 2069 else 2070 ... 2071 ... 2072} 2073 2074struct tst_test test = { 2075 .test_all = run, 2076 .hugepages = {2, TST_REQUEST}, 2077 ... 2078}; 2079------------------------------------------------------------------------------- 2080 2081or, 2082 2083[source,c] 2084------------------------------------------------------------------------------- 2085#include "tst_test.h" 2086 2087static void run(void) 2088{ 2089 ... 2090} 2091 2092static void setup(void) 2093{ 2094 /* TST_NEEDS achieved this automatically in the library */ 2095 if (tst_hugepages != test.hugepages.number) 2096 tst_brk(TCONF, "..."); 2097} 2098 2099struct tst_test test = { 2100 .test_all = run, 2101 .hugepages = {2, TST_NEEDS}, 2102 ... 2103}; 2104------------------------------------------------------------------------------- 2105 21061.35 Checking for required commands 2107~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2108 2109Required commands can be checked with '.needs_cmds', which points to a 'NULL' 2110terminated array of strings such as: 2111 2112[source,c] 2113------------------------------------------------------------------------------- 2114.needs_cmds = (const char *const []) { 2115 "useradd", 2116 "userdel", 2117 NULL 2118}, 2119------------------------------------------------------------------------------- 2120 2121Also can check required command version whether is satisfied by using 'needs_cmds' 2122such as: 2123 2124[source,c] 2125------------------------------------------------------------------------------- 2126.needs_cmds = (const char *const []) { 2127 "mkfs.ext4 >= 1.43.0", 2128 NULL 2129}, 2130------------------------------------------------------------------------------- 2131 2132Currently, we only support mkfs.ext4 command version check. 2133If you want to support more commands, please fill your own .parser and .table_get 2134method in the version_parsers structure of lib/tst_cmd.c. 2135 21361.36 Assert sys or proc file value 2137~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2138Using TST_ASSERT_INT/STR(path, val) to assert that integer value or string stored in 2139the prefix field of file pointed by path equals to the value passed to this function. 2140 2141Also having a similar api pair TST_ASSERT_FILE_INT/STR(path, prefix, val) to assert 2142the field value of file. 2143 21441.37 Using Control Group 2145~~~~~~~~~~~~~~~~~~~~~~~~ 2146 2147Some LTP tests need specific Control Group configurations. 'tst_cgroup.h' 2148provides APIs to discover and use CGroups. There are many differences between 2149CGroups API V1 and V2. We encapsulate the details of configuring CGroups in 2150high-level functions which follow the V2 kernel API where possible. Allowing one 2151to write code that works on both V1 or V2. At least some of the time anyway; 2152often the behavioural differences between V1 and V2 are too great. In such cases 2153we revert to branching on the CGroup version. 2154 2155Also, the LTP library will automatically mount/umount and configure the CGroup 2156hierarchies if that is required (e.g. if you run the tests from init with no 2157system manager). 2158 2159[source,c] 2160------------------------------------------------------------------------------- 2161#include "tst_test.h" 2162 2163static void run(void) 2164{ 2165 ... 2166 // do test under cgroup 2167 ... 2168} 2169 2170static void setup(void) 2171{ 2172 SAFE_CG_PRINTF(tst_cg, "cgroup.procs", "%d", getpid()); 2173 SAFE_CG_PRINTF(tst_cg, "memory.max", "%lu", MEMSIZE); 2174 if (SAFE_CG_HAS(tst_cg, "memory.swap.max")) 2175 SAFE_CG_PRINTF(tst_cg, "memory.swap.max", "%zu", memsw); 2176} 2177 2178struct tst_test test = { 2179 .setup = setup, 2180 .test_all = run, 2181 .cleanup = cleanup, 2182 .needs_cgroup_ctrls = (const char *const []){ "memory", NULL }, 2183 ... 2184}; 2185------------------------------------------------------------------------------- 2186 2187Above, we first ensure the memory controller is available on the 2188test's CGroup with '.needs_cgroup_ctrls'. This populates a structure, 2189'tst_cg', which represents the test's CGroup. 2190 2191We then write the current processes PID into 'cgroup.procs', which 2192moves the current process into the test's CGroup. After which we set 2193the maximum memory size by writing to 'memory.max'. If the memory 2194controller is mounted on CGroups V1 then the library will actually 2195write to 'memory.limit_in_bytes'. As a general rule, if a file exists 2196on both CGroup versions, then we use the V2 naming. 2197 2198Some controller features, such as 'memory.swap', can be 2199disabled. Therefor we need to check if they exist before accessing 2200them. This can be done with 'SAFE_CG_HAS' which can be called on 2201any control file or feature. 2202 2203Most tests only require setting a few limits similar to the above. In 2204such cases the differences between V1 and V2 are hidden. Setup and 2205cleanup is also mostly hidden. However things can get much worse. 2206 2207[source,c] 2208------------------------------------------------------------------------------- 2209static struct tst_cg_group *cg_child; 2210 2211static void run(void) 2212{ 2213 char buf[BUFSIZ]; 2214 size_t mem = 0; 2215 2216 cg_child = tst_cg_group_mk(tst_cg, "child"); 2217 SAFE_CG_PRINTF(cg_child, "cgroup.procs", "%d", getpid()); 2218 2219 if (!TST_CG_VER_IS_V1(tst_cg, "memory")) 2220 SAFE_CG_PRINT(tst_cg, "cgroup.subtree_control", "+memory"); 2221 if (!TST_CG_VER_IS_V1(tst_cg, "cpuset")) 2222 SAFE_CG_PRINT(tst_cg, "cgroup.subtree_control", "+cpuset"); 2223 2224 if (!SAFE_FORK()) { 2225 SAFE_CG_PRINTF(cg_child, "cgroup.procs", "%d", getpid()); 2226 2227 if (SAFE_CG_HAS(cg_child, "memory.swap")) { 2228 SAFE_CG_SCANF(cg_child, 2229 "memory.swap.current", "%zu", &mem); 2230 } 2231 SAFE_CG_READ(cg_child, "cpuset.mems", buf, sizeof(buf)); 2232 2233 // Do something with cpuset.mems and memory.current values 2234 ... 2235 2236 exit(0); 2237 } 2238 2239 tst_reap_children(); 2240 SAFE_CG_PRINTF(tst_cg_drain, "cgroup.procs", "%d", getpid()); 2241 cg_child = tst_cg_group_rm(cg_child); 2242} 2243 2244static void cleanup(void) 2245{ 2246 if (cg_child) { 2247 SAFE_CG_PRINTF(tst_cg_drain, "cgroup.procs", "%d", getpid()); 2248 cg_child = tst_cg_group_rm(cg_child); 2249 } 2250} 2251 2252struct tst_test test = { 2253 .setup = setup, 2254 .test_all = run, 2255 .needs_cgroup_ctrls = (const char *const []){ 2256 "cpuset", 2257 "memory", 2258 NULL 2259 }, 2260 ... 2261}; 2262------------------------------------------------------------------------------- 2263 2264Starting with setup; we can see here that we fetch the 'drain' 2265CGroup. This is a shared group (between parallel tests) which may 2266contain processes from other tests. It should have default settings 2267and these should not be changed by the test. It can be used to remove 2268processes from other CGroups incase the hierarchy root is not 2269accessible. 2270 2271Note that 'tst_cg_get_drain_group' should not be called many times, 2272as it is allocated in a guarded buffer (See section 2.2.31). Therefor 2273it is best to call it once in 'setup' and not 'run' because 'run' may 2274be repeated with the '-i' option. 2275 2276In 'run', we first create a child CGroup with 'tst_cg_mk'. As we 2277create this CGroup in 'run' we should also remove it at the end of 2278run. We also need to check if it exists and remove it in cleanup as 2279well. Because there are 'SAFE_' functions which may jump to cleanup. 2280 2281We then move the main test process into the child CGroup. This is 2282important as it means that before we destroy the child CGroup we have 2283to move the main test process elsewhere. For that we use the 'drain' 2284group. 2285 2286Next we enable the memory and cpuset controller configuration on the 2287test CGroup's descendants (i.e. 'cg_child'). This allows each child to 2288have its own settings. The file 'cgroup.subtree_control' does not 2289exist on V1. Because it is possible to have both V1 and V2 active at 2290the same time. We can not simply check if 'subtree_control' exists 2291before writing to it. We have to check if a particular controller is 2292on V2 before trying to add it to 'subtree_control'. Trying to add a V1 2293controller will result in 'ENOENT'. 2294 2295We then fork a child process and add this to the child CGroup. Within 2296the child process we try to read 'memory.swap.current'. It is possible 2297that the memory controller was compiled without swap support, so it is 2298necessary to check if 'memory.swap' is enabled. That is unless the 2299test will never reach the point where 'memory.swap.*' are used without 2300swap support. 2301 2302The parent process waits for the child process to be reaped before 2303destroying the child CGroup. So there is no need to transfer the child 2304to drain. However the parent process must be moved otherwise we will 2305get 'EBUSY' when trying to remove the child CGroup. 2306 2307Another example of a behavioral difference between versions is shown below. 2308 2309[source,c] 2310------------------------------------------------------------------------------- 2311 if (TST_CG_VER_IS_V1(tst_cg, "memory")) 2312 SAFE_CG_PRINTF(tst_cg, "memory.swap.max", "%lu", ~0UL); 2313 else 2314 SAFE_CG_PRINT(tst_cg, "memory.swap.max", "max"); 2315------------------------------------------------------------------------------- 2316 2317CGroups V2 introduced a feature where 'memory[.swap].max' could be set to 2318"max". This does not appear to work on V1 'limit_in_bytes' however. For most 2319tests, simply using a large number is sufficient and there is no need to use 2320"max". Importantly though, one should be careful to read both the V1 and V2 2321kernel docs. Presently the LTP library does not attempt to handle most 2322differences in semantics. It does the minimal amount of work to make testing on 2323both V1 and V2 feasible. 2324 23251.38 Require minimum numbers of CPU for a testcase 2326~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2327 2328Some tests require more than specific number of CPU. It can be defined with 2329`.min_cpus = N`. 2330 23311.39 Require minimum memory or swap size for a testcase 2332~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2333 2334Some tests require at least size(MB) of free RAM or Swap. 2335 2336To make sure that test will run only on systems with more than minimal 2337required amount of RAM set `.min_mem_avail = N`. 2338 2339Similarily for tests that require certain amount of free Swap use 2340`.min_swap_avail = N`. 2341 23421.40 Test tags 2343~~~~~~~~~~~~~~ 2344 2345Test tags are name-value pairs that can hold any test metadata. 2346 2347We have additional support for CVE entries, git commit in mainline kernel, 2348stable kernel or glibc git repository. If a test is a regression test it 2349should include these tags. They are printed when test fails and exported 2350into documentation. 2351 2352CVE, mainline and stable kernel git commits in a regression test for a kernel bug: 2353[source,c] 2354------------------------------------------------------------------------------- 2355struct tst_test test = { 2356 ... 2357 .tags = (const struct tst_tag[]) { 2358 {"linux-git", "9392a27d88b9"}, 2359 {"linux-git", "ff002b30181d"}, 2360 {"known-fail", "ustat() is known to fail with EINVAL on Btrfs"}, 2361 {"linux-stable-git", "c4a23c852e80"}, 2362 {"CVE", "2020-29373"}, 2363 {} 2364 } 2365}; 2366------------------------------------------------------------------------------- 2367 2368NOTE: We don't track all backports to stable kernel but just those which are 2369 stable branch specific (unique), i.e. no commit in mainline. Example of 2370 commits: c4a23c852e80, cac68d12c531. 2371 2372Glibc and musl git commits in a regression test for glibc and musl bugs: 2373[source,c] 2374------------------------------------------------------------------------------- 2375struct tst_test test = { 2376 ... 2377 .tags = (const struct tst_tag[]) { 2378 {"glibc-git", "574500a108be"}, 2379 {"musl-git", "fa4a8abd06a4"}, 2380 {} 2381 } 2382}; 2383------------------------------------------------------------------------------- 2384 23851.41 Testing on the specific architecture 2386~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2387Testcases for specific arch should be limited on that only being supported 2388platform to run, we now involve a '.supported_archs' to achieve this feature 2389in LTP library. All you need to run a test on the expected arch is to set 2390the '.supported_archs' array in the 'struct tst_test' to choose the required 2391arch list. e.g. 2392 2393 .supported_archs = (const char *const []){"x86_64", "ppc64", NULL} 2394 2395This helps move the TCONF info from code to tst_test metadata as well. 2396 2397And, we also export a struct tst_arch to save the system architecture for 2398using in the whole test cases. 2399 2400 extern const struct tst_arch { 2401 char name[16]; 2402 enum tst_arch_type type; 2403 } tst_arch; 2404 2405[source,c] 2406------------------------------------------------------------------------------- 2407#include "tst_test.h" 2408 2409static struct tst_test test = { 2410 ... 2411 .setup = setup, 2412 .supported_archs = (const char *const []) { 2413 "x86_64", 2414 "ppc64", 2415 "s390x", 2416 NULL 2417 }, 2418}; 2419------------------------------------------------------------------------------- 2420 24211.42 Skipping test based on system state 2422~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2423Test can be skipped on various conditions: on enabled SecureBoot 2424('.skip_in_secureboot = 1'), lockdown ('.skip_in_lockdown = 1') or in 32-bit 2425compat mode ('.skip_in_compat = 1'). 2426 24272. Common problems 2428------------------ 2429 2430This chapter describes common problems/misuses and less obvious design patters 2431(quirks) in UNIX interfaces. Read it carefully :) 2432 24332.1 umask() 2434~~~~~~~~~~~ 2435 2436I've been hit by this one several times already... When you create files 2437with 'open()' or 'creat()' etc, the mode specified as the last parameter *is 2438not* the mode the file is created with. The mode depends on current 'umask()' 2439settings which may clear some of the bits. If your test depends on specific 2440file permissions you need either to change umask to 0 or 'chmod()' the file 2441afterwards or use 'SAFE_TOUCH()' that does the 'chmod()' for you. 2442 24432.2 access() 2444~~~~~~~~~~~~ 2445 2446If 'access(some_file, W_OK)' is executed by root, it will return success even 2447if the file doesn't have write permission bits set (the same holds for R_OK 2448too). For sysfs files you can use 'open()' as a workaround to check file 2449read/write permissions. It might not work for other filesystems, for these you 2450have to use 'stat()', 'lstat()' or 'fstat()'. 2451 24522.3 umount() EBUSY 2453~~~~~~~~~~~~~~~~~~ 2454 2455Various desktop daemons (gvfsd-trash is known for that) may be stupid enough 2456to probe all newly mounted filesystem which results in 'umount(2)' failing 2457with 'EBUSY'; use 'tst_umount()' described in 1.19 that retries in this case 2458instead of plain 'umount(2)'. 2459 24602.4 FILE buffers and fork() 2461~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2462 2463Be vary that if a process calls 'fork(2)' the child process inherits open 2464descriptors as well as copy of the parent memory so especially if there are 2465any open 'FILE' buffers with a data in them they may be written both by the 2466parent and children resulting in corrupted/duplicated data in the resulting 2467files. 2468 2469Also open 'FILE' streams are flushed and closed at 'exit(3)' so if your 2470program works with 'FILE' streams, does 'fork(2)', and the child may end up 2471calling 'exit(3)' you will likely end up with corrupted files. 2472 2473The solution to this problem is either simply call 'fflush(NULL)' that flushes 2474all open output 'FILE' streams just before doing 'fork(2)'. You may also use 2475'_exit(2)' in child processes which does not flush 'FILE' buffers and also 2476skips 'atexit(3)' callbacks. 2477