1LTP Test Writing Guidelines 2=========================== 3 4This document describes LTP guidelines and LTP test interface and is intended 5for anybody who want to write or modify a LTP testcase. It's not a definitive 6guide and it's not, by any means, a substitute for common sense. 7 81. General Rules 9---------------- 10 111.1 Simplicity 12~~~~~~~~~~~~~~ 13 14For all it's worth keep the testcases simple or better as simple as possible. 15The kernel and libc are tricky beasts and the complexity imposed by their 16interfaces is quite high. Concentrate on the interface you want to test and 17follow the UNIX philosophy. It's a good idea to make the test as 18self-contained as possible too (it should not depend on tools or libraries 19that are not widely available). 20 21Do not reinvent the wheel! 22 23* Use LTP standard interface 24* Do not add custom PASS/FAIL reporting functions 25* Do not write Makefiles from scratch, 26 use LTP build system instead, etc. 27* ... 28 291.2 Code duplication 30~~~~~~~~~~~~~~~~~~~~ 31 32Copy & paste is a good servant but very poor master. If you are about to copy a 33large part of the code from one testcase to another, think what would happen if 34you find bug in the code that has been copied all around the tree. What about 35moving it to a library instead? 36 37The same goes for short but complicated parts, whenever you are about to copy & 38paste a syscall wrapper that packs arguments accordingly to machine 39architecture or similarly complicated code, put it into a header instead. 40 411.3 Coding style 42~~~~~~~~~~~~~~~~ 43 441.3.1 C coding style 45^^^^^^^^^^^^^^^^^^^^ 46 47LTP adopted Linux kernel coding style. If you aren't familiar with its rules 48locate 'linux/Documentation/CodingStyle' in the kernel sources and read it, 49it's a well written introduction. 50 51There is also a checkpatch (see 'linux/scripts/checkpatch.pl') script that can 52be used to check your patches before the submission. 53 54NOTE: If checkpatch does not report any problems, the code still may be wrong 55 as the tool only looks for common mistakes. 56 571.3.2 Shell coding style 58^^^^^^^^^^^^^^^^^^^^^^^^ 59 60When writing testcases in shell write in *portable shell* only, it's a good 61idea to try to run the test using alternative shell (alternative to bash, for 62example dash) too. 63 64*Portable shell* means Shell Command Language as defined by POSIX with a 65exception of few widely used extensions, namely 'local' keyword used inside of 66functions and '-o' and '-a' test parameters (that are marked as obsolete in 67POSIX). 68 69You can either try to run the testcases on Debian which has '/bin/sh' pointing 70to 'dash' by default or install 'dash' on your favorite distribution and use 71it to run the tests. If your distribution lacks 'dash' package you can always 72compile it from http://gondor.apana.org.au/~herbert/dash/files/[source]. 73 74Debian also has nice devscript 75https://anonscm.debian.org/cgit/collab-maint/devscripts.git/tree/scripts/checkbashisms.pl[checkbashism.pl] 76that can be used to check for non-portable shell code. 77 78Here are some common sense style rules for shell 79 80* Keep lines under 80 chars 81 82* Use tabs for indentation 83 84* Keep things simple, avoid unnecessary subshells 85 86* Don't do confusing things (i.e. don't name your functions like common shell 87 commands, etc.) 88 89* Quote variables 90 91* Be consistent 92 931.4 Commenting code 94~~~~~~~~~~~~~~~~~~~ 95 96Comments can sometimes save you day but they can easily do more harm than 97good. There has been several cases where comments and actual implementation 98were drifting slowly apart which yielded into API misuses and hard to find 99bugs. Remember there is only one thing worse than no documentation, wrong 100documentation. 101 102Generally everybody should write code that is obvious (which unfortunately 103isn't always possible). If there is a code that needs to be commented keep it 104short and to the point. Never ever comment the obvious. 105 106In case of LTP testcases it's customary to add a paragraph with highlevel test 107description somewhere at the beginning of the file (usually right under the GPL 108header). This helps other people to understand the overall goal of the test 109before they dive into the technical details. 110 1111.5 Backwards compatibility 112~~~~~~~~~~~~~~~~~~~~~~~~~~~ 113 114LTP test should be as backward compatible as possible. Think of an enterprise 115distributions with long term support (more than five years since the initial 116release) or of an embedded platform that needs to use several years old 117toolchain supplied by the manufacturer. 118 119Therefore LTP test for more current features should be able to cope with older 120systems. It should at least compile fine and if it's not appropriate for the 121configuration it should return 'TCONF' (see test interface description below). 122 123There are several types of checks we use: 124 125The *configure script* is usually used to detect availability of a function 126declarations in system headers. It's used to disable tests at compile time. 127 128We also have runtime kernel version detection that can be used to disable 129tests at runtime. 130 131Checking the *errno* value is another type of runtime check. Most of the 132syscalls returns either 'EINVAL' or 'ENOSYS' when syscall was not implemented 133or was disabled upon kernel compilation. 134 135Sometimes it also makes sense to define a few macros instead of creating 136configure test. One example are Linux specific POSIX clock ids in 137'include/lapi/posix_clocks.h'. 138 1391.6 Dealing with messed up legacy code 140~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 141 142LTP contains a lot of old and messy code and we are cleaning it up as fast as 143we can but despite the efforts there is still a lot. If you start modifying 144old or a messed up testcase and your changes are more complicated than simple 145typo fixes you should do a cleanup first (in a separate patch). It's easier to 146review the changes if you separate the formatting fixes from the changes that 147affects the test behavior. 148 149The same goes for moving files. If you need a rename or move file do it in a 150separate patch. 151 1521.7 License 153~~~~~~~~~~~ 154 155Code contributed to LTP should be licensed under GPLv2+ (GNU GPL version 2 or 156any later version). 157 1582. Writing a testcase 159--------------------- 160 1612.1 LTP Structure 162~~~~~~~~~~~~~~~~~ 163 164The structure of LTP is quite simple. Each test is a binary written either in 165portable shell or C. The test gets a configuration via environment variables 166and/or command line parameters, it prints additional information into the 167stdout and reports overall success/failure via the exit value. 168 169Tests are generally placed under the 'testcases/' directory. Everything that 170is a syscall or (slightly confusingly) libc syscall wrapper goes under 171'testcases/kernel/syscalls/'. Then there is 'testcases/open_posix_testsuite' 172which is a well maintained fork of the upstream project that has been dead 173since 2005 and also a number of directories with tests for more specific 174features. 175 1762.1.1 Runtest Files 177^^^^^^^^^^^^^^^^^^^ 178 179The list of tests to be executed is stored in runtest files under the 180'runtest/' directory. The default set of runtest files to be executed is 181stored in 'scenario_groups/default'. When you add a test you should add 182corresponding entries into some runtest file(s) as well. 183 184For syscall tests (these placed under 'testcases/kernel/syscalls/') use 185'runtest/syscalls' file, for kernel related tests for memory management we 186have 'runtest/mm', etc. 187 188IMPORTANT: The runtest files should have one entry per a test. Creating a 189 wrapper that runs all your tests and adding it as a single test 190 into runtest file is strongly discouraged. 191 1922.1.2 Datafiles 193^^^^^^^^^^^^^^^ 194 195If your test needs datafiles to work, these should be put into a subdirectory 196named 'datafiles' and installed into the 'testcases/data/$TCID' directory (to 197do that you have to add 'INSTALL_DIR := testcases/data/TCID' into the 198'datafiles/Makefile'). 199 200You can obtain path to datafiles via $TST_DATAROOT provided by test.sh 201'$TST_DATAROOT/...' 202or via C function 'tst_dataroot()' provided by libltp: 203 204[source,c] 205------------------------------------------------------------------------------- 206const char *dataroot = tst_dataroot(); 207------------------------------------------------------------------------------- 208 209Datafiles can also be accessed as '$LTPROOT/testcases/data/$TCID/...', 210but '$TST_DATAROOT' and 'tst_dataroot()' are preferred as these can be used 211when running testcases directly in git tree as well as from install 212location. 213 214The path is constructed according to these rules: 215 2161. if '$LTPROOT' is set, return '$LTPROOT/testcases/data/$TCID' 2172. else if 'tst_tmpdir()' was called return '$STARTWD/datafiles' 218 (where '$STARTWD' is initial working directory as recorded by 'tst_tmdir()') 2193. else return '$CWD/datafiles' 220 221See 'testcases/commands/file/' for example. 222 2232.1.3 Subexecutables 224^^^^^^^^^^^^^^^^^^^^ 225 226If you test needs to execute a binary, place it in the same directory as the 227testcase and name the file starting with testname_ ('.tid' see below). 228Once the test is executed by the framework, the path to the directory with all 229LTP binaries is added to the '$PATH' and you can execute it just by its name. 230 231TIP: If you need to execute such test from the LTP tree, you can add path to 232 current directory to '$PATH' manually with: 'PATH="$PATH:$PWD" ./foo01'. 233 2342.2 Writing a test in C 235~~~~~~~~~~~~~~~~~~~~~~~ 236 2372.2.1 Basic test structure 238^^^^^^^^^^^^^^^^^^^^^^^^^^ 239 240Let's start with an example, following code is a simple test for a 'getenv()'. 241 242[source,c] 243------------------------------------------------------------------------------- 244/* 245 * This is test for basic functionality of getenv(). 246 * 247 * - create an env variable and verify that getenv() can get get it 248 * - call getenv() with nonexisting variable name, check that it returns NULL 249 */ 250 251#include "tst_test.h" 252 253#define ENV1 "LTP_TEST_ENV" 254#define ENV2 "LTP_TEST_THIS_DOES_NOT_EXIST" 255#define ENV_VAL "val" 256 257static void setup(void) 258{ 259 if (setenv(ENV1, ENV_VAL, 1)) 260 tst_brk(TBROK | TERRNO, "setenv() failed"); 261} 262 263static void test(void) 264{ 265 char *ret; 266 267 ret = getenv(ENV1); 268 269 if (!ret) { 270 tst_res(TFAIL, "getenv(" ENV1 ") = NULL"); 271 goto next; 272 } 273 274 if (!strcmp(ret, ENV_VAL)) { 275 tst_res(TPASS, "getenv(" ENV1 ") = '"ENV_VAL "'"); 276 } else { 277 tst_res(TFAIL, "getenv(" ENV1 ") = '%s', expected '" 278 ENV_VAL "'", ret); 279 } 280 281next: 282 ret = getenv(ENV2); 283 284 if (ret) 285 tst_res(TFAIL, "getenv(" ENV2 ") = '%s'", ret); 286 else 287 tst_res(TPASS, "getenv(" ENV2 ") = NULL"); 288} 289 290static struct tst_test test = { 291 .tid = "getenv01", 292 .test_all = test, 293 .setup = setup, 294}; 295------------------------------------------------------------------------------- 296 297Each test includes the 'tst_test.h' header and must define the 'struct 298tst_test test' structure. 299 300The '.tid' defines test name (usually syscall/libcall name + number). The name 301is used in the test results as well as a base for temporary directory name if 302temporary directory is needed. In most of the cases it's the same as test 303filename without the extension. 304 305The overall test initialization is done in the 'setup()' function. 306 307The overall cleanup is done in a 'cleanup()' function. Here 'cleanup()' is 308omitted as the test does not have anything to clean up. If cleanup is set in 309the test structure it's called on test exit just before the test library 310cleanup. That especially means that cleanup can be called at any point in a 311test execution. For example even when a test setup step has failed, therefore 312the 'cleanup()' function must be able to cope with unfinished initialization, 313and so on. 314 315The test itself is done in the 'test()' function. The test function must work 316fine if called in a loop. 317 318There are two types of a test function pointers in the test structure. The 319first one is a '.test_all' pointer that is used when test is implemented as a 320single function. Then there is a '.test' function along with the number of 321tests '.tcnt' that allows for more detailed result reporting. If the '.test' 322pointer is set the function is called '.tcnt' times with an integer parameter 323in range of [0, '.tcnt' - 1]. 324 325IMPORTANT: Only one of '.test' and '.test_all' can be set at a time. 326 327Each test has a default timeout set to 300s. The default timeout can be 328overriden by setting '.timeout' in the test structure or by calling 329'tst_set_timeout()' in the test 'setup()'. 330 331A word about the cleanup() callback 332+++++++++++++++++++++++++++++++++++ 333 334There are a few rules that needs to be followed in order to write correct 335cleanup() callback. 336 3371. Free only resources that were initialized. Keep in mind that callback can 338 be executed at any point in the test run. 339 3402. Make sure to free resources in the reverse order they were 341 initialized. (Some of the steps may not depend on others and everything 342 will work if there were swapped but let's keep it in order.) 343 344The first rule may seem complicated at first however, on the contrary, it's 345quite easy. All you have to do is to keep track of what was already 346initialized. For example file descriptors needs to be closed only if they were 347assigned a valid file descriptor. For most of the things you need to create 348extra flag that is set right after successful initialization though. Consider, 349for example, test setup below. 350 351[source,c] 352------------------------------------------------------------------------------- 353static int fd0, fd1, mount_flag; 354 355#define MNTPOINT "mntpoint" 356#define FILE1 "mntpoint/file1" 357#define FILE2 "mntpoint/file2" 358 359static void setup(void) 360{ 361 SAFE_MKDIR(MNTPOINT, 0777); 362 SAFE_MKFS(tst_device->dev, tst_device->fs_type, NULL, NULL); 363 SAFE_MOUNT(tst_device->dev, MNTPOINT, tst_device->fs_type, 0, 0); 364 mount_flag = 1; 365 366 fd0 = SAFE_OPEN(cleanup, FILE1, O_CREAT | O_RDWR, 0666); 367 fd1 = SAFE_OPEN(cleanup, FILE2, O_CREAT | O_RDWR, 0666); 368} 369------------------------------------------------------------------------------- 370 371In this case the 'cleanup()' function may be invoked when any of the 'SAFE_*' 372macros has failed and therefore must be able to work with unfinished 373initialization as well. Since global variables are initialized to zero we can 374just check that fd > 0 before we attempt to close it. The mount function 375requires extra flag to be set after device was successfully mounted. 376 377[source,c] 378------------------------------------------------------------------------------- 379static void cleanup(void) 380{ 381 if (fd1 > 0) 382 SAFE_CLOSE(fd1); 383 384 if (fd0 > 0) 385 SAFE_CLOSE(fd0); 386 387 if (mount_flag && tst_umouont(MNTPOINT)) 388 tst_res(TWARN | TERRNO, "umount(%s)", MNTPOINT); 389} 390------------------------------------------------------------------------------- 391 392IMPORTANT: 'SAFE_MACROS()' used in cleanup *do not* exit the test. Failure 393 only produces a warning and the 'cleanup()' carries on. This is 394 intentional as we want to execute as much 'cleanup()' as possible. 395 396WARNING: Calling tst_brk() in test 'cleanup()' does not exit the test as well 397 and 'TBROK' is converted to 'TWARN'. 398 399NOTE: Creation and removal of the test temporary directory is handled in 400 the test library and the directory is removed recursively. Therefore 401 we do not have to remove files and directories in the test cleanup. 402 4032.2.2 Basic test interface 404^^^^^^^^^^^^^^^^^^^^^^^^^^ 405 406[source,c] 407------------------------------------------------------------------------------- 408void tst_res(int ttype, char *arg_fmt, ...); 409------------------------------------------------------------------------------- 410 411Printf-like function to report test result, it's mostly used with ttype: 412 413|============================== 414| 'TPASS' | Test has passed. 415| 'TFAIL' | Test has failed. 416| 'TINFO' | General message. 417|============================== 418 419The 'ttype' can be combined bitwise with 'TERRNO' or 'TTERRNO' to print 420'errno', 'TEST_ERRNO' respectively. 421 422[source,c] 423------------------------------------------------------------------------------- 424void tst_brk(int ttype, char *arg_fmt, ...); 425------------------------------------------------------------------------------- 426 427Printf-like function to report error and exit the test, it can be used with ttype: 428 429|============================================================ 430| 'TBROK' | Something has failed in test preparation phase. 431| 'TCONF' | Test is not appropriate for current configuration 432 (syscall not implemented, unsupported arch, ...) 433|============================================================ 434 435The 'ttype' can be combined bitwise with 'TERRNO' or 'TTERRNO' to print 436'errno', 'TEST_ERRNO' respectively. 437 438[source,c] 439------------------------------------------------------------------------------- 440const char *tst_strsig(int sig); 441------------------------------------------------------------------------------- 442 443Return the given signal number's corresponding string. 444 445[source,c] 446------------------------------------------------------------------------------- 447const char *tst_strerrno(int err); 448------------------------------------------------------------------------------- 449 450Return the given errno number's corresponding string. Using this function to 451translate 'errno' values to strings is preferred. You should not use the 452'strerror()' function in the testcases. 453 454[source,c] 455------------------------------------------------------------------------------- 456void tst_set_timeout(unsigned int timeout); 457------------------------------------------------------------------------------- 458 459Allows for setting timeout per test iteration dymanically in the test setup(), 460the timeout is specified in seconds. 461 4622.2.3 Test temporary directory 463^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 464 465If '.needs_tmpdir' is set to '1' in the 'struct tst_test' unique test 466temporary is created and it's set as the test working directory. Tests *MUST 467NOT* create temporary files outside that directory. 468 469IMPORTANT: Close all file descriptors (that point to files in test temporary 470 directory, even the unlinked ones) either in the 'test()' function 471 or in the test 'cleanup()' otherwise the test may break temporary 472 directory removal on NFS (look for "NFS silly rename"). 473 474[[2.2.4]] 4752.2.4 Safe macros 476^^^^^^^^^^^^^^^^^ 477 478Safe macros aim to simplify error checking in test preparation. Instead of 479calling system API functions, checking for their return value and aborting the 480test if the operation has failed, you just use corresponding safe macro. 481 482Use them whenever it's possible. 483 484Instead of writing: 485 486[source,c] 487------------------------------------------------------------------------------- 488 fd = open("/dev/null", O_RDONLY); 489 if (fd < 0) 490 tst_brk(TBROK | TERRNO, "opening /dev/null failed"); 491------------------------------------------------------------------------------- 492 493You write just: 494 495[source,c] 496------------------------------------------------------------------------------- 497 fd = SAFE_OPEN("/dev/null", O_RDONLY); 498------------------------------------------------------------------------------- 499 500IMPORTANT: The SAFE_CLOSE() function also sets the passed file descriptor to -1 501 after it's successfully closed. 502 503They can also simplify reading and writing of sysfs files, you can, for 504example, do: 505 506[source,c] 507------------------------------------------------------------------------------- 508 SAFE_FILE_SCANF("/proc/sys/kernel/pid_max", "%lu", &pid_max); 509------------------------------------------------------------------------------- 510 511See 'include/tst_safe_macros.h', 'include/tst_safe_stdio.h' and 512'include/tst_safe_file_ops.h' and 'include/tst_safe_net.h' for a complete list. 513 5142.2.5 Test specific command line options 515^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 516 517[source,c] 518------------------------------------------------------------------------------- 519struct tst_option { 520 char *optstr; 521 char **arg; 522 char *help; 523}; 524------------------------------------------------------------------------------- 525 526Test specific command line parameters can be passed with the 'NULL'-terminated 527array of 'struct tst_option'. The 'optstr' is the command line option i.e. "o" 528or "o:" if option has a parameter. Only short options are supported. The 'arg' 529is where 'optarg' is stored upon match. If option has no parameter it's set to 530non-'NULL' value if option was present. The 'help' is a short help string. 531 532NOTE: The test parameters must not collide with common test parameters defined 533 in the library the currently used ones are +-i+, +-I+, +-C+, and +-h+. 534 535[source,c] 536------------------------------------------------------------------------------- 537int tst_parse_int(const char *str, int *val, int min, int max); 538int tst_parse_float(const char *str, float *val, float min, float max); 539------------------------------------------------------------------------------- 540 541Helpers for parsing the the strings returned in the 'struct tst_option'. 542 543Both return zero on success and 'errno', mostly 'EINVAL' or 'ERANGE', on 544failure. 545 546Both functions are no-op if 'str' is 'NULL'. 547 548The valid range for result includes both 'min' and 'max'. 549 550.Example Usage 551[source,c] 552------------------------------------------------------------------------------- 553#include <limits.h> 554#include "tst_test.h" 555 556static char *str_threads; 557static int threads = 10; 558 559static struct tst_option options[] = { 560 {"t:", &str_threads, "Number of threads (default 10)"}, 561 ... 562 {NULL, NULL, NULL} 563}; 564 565static void setup(void) 566{ 567 if (tst_parse_int(str_threads, &threads, 1, INT_MAX)) 568 tst_brk(TBROK, "Invalid number of threads '%s'", str_threads); 569 570 ... 571} 572 573static void test_threads(void) 574{ 575 ... 576 577 for (i = 0; i < threads; i++) { 578 ... 579 } 580 581 ... 582} 583 584static struct tst_test test = { 585 ... 586 .options = options, 587 ... 588}; 589------------------------------------------------------------------------------- 590 591 5922.2.6 Runtime kernel version detection 593^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 594 595Testcases for newly added kernel functionality require kernel newer than a 596certain version to run. All you need to skip a test on older kernels is to 597set the '.min_kver' string in the 'struct tst_test' to a minimal required 598kernel version, e.g. '.min_kver = "2.6.30"'. 599 600For more complicated operations such as skipping a test for a certain range 601of kernel versions, following functions could be used: 602 603[source,c] 604------------------------------------------------------------------------------- 605int tst_kvercmp(int r1, int r2, int r3); 606 607struct tst_kern_exv { 608 char *dist_name; 609 char *extra_ver; 610}; 611 612int tst_kvercmp2(int r1, int r2, int r3, struct tst_kern_exv *vers); 613------------------------------------------------------------------------------- 614 615These two functions are intended for runtime kernel version detection. They 616parse the output from 'uname()' and compare it to the passed values. 617 618The return value is similar to the 'strcmp()' function, i.e. zero means equal, 619negative value means that the kernel is older than than the expected value and 620positive means that it's newer. 621 622The second function 'tst_kvercmp2()' allows for specifying per-vendor table of 623kernel versions as vendors typically backport fixes to their kernels and the 624test may be relevant even if the kernel version does not suggests so. See 625'testcases/kernel/syscalls/inotify/inotify04.c' for example usage. 626 627WARNING: The shell 'tst_kvercmp' maps the result into unsigned integer - the 628 process exit value. 629 6302.2.7 Fork()-ing 631^^^^^^^^^^^^^^^^ 632 633Be wary that if the test forks and there were messages printed by the 634'tst_*()' interfaces, the data may still be in libc/kernel buffers and these 635*ARE NOT* flushed automatically. 636 637This happens when 'stdout' gets redirected to a file. In this case, the 638'stdout' is not line buffered, but block buffered. Hence after a fork content 639of the buffers will be printed by the parent and each of the children. 640 641To avoid that you should use 'SAFE_FORK()'. 642 643IMPORTANT: You have to set the '.forks_child' flag in the test structure 644 if your testcase forks. 645 646[[2.2.8]] 6472.2.8 Doing the test in the child process 648^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 649 650Results reported by 'tst_res()' are propagated to the parent test process via 651block of shared memory. 652 653Calling 'tst_brk()' causes child process to exit with non-zero exit value. 654Which means that it's safe to use 'SAFE_*()' macros in the child processes as 655well. 656 657Children that outlive the 'test()' function execution are waited for in the 658test library. Unclean child exit (killed by signal, non-zero exit value, etc.) 659will cause the main test process to exit with 'tst_brk()', which especially 660means that 'TBROK' propagated from a child process will cause the whole test 661to exit with 'TBROK'. 662 663If a test needs a child that segfaults or does anything else that cause it to 664exit uncleanly all you need to do is to wait for such children from the 665'test()' function so that it's reaped before the main test exits the 'test()' 666function. 667 668[source,c] 669------------------------------------------------------------------------------- 670#include "tst_test.h" 671 672void tst_reap_children(void); 673------------------------------------------------------------------------------- 674 675The 'tst_reap_children()' function makes the process wait for all of its 676children and exits with 'tst_brk(TBROK, ...)' if any of them returned 677a non zero exit code. 678 6792.2.9 Fork() and Parent-child synchronization 680^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 681 682As LTP tests are written for Linux, most of the tests involve fork()-ing and 683parent-child process synchronization. LTP includes a checkpoint library that 684provides wait/wake futex based functions. 685 686In order to use checkpoints the '.needs_checkpoints' flag in the 'struct 687tst_test' must be set to '1', this causes the test library to initialize 688checkpoints before the 'test()' function is called. 689 690[source,c] 691------------------------------------------------------------------------------- 692#include "tst_test.h" 693 694TST_CHECKPOINT_WAIT(id) 695 696TST_CHECKPOINT_WAIT2(id, msec_timeout) 697 698TST_CHECKPOINT_WAKE(id) 699 700TST_CHECKPOINT_WAKE2(id, nr_wake) 701 702TST_CHECKPOINT_WAKE_AND_WAIT(id) 703------------------------------------------------------------------------------- 704 705The checkpoint interface provides pair of wake and wait functions. The 'id' is 706unsigned integer which specifies checkpoint to wake/wait for. As a matter of 707fact it's an index to an array stored in a shared memory, so it starts on 708'0' and there should be enough room for at least of hundred of them. 709 710The 'TST_CHECKPOINT_WAIT()' and 'TST_CHECKPOINT_WAIT2()' suspends process 711execution until it's woken up or until timeout is reached. 712 713The 'TST_CHECKPOINT_WAKE()' wakes one process waiting on the checkpoint. 714If no process is waiting the function retries until it success or until 715timeout is reached. 716 717If timeout has been reached process exits with appropriate error message (uses 718'tst_brk()'). 719 720The 'TST_CHECKPOINT_WAKE2()' does the same as 'TST_CHECKPOINT_WAKE()' but can 721be used to wake precisely 'nr_wake' processes. 722 723The 'TST_CHECKPOINT_WAKE_AND_WAIT()' is a shorthand for doing wake and then 724immediately waiting on the same checkpoint. 725 726Child processes created via 'SAFE_FORK()' are ready to use the checkpoint 727synchronization functions, as they inherited the mapped page automatically. 728 729Child processes started via 'exec()', or any other processes not forked from 730the test process must initialize the checkpoint by calling 'tst_reinit()'. 731 732For the details of the interface, look into the 'include/tst_checkpoint.h'. 733 734[source,c] 735------------------------------------------------------------------------------- 736#include "tst_test.h" 737 738/* 739 * Waits for process state change. 740 * 741 * The state is one of the following: 742 * 743 * R - process is running 744 * S - process is sleeping 745 * D - process sleeping uninterruptibly 746 * Z - zombie process 747 * T - process is traced 748 */ 749TST_PROCESS_STATE_WAIT(pid, state) 750------------------------------------------------------------------------------- 751 752The 'TST_PROCESS_STATE_WAIT()' waits until process 'pid' is in requested 753'state'. The call polls +/proc/pid/stat+ to get this information. 754 755It's mostly used with state 'S' which means that process is sleeping in kernel 756for example in 'pause()' or any other blocking syscall. 757 7582.2.10 Signal handlers 759^^^^^^^^^^^^^^^^^^^^^^ 760 761If you need to use signal handlers, keep the code short and simple. Don't 762forget that the signal handler is called asynchronously and can interrupt the 763code execution at any place. 764 765This means that problems arise when global state is changed both from the test 766code and signal handler, which will occasionally lead to: 767 768* Data corruption (data gets into inconsistent state), this may happen, for 769 example, for any operations on 'FILE' objects. 770 771* Deadlock, this happens, for example, if you call 'malloc(2)', 'free(2)', 772 etc. from both the test code and the signal handler at the same time since 773 'malloc' has global lock for it's internal data structures. (Be wary that 774 'malloc(2)' is used by the libc functions internally too.) 775 776* Any other unreproducible and unexpected behavior. 777 778Quite common mistake is to call 'exit(3)' from a signal handler. Note that this 779function is not signal-async-safe as it flushes buffers, etc. If you need to 780exit a test immediately from a signal handler use '_exit(2)' instead. 781 782TIP: See 'man 7 signal' for the list of signal-async-safe functions. 783 784If a signal handler sets a variable, its declaration must be 'volatile', 785otherwise compiler may misoptimize the code. This is because the variable may 786not be changed in the compiler code flow analysis. There is 'sig_atomic_t' 787type defined in C99 but this one *DOES NOT* imply 'volatile' (it's just a 788'typedef' to 'int'). So the correct type for a flag that is changed from a 789signal handler is either 'volatile int' or 'volatile sig_atomic_t'. 790 7912.2.11 Kernel Modules 792^^^^^^^^^^^^^^^^^^^^^ 793 794There are certain cases where the test needs a kernel part and userspace part, 795happily, LTP can build a kernel module and then insert it to the kernel on test 796start for you. See 'testcases/kernel/device-drivers/block' for details. 797 7982.2.11 Useful macros 799^^^^^^^^^^^^^^^^^^^^^ 800 801[source,c] 802------------------------------------------------------------------------------- 803ARRAY_SIZE(arr) 804------------------------------------------------------------------------------- 805 806Returns the size of statically defined array, i.e. 807'(sizeof(arr) / sizeof(*arr))' 808 809[source,c] 810------------------------------------------------------------------------------- 811LTP_ALIGN(x, a) 812------------------------------------------------------------------------------- 813 814Aligns the x to be next multiple of a. The a must be power of 2. 815 8162.2.12 Filesystem type detection 817^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 818 819Some tests are known to fail on certain filesytems (you cannot swap on TMPFS, 820there are unimplemented 'fcntl()' etc.). 821 822If your test needs to be skipped on certain filesystems, use the interface 823below: 824 825[source,c] 826------------------------------------------------------------------------------- 827#include "tst_test.h" 828 829 /* 830 * Unsupported only on NFS. 831 */ 832 if (tst_fs_type(".") == TST_NFS_MAGIC) 833 tst_brk(TCONF, "Test not supported on NFS filesystem"); 834 835 836 /* 837 * Unsupported on NFS, TMPFS and RAMFS 838 */ 839 long type; 840 841 switch ((type = tst_fs_type("."))) { 842 case TST_NFS_MAGIC: 843 case TST_TMPFS_MAGIC: 844 case TST_RAMFS_MAGIC: 845 tst_brk(TCONF, "Test not supported on %s filesystem", 846 tst_fs_type_name(type)); 847 break; 848 } 849------------------------------------------------------------------------------- 850 8512.2.13 Thread-safety in the LTP library 852^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 853 854It is safe to use library 'tst_res()' function in multi-threaded tests. 855 856Only the main thread must return from the 'test()' function to the test 857library and that must be done only after all threads that may call any library 858function has been terminated. That especially means that threads that may call 859'tst_brk()' must terminate before the execution of the 'test()' function 860returns to the library. This is usually done by the main thread joining all 861worker threads at the end of the 'test()' function. Note that the main thread 862will never get to the library code in a case that 'tst_brk()' was called from 863one of the threads since it will sleep at least in 'pthread_join()' on the 864thread that called the 'tst_brk()' till 'exit()' is called by 'tst_brk()'. 865 866The test-supplied cleanup function runs *concurrently* to the rest of the 867threads in a case that cleanup was entered from 'tst_brk()'. Subsequent 868threads entering 'tst_brk()' must be suspended or terminated at the start of 869the the user supplied cleanup function. It may be necessary to stop or exit 870the rest of the threads before the test cleans up as well. For example threads 871that create new files should be stopped before temporary directory is be 872removed. 873 874Following code example shows thread safe cleanup function example using atomic 875increment as a guard. The library calls its cleanup after the execution returns 876from the user supplied cleanup and expects that only one thread returns from 877the user supplied cleanup to the test library. 878 879[source,c] 880------------------------------------------------------------------------------- 881#include "tst_test.h" 882 883static void cleanup(void) 884{ 885 static int flag; 886 887 if (tst_atomic_inc(&flag) != 1) 888 pthread_exit(NULL); 889 890 /* if needed stop the rest of the threads here */ 891 892 ... 893 894 /* then do cleanup work */ 895 896 ... 897 898 /* only one thread returns to the library */ 899} 900------------------------------------------------------------------------------- 901 902 9032.2.14 Testing with a block device 904^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 905 906Some tests needs a block device (inotify tests, syscall 'EROFS' failures, 907etc.). LTP library contains a code to prepare a testing device. 908 909If '.needs_device' flag in the 'struct tst_test' is set the the 'tst_device' 910structure is initialized with a path to a test device and default filesystem 911to be used. 912 913You can also request minimal device size in megabytes by setting 914'.dev_min_size' the device is guaranteed to have at least the requested size 915then. 916 917If '.format_device' flag is set the device is formatted with a filesystem as 918well. You can use '.dev_fs_type' to override the default filesystem type if 919needed and pass additional options to mkfs via '.dev_fs_opts' and 920'.dev_extra_opt' pointers. 921 922If '.mount_device' is set, the device is mounted at '.mntpoint' which is used 923to pass a directory name that will be created and used as mount destination. 924You can pass additional flags and data to the mount command via '.mnt_flags' 925and '.mnt_data' pointers. 926 927[source,c] 928------------------------------------------------------------------------------- 929#include "tst_test.h" 930 931struct tst_device { 932 const char *dev; 933 const char *fs_type; 934}; 935 936extern struct tst_device *tst_device; 937 938int tst_umount(const char *path); 939------------------------------------------------------------------------------- 940 941In case that 'LTP_DEV' is passed to the test in an environment, the library 942checks that the file exists and that it's a block device, if 943'.device_min_size' is set the device size is checked as well. If 'LTP_DEV' 944wasn't set or if size requirements were not met a temporary file is created 945and attached to a free loop device. 946 947If there is no usable device and loop device couldn't be initialized the test 948exits with 'TCONF'. 949 950The 'tst_umount()' function works exactly as 'umount(2)' but retries several 951times on 'EBUSY'. This is because various desktop daemons (gvfsd-trash is known 952for that) may be stupid enough to probe all newly mounted filesystem which 953results in 'umount(2)' failing with 'EBUSY'. 954 955IMPORTANT: All testcases should use 'tst_umount()' instead of 'umount(2)' to 956 umount filesystems. 957 9582.2.15 Formatting a device with a filesystem 959^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 960 961[source,c] 962------------------------------------------------------------------------------- 963#include "tst_test.h" 964 965static void setup(void) 966{ 967 ... 968 SAFE_MKFS(tst_device->dev, tst_device->fs_type, NULL, NULL); 969 ... 970} 971------------------------------------------------------------------------------- 972 973This function takes a path to a device, filesystem type and an array of extra 974options passed to mkfs. 975 976The fs options 'fs_opts' should either be 'NULL' if there are none, or a 977'NULL' terminated array of strings such as: 978+const char *const opts[] = {"-b", "1024", NULL}+. 979 980The extra option 'extra_opt' should either be 'NULL' if there is none, or a 981string such as '"102400"'; 'extra_opt' will be passed after device name. e.g: 982+mkfs -t ext4 -b 1024 /dev/sda1 102400+ in this case. 983 9842.2.16 Verifying a filesystem's free space 985^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 986 987Some tests have size requirements for the filesystem's free space. If these 988requirements are not satisfied, the tests should be skipped. 989 990[source,c] 991------------------------------------------------------------------------------- 992#include "tst_test.h" 993 994int tst_fs_has_free(const char *path, unsigned int size, unsigned int mult); 995------------------------------------------------------------------------------- 996 997The 'tst_fs_has_free()' function returns 1 if there is enough space and 0 if 998there is not. 999 1000The 'path' is the pathname of any directory/file within a filesystem. 1001 1002The 'mult' is a multiplier, one of 'TST_BYTES', 'TST_KB', 'TST_MB' or 'TST_GB'. 1003 1004The required free space is calculated by 'size * mult', e.g. 1005'tst_fs_has_free("/tmp/testfile", 64, TST_MB)' will return 1 if the 1006filesystem, which '"/tmp/testfile"' is in, has 64MB free space at least, and 0 1007if not. 1008 10092.2.17 Files, directories and fs limits 1010^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1011 1012Some tests need to know the maximum count of links to a regular file or 1013directory, such as 'rename(2)' or 'linkat(2)' to test 'EMLINK' error. 1014 1015[source,c] 1016------------------------------------------------------------------------------- 1017#include "tst_test.h" 1018 1019int tst_fs_fill_hardlinks(const char *dir); 1020------------------------------------------------------------------------------- 1021 1022Try to get maximum count of hard links to a regular file inside the 'dir'. 1023 1024NOTE: This number depends on the filesystem 'dir' is on. 1025 1026This function uses 'link(2)' to create hard links to a single file until it 1027gets 'EMLINK' or creates 65535 links. If the limit is hit, the maximum number of 1028hardlinks is returned and the 'dir' is filled with hardlinks in format 1029"testfile%i", where i belongs to [0, limit) interval. If no limit is hit or if 1030'link(2)' failed with 'ENOSPC' or 'EDQUOT', zero is returned and previously 1031created files are removed. 1032 1033[source,c] 1034------------------------------------------------------------------------------- 1035#include "tst_test.h" 1036 1037int tst_fs_fill_subdirs(const char *dir); 1038------------------------------------------------------------------------------- 1039 1040Try to get maximum number of subdirectories in directory. 1041 1042NOTE: This number depends on the filesystem 'dir' is on. For current kernel, 1043subdir limit is not available for all filesystems (available for ext2, ext3, 1044minix, sysv and more). If the test runs on some other filesystems, like ramfs, 1045tmpfs, it will not even try to reach the limit and return 0. 1046 1047This function uses 'mkdir(2)' to create directories in 'dir' until it gets 1048'EMLINK' or creates 65535 directories. If the limit is hit, the maximum number 1049of subdirectories is returned and the 'dir' is filled with subdirectories in 1050format "testdir%i", where i belongs to [0, limit - 2) interval (because each 1051newly created dir has two links already - the '.' and the link from parent 1052dir). If no limit is hit or if 'mkdir(2)' failed with 'ENOSPC' or 'EDQUOT', 1053zero is returned and previously created directories are removed. 1054 1055[source,c] 1056------------------------------------------------------------------------------- 1057#include "tst_test.h" 1058 1059int tst_dir_is_empty(const char *dir, int verbose); 1060------------------------------------------------------------------------------- 1061 1062Returns non-zero if directory is empty and zero otherwise. 1063 1064Directory is considered empty if it contains only '.' and '..'. 1065 10662.2.18 Getting an unused PID number 1067^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1068 1069Some tests require a 'PID', which is not used by the OS (does not belong to 1070any process within it). For example, kill(2) should set errno to 'ESRCH' if 1071it's passed such 'PID'. 1072 1073[source,c] 1074------------------------------------------------------------------------------- 1075#include "tst_test.h" 1076 1077pid_t tst_get_unused_pid(void); 1078------------------------------------------------------------------------------- 1079 1080Return a 'PID' value not used by the OS or any process within it. 1081 1082[source,c] 1083------------------------------------------------------------------------------- 1084#include "tst_test.h" 1085 1086int tst_get_free_pids(void); 1087------------------------------------------------------------------------------- 1088 1089Returns number of unused pids in the system. Note that this number may be 1090different once the call returns and should be used only for rough estimates. 1091 10922.2.20 Running executables 1093^^^^^^^^^^^^^^^^^^^^^^^^^^ 1094 1095[source,c] 1096------------------------------------------------------------------------------- 1097#include "tst_test.h" 1098 1099int tst_run_cmd(const char *const argv[], 1100 const char *stdout_path, 1101 const char *stderr_path, 1102 int pass_exit_val); 1103------------------------------------------------------------------------------- 1104 1105'tst_run_cmd' is a wrapper for 'vfork() + execvp()' which provides a way 1106to execute an external program. 1107 1108'argv[]' is a NULL-terminated array of strings starting with the program name 1109which is followed by optional arguments. 1110 1111A non-zero 'pass_exit_val' makes 'tst_run_cmd' return the program exit code to 1112the caller. A zero for 'pass_exit_val' makes 'tst_run_cmd' exit the tests 1113on failure. 1114 1115In case that 'execvp()' has failed and the 'pass_exit_val' flag was set, the 1116return value is '255' if 'execvp()' failed with 'ENOENT' and '254' otherwise. 1117 1118'stdout_path' and 'stderr_path' determine where to redirect the program 1119stdout and stderr I/O streams. 1120 1121.Example 1122[source,c] 1123------------------------------------------------------------------------------- 1124#include "tst_test.h" 1125 1126const char *const cmd[] = { "ls", "-l", NULL }; 1127 1128... 1129 /* Store output of 'ls -l' into log.txt */ 1130 tst_run_cmd(cmd, "log.txt", NULL, 0); 1131... 1132------------------------------------------------------------------------------- 1133 11342.2.21 Measuring elapsed time and helper functions 1135^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1136 1137[source,c] 1138------------------------------------------------------------------------------- 1139#include "tst_test.h" 1140 1141void tst_timer_check(clockid_t clk_id); 1142 1143void tst_timer_start(clockid_t clk_id); 1144 1145void tst_timer_stop(void); 1146 1147struct timespec tst_timer_elapsed(void); 1148 1149long long tst_timer_elapsed_ms(void); 1150 1151long long tst_timer_elapsed_us(void); 1152------------------------------------------------------------------------------- 1153 1154The 'tst_timer_check()' function checks if specified 'clk_id' is suppored and 1155exits the test with 'TCONF' otherwise. It's expected to be used in test 1156'setup()' before any resources that needs to be cleaned up are initialized, 1157hence it does not include a cleanup function parameter. 1158 1159The 'tst_timer_start()' marks start time and stores the 'clk_id' for further 1160use. 1161 1162The 'tst_timer_stop()' marks the stop time using the same 'clk_id' as last 1163call to 'tst_timer_start()'. 1164 1165The 'tst_timer_elapsed*()' returns time difference between the timer start and 1166last timer stop in several formats and units. 1167 1168IMPORTANT: The timer functions use 'clock_gettime()' internally which needs to 1169 be linked with '-lrt' on older glibc. Please do not forget to add 1170 'LDLIBS+=-lrt' in Makefile. 1171 1172[source,c] 1173------------------------------------------------------------------------------- 1174long long tst_timespec_to_us(struct timespec t); 1175long long tst_timespec_to_ms(struct timespec t); 1176 1177struct timeval tst_us_to_timeval(long long us); 1178struct timeval tst_ms_to_timeval(long long ms); 1179 1180int tst_timespec_lt(struct timespec t1, struct timespec t2); 1181 1182struct timespec tst_timespec_add_us(struct timespec t, long long us); 1183 1184struct timespec tst_timespec_diff(struct timespec t1, struct timespec t2); 1185long long tst_timespec_diff_us(struct timespec t1, struct timespec t2); 1186long long tst_timespec_diff_ms(struct timespec t1, struct timespec t2); 1187 1188struct timespec tst_timespec_abs_diff(struct timespec t1, struct timespec t2); 1189long long tst_timespec_abs_diff_us(struct timespec t1, struct timespec t2); 1190long long tst_timespec_abs_diff_ms(struct timespec t1, struct timespec t2); 1191------------------------------------------------------------------------------- 1192 1193The first four functions are simple inline conversion functions. 1194 1195The 'tst_timespec_lt()' function returns non-zero if 't1' is earlier than 1196't2'. 1197 1198The 'tst_timespec_add_us()' function adds 'us' microseconds to the timespec 1199't'. The 'us' is expected to be positive. 1200 1201The 'tst_timespec_diff*()' functions returns difference between two times, the 1202't1' is expected to be later than 't2'. 1203 1204The 'tst_timespec_abs_diff*()' functions returns absolute value of difference 1205between two times. 1206 1207NOTE: All conversions to ms and us rounds the value. 1208 12092.2.22 Datafiles 1210^^^^^^^^^^^^^^^^ 1211 1212[source,c] 1213------------------------------------------------------------------------------- 1214#include "tst_test.h" 1215 1216static const char *const res_files[] = { 1217 "foo", 1218 "bar", 1219 NULL 1220}; 1221 1222static struct tst_test test = { 1223 ... 1224 .resource_files = res_files, 1225 ... 1226} 1227------------------------------------------------------------------------------- 1228 1229If the test needs additional files to be copied to the test temporary 1230directory all you need to do is to list their filenames in the 1231'NULL'-terminated array '.resource_files' in the tst_test structure. 1232 1233When resource files is set test temporary directory is created automatically, 1234there is need to set '.needs_tmpdir' as well. 1235 1236The test library looks for datafiles first, these are either stored in a 1237directory called +datafiles+ in the +$PWD+ at the start of the test or in 1238+$LTPROOT/testcases/data/$tid+. If the file is not found the library looks 1239into +$LTPROOT/testcases/bin/+ and to +$PWD+ at the start of the test. This 1240ensures that the testcases can copy the file(s) effortlessly both when test is 1241started from the directory it was compiled in as well as when LTP was 1242installed. 1243 1244The file(s) are copied to the newly created test temporary directory which is 1245set as the test working directory when the 'test()' functions is executed. 1246 12472.2.23 Code path tracing 1248^^^^^^^^^^^^^^^^^^^^^^^^ 1249 1250'tst_res' is a macro, so on when you define a function in one file: 1251 1252[source,c] 1253------------------------------------------------------------------------------- 1254int do_action(int arg) 1255{ 1256 ... 1257 1258 if (ok) { 1259 tst_res(TPASS, "check passed"); 1260 return 0; 1261 } else { 1262 tst_res(TFAIL, "check failed"); 1263 return -1; 1264 } 1265} 1266------------------------------------------------------------------------------- 1267 1268and call it from another file, the file and line reported by 'tst_res' in this 1269function will be from the former file. 1270 1271'TST_TRACE' can make the analysis of such situations easier. It's a macro which 1272inserts a call to 'tst_res(TINFO, ...)' in case its argument evaluates to 1273non-zero. In this call to 'tst_res(TINFO, ...)' the file and line will be 1274expanded using the actual location of 'TST_TRACE'. 1275 1276For example, if this another file contains: 1277 1278[source,c] 1279------------------------------------------------------------------------------- 1280#include "tst_test.h" 1281 1282if (TST_TRACE(do_action(arg))) { 1283 ... 1284} 1285------------------------------------------------------------------------------- 1286 1287the generated output may look similar to: 1288 1289------------------------------------------------------------------------------- 1290common.h:9: FAIL: check failed 1291test.c:8: INFO: do_action(arg) failed 1292------------------------------------------------------------------------------- 1293 12942.3 Writing a testcase in shell 1295~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1296 1297LTP supports testcases to be written in a portable shell too. 1298 1299There is a shell library modeled closely to the C interface at 1300'testcases/lib/tst_test.sh'. 1301 1302WARNING: All identifiers starting with TST_ or tst_ are reserved for the 1303 test library. 1304 13052.3.1 Basic test interface 1306^^^^^^^^^^^^^^^^^^^^^^^^^^ 1307 1308[source,sh] 1309------------------------------------------------------------------------------- 1310#!/bin/sh 1311# 1312# This is a basic test for true shell buildin 1313# 1314 1315TST_ID="true01" 1316TST_TESTFUNC=do_test 1317. tst_test.sh 1318 1319do_test() 1320{ 1321 true 1322 ret=$? 1323 1324 if [ $ret -eq 0 ]; then 1325 tst_res TPASS "true returned 0" 1326 else 1327 tst_res TFAIL "true returned $ret" 1328 fi 1329} 1330 1331tst_run 1332------------------------------------------------------------------------------- 1333 1334TIP: To execute this test the 'tst_test.sh' library must be in '$PATH'. If you 1335 are executing the test from a git checkout you can run it as 1336 'PATH="$PATH:../../lib" ./foo01.sh' 1337 1338The shell library expects test setup, cleanup and the test function executing 1339the test in the '$TST_SETUP', '$TST_CLEANUP' and '$TST_TESTFUNC' variables. 1340 1341Both '$TST_SETUP' and '$TST_CLEANUP' are optional. 1342 1343The '$TST_TESTFUNC' may be called several times if more than one test 1344iteration was requested by passing right command line options to the test. 1345 1346The '$TST_CLEANUP' may be called even in the middle of the setup and must be 1347able to clean up correctly even in this situation. The easiest solution for 1348this is to keep track of what was initialized and act accordingly in the 1349cleanup. 1350 1351Notice also the 'tst_run' function called at the end of the test that actually 1352starts the test. 1353 1354[source,sh] 1355------------------------------------------------------------------------------- 1356#!/bin/sh 1357# 1358# Example test with tests in separate functions 1359# 1360 1361TST_ID="example01" 1362TST_TESTFUNC=test 1363TST_CNT=2 1364. tst_test.sh 1365 1366test1() 1367{ 1368 tst_res TPASS "Test 1 passed" 1369} 1370 1371test2() 1372{ 1373 tst_res TPASS "Test 2 passed" 1374} 1375 1376tst_run 1377------------------------------------------------------------------------------- 1378 1379If '$TST_CNT' is set, the test library looks if there are functions named 1380'$\{TST_TESTFUNC\}1', ..., '$\{TST_TESTFUNC\}$\{TST_CNT\}' and if these are 1381found they are executed one by one. 1382 1383[source,sh] 1384------------------------------------------------------------------------------- 1385#!/bin/sh 1386# 1387# Example test with tests in a single function 1388# 1389 1390TST_ID="example02" 1391TST_TESTFUNC=do_test 1392TST_CNT=2 1393. tst_test.sh 1394 1395do_test() 1396{ 1397 case $1 in 1398 1) tst_res TPASS "Test 1 passed";; 1399 2) tst_res TPASS "Test 2 passed";; 1400 esac 1401} 1402 1403tst_run 1404------------------------------------------------------------------------------- 1405 1406Otherwise, if '$TST_CNT' is set but there is no '$\{TST_TESTFUNC\}1', etc., 1407the '$TST_TESTFUNC' is executed '$TST_CNT' times and the test number is passed 1408to it in the '$1'. 1409 14102.3.2 Library variables 1411^^^^^^^^^^^^^^^^^^^^^^^ 1412 1413Similarily to the C library various checks and preparations can be requested 1414simply by setting right '$TST_NEEDS_FOO'. 1415 1416[options="header"] 1417|============================================================================= 1418| Variable name | Action done 1419| 'TST_NEEDS_ROOT' | Exit the test with 'TCONF' unless executed under root 1420| 'TST_NEEDS_TMPDIR' | Create test temporary directory and cd into it. 1421| 'TST_NEEDS_DEVICE' | Prepare test temporary device, the path to testing 1422 device is stored in '$TST_DEVICE' variable. 1423| 'TST_NEEDS_CMDS' | String with command names that has to be present for 1424 the test (see below). 1425| 'TST_NEEDS_MODULE' | Test module name needed for the test (see below). 1426|============================================================================= 1427 1428Checking for presence of commands 1429+++++++++++++++++++++++++++++++++ 1430 1431[source,sh] 1432------------------------------------------------------------------------------- 1433#!/bin/sh 1434 1435... 1436 1437TST_NEEDS_CMDS="modinfo modprobe" 1438. tst_test.sh 1439 1440... 1441 1442------------------------------------------------------------------------------- 1443 1444Setting '$TST_NEEDS_CMDS' to a string listing required commands will check for 1445existence each of them and exits the test with 'TCONF' on first misssing. 1446 1447Alternatively the 'tst_check_cmds()' function can be used to do the same on 1448runtime, since sometimes we need to the check at runtime too. 1449 1450Locating kernel modules 1451+++++++++++++++++++++++ 1452 1453The LTP build system can build kernel modules as well, setting 1454'$TST_NEEDS_MODULE' to module name will cause to library to look for the 1455module in a few possible paths. 1456 1457If module was found the path to it will be stored into '$TST_MODPATH' 1458variable, if module wasn't found the test will exit with 'TCONF'. 1459 14602.3.3 Optional command line parameters 1461^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1462 1463[source,sh] 1464------------------------------------------------------------------------------- 1465#!/bin/sh 1466# 1467# Optional test command line parameters 1468# 1469 1470TST_ID="example03" 1471TST_OPTS="af:" 1472TST_USAGE=usage 1473TST_PARSE_ARGS=parse_args 1474TST_TESTFUNC=do_test 1475 1476. tst_test.sh 1477 1478ALTERNATIVE=0 1479MODE="foo" 1480 1481usage() 1482{ 1483 cat << EOF 1484usage: $0 [-a] [-f <foo|bar>] 1485 1486OPTIONS 1487-a Enable support for alternative foo 1488-f Specify foo or bar mode 1489EOF 1490} 1491 1492parse_args() 1493{ 1494 case $1 in 1495 a) ALTERNATIVE=1 1496 f) MODE="$2" 1497 esac 1498} 1499 1500do_test() 1501{ 1502 ... 1503} 1504 1505tst_run 1506------------------------------------------------------------------------------- 1507 1508The 'getopts' string for optional parameters is passed in the '$TST_OPTS' 1509variable. There are a few default parameters that cannot be used by a test, 1510these can be listed with passing help '-h' option to any test. 1511 1512The function that prints the usage is passed in '$TST_USAGE', the help for 1513the options implemented in the library is appended when usage is printed. 1514 1515Lastly the fucntion '$PARSE_ARGS' is called with the option name in '$1' and, 1516if option has argument, its value in '$2'. 1517 1518[source,sh] 1519------------------------------------------------------------------------------- 1520#!/bin/sh 1521# 1522# Optional test positional paramters 1523# 1524 1525TST_ID="example04" 1526TST_POS_ARGS=3 1527TST_USAGE=usage 1528TST_TESTFUNC=do_test 1529 1530. tst_test.sh 1531 1532usage() 1533{ 1534 cat << EOF 1535usage: $0 [min] [max] [size] 1536 1537EOF 1538} 1539 1540min="$1" 1541max="$2" 1542size="$3" 1543 1544do_test() 1545{ 1546 ... 1547} 1548 1549tst_run 1550------------------------------------------------------------------------------- 1551 1552You can also request a number of positional parameters by setting the 1553'$TST_POS_ARGS' variable. If you do, these will be available as they were 1554passed directly to the script in '$1', '$2', ..., '$n'. 1555 15562.3.4 Usefull library functions 1557^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1558 1559Sleeping for subsecond intervals 1560++++++++++++++++++++++++++++++++ 1561 1562Albeit there is a sleep command available basically everywhere not all 1563implementations can support sleeping for less than one second. And most of the 1564time sleeping for a second is too much. Therefore LTP includes 'tst_sleep' 1565that can sleep for defined amount of seconds, milliseconds or microseconds. 1566 1567[source,sh] 1568------------------------------------------------------------------------------- 1569# sleep for 100 milliseconds 1570tst_sleep 100ms 1571------------------------------------------------------------------------------- 1572 1573Checking for integers 1574+++++++++++++++++++++ 1575 1576[source,sh] 1577------------------------------------------------------------------------------- 1578# returns zero if passed an integer parameter, non-zero otherwise 1579tst_is_int "$FOO" 1580------------------------------------------------------------------------------- 1581 1582Obtaining random numbers 1583++++++++++++++++++++++++ 1584 1585There is no '$RANDOM' in portable shell, use 'tst_random' instead. 1586 1587[source,sh] 1588------------------------------------------------------------------------------- 1589# get random integer between 0 and 1000 (including 0 and 1000) 1590tst_random 0 1000 1591------------------------------------------------------------------------------- 1592 1593Formatting device with a filesystem 1594+++++++++++++++++++++++++++++++++++ 1595 1596The 'tst_mkfs' helper will format device with the filesystem. 1597 1598[source,sh] 1599------------------------------------------------------------------------------- 1600# format test device with ext2 1601tst_mkfs ext2 $TST_DEVICE 1602------------------------------------------------------------------------------- 1603 1604Umounting filesystems 1605+++++++++++++++++++++ 1606 1607The 'tst_umount' helper is a safe way to umount a filesystem. 1608 1609If the path passed to the function is not mounted (present in '/proc/mounts') 1610it's noop. 1611 1612Otherwise it retries to umount the filesystem a few times on a failure, which 1613is a workaround since there are a daemons dumb enough to probe all newly 1614mounted filesystems, which prevents them from umounting shortly after they 1615were mounted. 1616 1617Running commands as different user with 'su' 1618++++++++++++++++++++++++++++++++++++++++++++ 1619 1620While some distributions retain paths added to +$PATH+ when doing 1621+su user -c "command"+ this does not work at least in Debian. If you want to 1622run LTP binaries as a different user you must use 'tst_su' instead which sets 1623up +$PATH+ and the runs the command. 1624 1625.Run test child binary as a test user 1626[source,sh] 1627------------------------------------------------------------------------------- 1628#!/bin/sh 1629TCID=foo01 1630. test.sh 1631 1632tst_su testusr foo01_child 1633if [ $? -ne 0 ]; then 1634 tst_resm TFAIL "foo failed" 1635else 1636 tst_resm TPASS "foo passed" 1637fi 1638------------------------------------------------------------------------------- 1639 1640ROD and ROD_SILENT 1641++++++++++++++++++ 1642 1643These functions supply the 'SAFE_MACROS' used in C although they work and are 1644named differently. 1645 1646[source,sh] 1647------------------------------------------------------------------------------- 1648ROD_SILENT command arg1 arg2 ... 1649 1650# is shorthand for: 1651 1652command arg1 arg2 ... > /dev/null 2>&1 1653if [ $? -ne 0 ]; then 1654 tst_brkm TBROK "..." 1655fi 1656 1657 1658ROD command arg1 arg2 ... 1659 1660# is shorthand for: 1661 1662ROD arg1 arg2 ... 1663if [ $? -ne 0 ]; then 1664 tst_brkm TBROK "..." 1665fi 1666------------------------------------------------------------------------------- 1667 1668WARNING: Keep in mind that output redirection (to a file) happens in the 1669 caller rather than in the ROD function and cannot be checked for 1670 write errors by the ROD function. 1671 1672As a matter of a fact doing +ROD echo a > /proc/cpuinfo+ would work just fine 1673since the 'ROD' function will only get the +echo a+ part that will run just 1674fine. 1675 1676[source,sh] 1677------------------------------------------------------------------------------- 1678# Redirect output to a file with ROD 1679ROD echo foo \> bar 1680------------------------------------------------------------------------------- 1681 1682Note the '>' is escaped with '\', this causes that the '>' and filename are 1683passed to the 'ROD' function as parameters and the 'ROD' function contains 1684code to split '$@' on '>' and redirects the output to the file. 1685 1686EXPECT_PASS and EXPECT_FAIL 1687+++++++++++++++++++++++++++ 1688 1689[source,sh] 1690------------------------------------------------------------------------------- 1691EXPECT_PASS command arg1 arg2 ... [ \> file ] 1692EXPECT_FAIL command arg1 arg2 ... [ \> file ] 1693------------------------------------------------------------------------------- 1694 1695'EXPECT_PASS' calls 'tst_resm TPASS' if the command exited with 0 exit code, 1696and 'tst_resm TFAIL' otherwise. 'EXPECT_FAIL' does vice versa. 1697 1698Output redirection rules are the same as for the 'ROD' function. In addition 1699to that, 'EXPECT_FAIL' always redirects the command's stderr to '/dev/null'. 1700 1701tst_kvcmp 1702+++++++++ 1703 1704This command compares the currently running kernel version given conditions 1705with syntax similar to the shell test command. 1706 1707[source,sh] 1708------------------------------------------------------------------------------- 1709# Exit the test if kernel version is older or equal to 2.6.8 1710if tst_kvcmp -le 2.6.8; then 1711 tst_brk TCONF "Kernel newer than 2.6.8 is needed" 1712fi 1713 1714# Exit the test if kernel is newer than 3.8 and older than 4.0.1 1715if tst_kvcmp -gt 3.8 -a -lt 4.0.1; then 1716 tst_brk TCONF "Kernel must be older than 3.8 or newer than 4.0.1" 1717fi 1718------------------------------------------------------------------------------- 1719 1720[options="header"] 1721|======================================================================= 1722| expression | description 1723| -eq kver | Returns true if kernel version is equal 1724| -ne kver | Returns true if kernel version is not equal 1725| -gt kver | Returns true if kernel version is greater 1726| -ge kver | Returns true if kernel version is greater or equal 1727| -lt kver | Returns true if kernel version is lesser 1728| -le kver | Returns true if kernel version is lesser or equal 1729| -a | Does logical and between two expressions 1730| -o | Does logical or between two expressions 1731|======================================================================= 1732 1733The format for kernel version has to either be with one dot e.g. '2.6' or with 1734two dots e.g. '4.8.1'. 1735 1736.tst_fs_has_free 1737[source,sh] 1738------------------------------------------------------------------------------- 1739#!/bin/sh 1740 1741... 1742 1743# whether current directory has 100MB free space at least. 1744if ! tst_fs_has_free . 100MB; then 1745 tst_brkm TCONF "Not enough free space" 1746fi 1747 1748... 1749------------------------------------------------------------------------------- 1750 1751The 'tst_fs_has_free' shell interface returns 0 if the specified free space is 1752satisfied, 1 if not, and 2 on error. 1753 1754The second argument supports suffixes kB, MB and GB, the default unit is Byte. 1755 1756.tst_retry 1757[source,sh] 1758------------------------------------------------------------------------------- 1759#!/bin/sh 1760 1761... 1762 1763# Retry ping command three times 1764tst_retry "ping -c 1 127.0.0.1" 1765 1766if [ $? -ne 0 ]; then 1767 tst_resm TFAIL "Failed to ping 127.0.0.1" 1768else 1769 tst_resm TPASS "Successfully pinged 127.0.0.1" 1770fi 1771 1772... 1773------------------------------------------------------------------------------- 1774 1775The 'tst_retry' function allows you to retry a command after waiting small 1776amount of time until it succeeds or until given amount of retries has been 1777reached (default is three attempts). 1778 17792.3.5 Restarting daemons 1780^^^^^^^^^^^^^^^^^^^^^^^^ 1781 1782Restarting system daemons is a complicated task for two reasons. 1783 1784* There are different init systems 1785 (SysV init, systemd, etc...) 1786 1787* Daemon names are not unified between distributions 1788 (apache vs httpd, cron vs crond, various syslog variations) 1789 1790To solve these problems LTP has 'testcases/lib/daemonlib.sh' library that 1791provides functions to start/stop/query daemons as well as variables that store 1792correct daemon name. 1793 1794.Supported operations 1795|============================================================================== 1796| start_daemon() | Starts daemon, name is passed as first parameter. 1797| stop_daemon() | Stops daemon, name is passed as first parameter. 1798| restart_daemon() | Restarts daemon, name is passed as first parameter. 1799| status_daemon() | Detect daemon status (exit code: 0: running, 1: not running). 1800|============================================================================== 1801 1802.Variables with detected names 1803|============================================================================== 1804| CROND_DAEMON | Cron daemon name (cron, crond). 1805| SYSLOG_DAEMON | Syslog daemon name (syslog, syslog-ng, rsyslog). 1806|============================================================================== 1807 1808.Cron daemon restart example 1809[source,sh] 1810------------------------------------------------------------------------------- 1811#!/bin/sh 1812# 1813# Cron daemon restart example 1814# 1815TCID=cron01 1816TST_COUNT=1 1817. test.sh 1818. daemonlib.sh 1819 1820... 1821 1822restart_daemon $CROND_DAEMON 1823 1824... 1825 1826tst_exit 1827------------------------------------------------------------------------------- 1828 18292.3.6 Access to the checkpoint interface 1830~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1831 1832The shell library provides an implementation of the checkpoint interface 1833compatible with the C version. All TST_CHECKPOINT_* functions are available. 1834 1835In order to initialize checkpoints '$TST_NEEDS_CHECKPOINTS' must be set to '1' 1836before the inclusion of 'test.sh': 1837 1838[source,sh] 1839------------------------------------------------------------------------------- 1840#!/bin/sh 1841 1842TST_NEEDS_CHECKPOINTS=1 1843. test.sh 1844------------------------------------------------------------------------------- 1845 1846Since both the implementations are compatible, it's also possible to start 1847a child binary process from a shell test and synchronize with it. This process 1848must have checkpoints initialized by calling tst_reinit()'. 1849 18503. Common problems 1851------------------ 1852 1853This chapter describes common problems/misuses and less obvious design patters 1854(quirks) in UNIX interfaces. Read it carefully :) 1855 18563.1 umask() 1857~~~~~~~~~~~ 1858 1859I've been hit by this one several times already... When you create files 1860with 'open()' or 'creat()' etc, the mode specified as the last parameter *is 1861not* the mode the file is created with. The mode depends on current 'umask()' 1862settings which may clear some of the bits. If your test depends on specific 1863file permissions you need either to change umask to 0 or 'chmod()' the file 1864afterwards or use SAFE_TOUCH() that does the 'chmod()' for you. 1865 18663.2 access() 1867~~~~~~~~~~~ 1868 1869If 'access(some_file, W_OK)' is executed by root, it will return success even 1870if the file doesn't have write permission bits set (the same holds for R_OK 1871too). For sysfs files you can use 'open()' as a workaround to check file 1872read/write permissions. It might not work for other filesystems, for these you 1873have to use 'stat()', 'lstat()' or 'fstat()'. 1874 18753.3 umount() EBUSY 1876~~~~~~~~~~~~~~~~~~ 1877 1878Various desktop daemons (gvfsd-trash is known for that) may be stupid enough 1879to probe all newly mounted filesystem which results in 'umount(2)' failing 1880with 'EBUSY'; use 'tst_umount()' described in 2.2.19 that retries in this case 1881instead of plain 'umount(2)'. 1882 18833.4 FILE buffers and fork() 1884~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1885 1886Be vary that if a process calls 'fork(2)' the child process inherits open 1887descriptors as well as copy of the parent memory so especially if there are 1888any open 'FILE' buffers with a data in them they may be written both by the 1889parent and children resulting in corrupted/duplicated data in the resulting 1890files. 1891 1892Also open 'FILE' streams are flushed and closed at 'exit(3)' so if your 1893program works with 'FILE' streams, does 'fork(2)', and the child may end up 1894calling 'exit(3)' you will likely end up with corrupted files. 1895 1896The solution to this problem is either simply call 'fflush(NULL)' that flushes 1897all open output 'FILE' streams just before doing 'fork(2)'. You may also use 1898'_exit(2)' in child processes which does not flush 'FILE' buffers and also 1899skips 'atexit(3)' callbacks. 1900 19014. Test Contribution Checklist 1902------------------------------ 1903 19041. Test compiles and runs fine (check with -i 10 too) 19052. Checkpatch does not report any errors 19063. The runtest entires are in place 19074. Test files are added into corresponding .gitignore files 19085. Patches apply over the latest git 1909 1910 19114.1 About .gitignore files 1912~~~~~~~~~~~~~~~~~~~~~~~~~~ 1913 1914There are numerous '.gitignore' files in the LTP tree. Usually there is a 1915'.gitignore' file per a group of tests. The reason for this setup is simple. 1916It's easier to maintain a '.gitignore' file per directory with tests, rather 1917than having single file in the project root directory. This way, we don't have 1918to update all the gitignore files when moving directories, and they get deleted 1919automatically when a directory with tests is removed. 1920