• Home
Name
Date
Size
#Lines
LOC

..--

android_libpthread/12-May-2024-94

android_librt/12-May-2024-94

newlib_tests/12-May-2024-3,9282,552

tests/12-May-2024-950429

MakefileD12-May-2024654 2916

README.mdD12-May-20247.5 KiB153121

cloner.cD12-May-20244.7 KiB17694

errnos.hD12-May-20243.8 KiB187154

get_path.cD12-May-20242.7 KiB10448

ltp.pc.inD12-May-2024192 119

parse_opts.cD12-May-202416 KiB622422

random_range.cD12-May-202421.6 KiB893504

safe_file_ops.cD12-May-20249.5 KiB454345

safe_macros.cD12-May-202428.6 KiB1,277992

safe_net.cD12-May-202412.9 KiB536414

safe_pthread.cD12-May-2024911 4429

safe_stdio.cD12-May-20242.5 KiB10267

self_exec.cD12-May-20244.7 KiB226164

signame.hD12-May-20242.1 KiB10778

tlibio.cD12-May-202460.1 KiB2,1621,488

tst_af_alg.cD12-May-20245.8 KiB253201

tst_ansi_color.cD12-May-2024966 6549

tst_arch.cD12-May-20241.8 KiB9579

tst_assert.cD12-May-20242.2 KiB8458

tst_bool_expr.cD12-May-20248.1 KiB407293

tst_buffers.cD12-May-20242.5 KiB144110

tst_capability.cD12-May-20242 KiB9170

tst_cgroup.cD12-May-202429.9 KiB1,209830

tst_checkpoint.cD12-May-20244.5 KiB170103

tst_checksum.cD12-May-20243.4 KiB8274

tst_clocks.cD12-May-20243.4 KiB158127

tst_clone.cD12-May-2024820 4730

tst_cmd.cD12-May-20248.2 KiB348261

tst_coredump.cD12-May-2024731 3620

tst_cpu.cD12-May-20242.8 KiB10058

tst_crypto.cD12-May-20242.5 KiB11380

tst_device.cD12-May-202411.5 KiB550395

tst_dir_is_empty.cD12-May-20241.3 KiB5022

tst_fill_file.cD12-May-20242.3 KiB11972

tst_fill_fs.cD12-May-20241.2 KiB6948

tst_fips.cD12-May-2024452 2514

tst_fs_has_free.cD12-May-20241.2 KiB4517

tst_fs_link_count.cD12-May-20244.8 KiB183130

tst_fs_setup.cD12-May-20241.1 KiB5142

tst_fs_type.cD12-May-20242.5 KiB9562

tst_get_bad_addr.cD12-May-20241 KiB3211

tst_hugepage.cD12-May-20241.5 KiB6849

tst_ioctl.cD12-May-2024728 3828

tst_kconfig.cD12-May-20249.9 KiB563404

tst_kernel.cD12-May-20244.3 KiB187112

tst_kvercmp.cD12-May-20244 KiB202131

tst_lockdown.cD12-May-20241.5 KiB7151

tst_memutils.cD12-May-20243.7 KiB174117

tst_mkfs.cD12-May-20242.9 KiB12687

tst_module.cD12-May-20243 KiB12587

tst_net.cD12-May-20244.7 KiB223160

tst_netdevice.cD12-May-202412.9 KiB515416

tst_parse_opts.cD12-May-20241.3 KiB3710

tst_path_has_mnt_flags.cD12-May-20242 KiB8447

tst_pid.cD12-May-20244 KiB15198

tst_process_state.cD12-May-20241.6 KiB9164

tst_res.cD12-May-202414.7 KiB607386

tst_resource.cD12-May-20243.7 KiB13887

tst_rtctime.cD12-May-20242.9 KiB12884

tst_rtnetlink.cD12-May-20248.9 KiB415313

tst_safe_file_at.cD12-May-20243.7 KiB198148

tst_safe_io_uring.cD12-May-20243.3 KiB10986

tst_safe_macros.cD12-May-202411.4 KiB555438

tst_safe_sysv_ipc.cD12-May-20245.7 KiB274210

tst_safe_timerfd.cD12-May-20241.5 KiB7051

tst_sig.cD12-May-20247.1 KiB256126

tst_sig_proc.cD12-May-2024471 3220

tst_status.cD12-May-2024872 5234

tst_supported_fs_types.cD12-May-20244.1 KiB208155

tst_sys_conf.cD12-May-20241.9 KiB10678

tst_taint.cD12-May-20242 KiB11487

tst_test.cD12-May-202432.7 KiB1,5711,191

tst_timer.cD12-May-20241.2 KiB6042

tst_timer_test.cD12-May-202410.7 KiB473319

tst_tmpdir.cD12-May-20248.4 KiB354206

tst_uid.cD12-May-20242.6 KiB12191

tst_virt.cD12-May-20244 KiB188124

tst_wallclock.cD12-May-20243.1 KiB11968

README.md

1# Test library design document
2
3## High-level picture
4
5    library process
6    +----------------------------+
7    | main                       |
8    |  tst_run_tcases            |
9    |   do_setup                 |
10    |   for_each_variant         |
11    |    for_each_filesystem     |   test process
12    |     fork_testrun ------------->+--------------------------------------------+
13    |      waitpid               |   | testrun                                    |
14    |                            |   |  do_test_setup                             |
15    |                            |   |   tst_test->setup                          |
16    |                            |   |  run_tests                                 |
17    |                            |   |   tst_test->test(i) or tst_test->test_all  |
18    |                            |   |  do_test_cleanup                           |
19    |                            |   |   tst_test->cleanup                        |
20    |                            |   |  exit(0)                                   |
21    |   do_exit                  |   +--------------------------------------------+
22    |    do_cleanup              |
23    |     exit(ret)              |
24    +----------------------------+
25
26## Test lifetime overview
27
28When a test is executed the very first thing to happen is that we check for
29various test prerequisites. These are described in the tst\_test structure and
30range from simple '.require\_root' to a more complicated kernel .config boolean
31expressions such as: "CONFIG\_X86\_INTEL\_UMIP=y | CONFIG\_X86\_UMIP=y".
32
33If all checks are passed the process carries on with setting up the test
34environment as requested in the tst\_test structure. There are many different
35setup steps that have been put into the test library again ranging from rather
36simple creation of a unique test temporary directory to a bit more complicated
37ones such as preparing, formatting, and mounting a block device.
38
39The test library also intializes shrared memory used for IPC at this step.
40
41Once all the prerequisites are checked and test environment has been prepared
42we can move on executing the testcase itself. The actual test is executed in a
43forked process, however there are a few hops before we get there.
44
45First of all there are test variants, which means that the test is re-executed
46several times with a slightly different setting. This is usually used to test a
47family of similar syscalls, where we test each of these syscalls exactly the
48same, but without re-executing the test binary itself. Test variants are
49implemented as a simple global variable counter that gets increased on each
50iteration. In a case of syscall tests we switch between which syscall to call
51based on the global counter.
52
53Then there is all\_filesystems flag which is mostly the same as test variants
54but executes the test for each filesystem supported by the system. Note that we
55can get cartesian product between test variants and all filesystems as well.
56
57In a pseudo code it could be expressed as:
58
59```
60for test_variants:
61	for all_filesystems:
62		fork_testrun()
63```
64
65Before we fork() the test process the test library sets up a timeout alarm and
66also a heartbeat signal handlers and also sets up an alarm(2) accordingly to
67the test timeout. When a test times out the test library gets SIGALRM and the
68alarm handler mercilessly kills all forked children by sending SIGKILL to the
69whole process group. The heartbeat handler is used by the test process to reset
70this timer for example when the test functions run in a loop.
71
72With that done we finally fork() the test process. The test process firstly
73resets signal handlers and sets its pid to be a process group leader so that we
74can slaughter all children if needed. The test library proceeds with suspending
75itself in waitpid() syscall and waits for the child to finish at this point.
76
77The test process goes ahead and calls the test setup() function if present in
78the tst\_test structure. It's important that we execute all test callbacks
79after we have forked the process, that way we cannot crash the test library
80process. The setup can also cause the test to exit prematurely by either direct
81or indirect (SAFE\_MACROS()) call to tst\_brk().  In this case the
82fork\_testrun() function exits, but the loops for test variants or filesystems
83carries on.
84
85All that is left to be done is to actually execute the tests, what happnes now
86depends on the -i and -I command line parameters that can request that the
87run() or run\_all() callbacks are executed N times or for N seconds. Again the
88test can exit at any time by direct or indirect call to tst\_brk().
89
90Once the test is finished all that is left for the test process is the test
91cleanup(). So if a there is a cleanup() callback in the tst\_test structure
92it's executed. The cleanup() callback runs in a special context where the
93tst\_brk(TBROK, ...) calls are converted into tst\_res(TWARN, ...) calls. This
94is because we found out that carrying on with partially broken cleanup is
95usually better option than exiting it in the middle.
96
97The test cleanup() is also called by the tst\_brk() handler in order to cleanup
98before exiting the test process, hence it must be able to cope even with
99partial test setup. Usually it suffices to make sure to clean up only
100resources that already have been set up and to do that in an inverse order that
101we did in setup().
102
103Once the test process exits or leaves the run() or run\_all() function the test
104library wakes up from the waitpid() call, and checks if the test process
105exitted normally.
106
107Once the testrun is finished the test library does a cleanup() as well to clean
108up resources set up in the test library setup(), reports test results and
109finally exits the process.
110
111### Test library and fork()-ing
112
113Things are a bit more complicated when fork()-ing is involved, however the test
114results are stored in a page of a shared memory and incremented by atomic
115operations, hence the results are stored right after the test reporting
116function returns from the test library and the access is, by definition,
117race-free as well.
118
119On the other hand the test library, apart from sending a SIGKILL to the whole
120process group on timeout, does not track grandchildren.
121
122This especially means that:
123
124- The test exits once the main test process exits.
125
126- While the test results are, by the design, propagated to the test library
127  we may still miss a child that gets killed by a signal or exits unexpectedly.
128
129The test writer should, because of this, take care for reaping these proceses
130properly, in most cases this could be simply done by calling
131tst\_reap\_children() to collect and dissect deceased.
132
133Also note that tst\_brk() does exit only the current process, so if a child
134process calls tst\_brk() the counters are incremented and only the process
135exits.
136
137### Test library and exec()
138
139The piece of mapped memory to store the results to is not preserved over
140exec(2), hence to use the test library from a binary started by an exec() it
141has to be remaped. In this case the process must to call tst\_reinit() before
142calling any other library functions. In order to make this happen the program
143environment carries LTP\_IPC\_PATH variable with a path to the backing file on
144tmpfs. This also allows us to use the test library from shell testcases.
145
146### Test library and process synchronization
147
148The piece of mapped memory is also used as a base for a futex-based
149synchronization primitives called checkpoints. And as said previously the
150memory can be mapped to any process by calling the tst\_reinit() function. As a
151matter of a fact there is even a tst\_checkpoint binary that allows us to use
152the checkpoints from shell code as well.
153