1# Perfetto Stress Test 2 3This is a test harness that to stress test the client library (DataSource-level 4only for now). 5 6The test is based on a number of configs in /test/stress_test/configs/*.cfg 7(NOTE: they must be listed in configs/BUILD.gn). 8The config is a /protos/perfetto/config/stress_test_config.proto message, which 9embeds the configuration of the test and a whole trace config. 10 11Each configs defines a testing scenario, determining the general trace config 12and all the settings of the test (e.g., how many producer processes to spawn, 13the write timings). 14 15The test is based on exec()-ing `traced` (the tracing service), `perfetto` (the 16consumer cmdline client) and a variable number of `stress_producer` instances. 17 18`stress_producer` emits events at a configurable rate, writing predictable 19sequences of numbers / string, so that the test harness can easily detect 20corruptions, out-of-order events or gaps. 21 22After running each test, the `stress_test` binary reads back the trace and 23performs a bunch of checks: 24 25- Checks that the number of sequences is exactly equal to #processes x #threads. 26- Checks that each sequence has all the expected packets in the right sequence 27- Checks the payload and correctness of proto nesting of each trace packet. 28- Reports CPU/Memory/Context-switch numbers for the service and producer 29 processes. 30 31Each test config is isolated from the others. All processes are killed and 32re-spawned for each test. 33 34The stdout/err of each process is saved in a dedicated /tmp/ folder, as well as 35the resulting trace. 36 37## Building and running the test 38 39```bash 40# This will recursively build traced, perfetto and stress_producer. 41ninja -C out/default stress_test 42 43out/default/stress_test 44``` 45 46will output: 47 48```txt 49[307.909] stress_test.cc:116 Saving test results in /tmp/perfetto-ltIBJgA0 50 51=============================================================== 52Config: simple 53=============================================================== 54Metric Expected Actual 55------ -------- ------ 56#Errors 0 0 57Duration [ms] 3000 3109 58Num threads 1 1 59Num packets 1000 1001 60Trace size [KB] 168 170 61Svc RSS [MB] 4 2 62Prod RSS [MB] --- 1 63Svc CPU [ms] --- 10 64Prod CPU [ms] --- 32 65Svc #ctxswitch --- 103 / 20 66Prod #ctxswitch --- 1022 / 1 67 68=============================================================== 69Config: bursts 70=============================================================== 71Metric Expected Actual 72------ -------- ------ 73#Errors 0 0 74Duration [ms] 2000 2381 75Num threads 10 10 76Num packets 2675 20021 77Trace size [KB] 449 11063 78Svc RSS [MB] 32 17 79Prod RSS [MB] --- 1 80Svc CPU [ms] --- 98 81Prod CPU [ms] --- 17 82Svc #ctxswitch --- 704 / 1327 83Prod #ctxswitch --- 421 / 1 84``` 85 86```bash 87$ ls -Rlh /tmp/perfetto-ltIBJgA0 88total 0 89drwxr-xr-x 16 primiano wheel 512B 5 Aug 09:16 bursts 90drwxr-xr-x 9 primiano wheel 288B 5 Aug 09:16 simple 91drwxr-xr-x 38 primiano wheel 1.2K 5 Aug 09:16 the_storm 92 93/tmp/perfetto-ltIBJgA0/bursts: 94total 22752 95-rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 errors.log 96-rw-r--r-- 1 primiano wheel 180B 5 Aug 09:16 perfetto.log 97-rw-r--r-- 1 primiano wheel 441B 5 Aug 09:16 producer.0.log 98... 99-rw-r--r-- 1 primiano wheel 441B 5 Aug 09:16 producer.9.log 100-rw------- 1 primiano wheel 11M 5 Aug 09:16 trace 101-rw-r--r-- 1 primiano wheel 407B 5 Aug 09:16 traced.log 102 103/tmp/perfetto-ltIBJgA0/simple: 104total 400 105srwxr-xr-x 1 primiano wheel 0B 5 Aug 09:16 consumer.sock 106-rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 errors.log 107-rw-r--r-- 1 primiano wheel 178B 5 Aug 09:16 perfetto.log 108-rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 producer.0.log 109srwxr-xr-x 1 primiano wheel 0B 5 Aug 09:16 producer.sock 110-rw------- 1 primiano wheel 167K 5 Aug 09:16 trace 111-rw-r--r-- 1 primiano wheel 406B 5 Aug 09:16 traced.log 112 113/tmp/perfetto-ltIBJgA0/the_storm: 114total 524432 115-rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 errors.log 116-rw-r--r-- 1 primiano wheel 184B 5 Aug 09:16 perfetto.log 117-rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 producer.0.log 118... 119-rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 producer.127.log 120-rw------- 1 primiano wheel 248M 5 Aug 09:16 trace 121-rw-r--r-- 1 primiano wheel 408B 5 Aug 09:16 traced.log 122``` 123 124## TODOs 125 126The following scenarios requires more coverage: 127 128- Nested messages. 129- Force losses and check that the last_dropped flag is consistent. 130- Flushes and scraping. 131- Report data losses in the test output. 132- Multibuffer scenarios. 133- write_into_file=true. 134- Vary page size, smb size. 135