1Automatic regression tests: 2--------------------------- 3The Valgrind gdbserver automatic tests are by default 4executed as part of the Valgrind test suite: 5 make regtest 6 7By default, these tests are running with the gdb found at 8Valgrind configure time. 9 10If you want to test with another gdb version, you can do: 11 make regtest GDB=/path/to/another/gdb 12 13or (to just run the gdbserver tests with another gdb): 14 cd gdbserver_tests 15 make check 16 cd .. 17 gdbserver_tests/make_local_links /path/to/another/gdb 18 perl tests/vg_regtest gdbserver_tests 19 20The minimum version to run the tests is gdb >= 6.5. 21 Previous versions do not have the 'target remote |' command. It 22 would be possible to use an older version by using the option 23 --port of vgdb, and using the tcp/ip variant of the 'target remote' 24 GDB command. 25 26The tests have been run on various platforms using gdb versions >= 7.2 27and on some platforms gdb 7.0 and 7.1. 28Some gdb tests implies a gdb >= 7.2. (these are automatically disabled 29if testing with a lower version). 30Test behaviour with gdb < 7.0 is unknown: some might fail, 31some might block or loop for a very long time. 32 33Some tests implies to have a vgdb "ptrace invoker" capable. 34 35The prerequisite are established during make regtest (using marker files). 36Each test verifies the prerequisite using the prereq: line. 37 38In case of failing tests 39------------------------ 40When executed with a new gdb version and/or depending on the OS version, 41gdbserver tests might often fail due to (irrelevant) differences. 42Such irrelevant differences have to be filtered by gdbserver_tests/filter_gdb. 43 44You are welcome to fix such bugs by enhancing filter_gdb. 45 46Alternatively, to report such problems, the best is to re-run 47the gdbserver tests the following way: 48 perl tests/vg_regtest --keep-unfiltered gdbserver_tests 49 50Then file a bug in bugzilla, giving the following information: 51 output of 52 gdbserver_tests/gdb --version 53 uname -a 54 cat /etc/issue 55 valgrind --version (and/or svn version) 56 and attach a tar file containing all the *.out and *.diff 57 files in gdbserver_tests directory 58 59If a gdbserver test fails for other reasons, you can run the test 60manually in two windows: 61 In one window, the valgrind 62 In another window, you launch gdb yourself, and you copy paste 63 the command from xxxx.stdinB.gdb. This might help to see what is 64 wrong. 65 66Another good trick is also to execute the same kind of actions 67using a gdb connected to the gdbserver part of gdb. 68You can examine what is happening by enabling the trace 69of the packets being sent using the gdb command: 70 set debug remote 1 71Note however that the packets might be different 72(e.g. the Valgrind gdbserver understands the 'P' packet, 73which might not be understood by the gdbserver of gdb). 74 75Naming conventions: 76------------------- 77 78The gdbserver tests are done with various Valgrind tools. A gdbserver 79test using a tool xxxxxx should have its name starting with the 'short 80two letters code' of this tool. For example, the test mcvabits.vgtest 81is using Memcheck tool, while the test mssnapshot.vgtest is using 82massif tool. 83 84Typically, a gdbserver test implies to launch two programs: 85 prog: a program running under valgrind 86 progB: another program (typically, either gdb or vgdb standalone) 87The conventions about how to specify the 2nd program in a .vgtest 88are explained in the file ../tests/vg_regtest.in 89Many tests are using gdb as progB. The input for gdb is named 90xxxxxx.stdinB.gdb. 91 92One day, we might imagine to run tests in parallel. 93So, each test gets its own '--vgdb-prefix' argument. 94This also help to avoid interactions between two successive tests: 95if a previous test stayed blocked, the vgdb of the next test 96will not report an error due to multiple possible FIFOs. 97 98 99Rational for test approach 100-------------------------- 101Two approaches have been looked at: 102 V: use the 'vg_regtest' approach used by the rest of Valgrind tests 103 G: use the gdb Dejagnu test framework. 104 105Advantages of V: much simpler that G, known by Valgrind developers, 106no additional dependency for the Valgrind build and test. 107 108Disadvantages of V: not well suited to testing of interactive tools, 109very unflexible way to test the results (everything is in "template" 110files), templates contains often irrelevant data for the test but it 111can make the test fail. After writing 13 tests, it looks like the 112template diff approach is quite fragile (e.g. changing the gdb version 113and/or the OS version influences the output of irrelevant things which 114are part of the template). 115 116A more flexible template filtering is needed. 117Maybe something like: 118The program under test is outputting its instructions to be filtered in 119special markers e.g. 120#pushf filter_addresses | filter_messages 121... some output 122#pushf an_additional_filter 123... some other output, filtered by both the first and second push 124#popf 125... here output filtered only by the first pushf 126#popf 127 128Advantages of G: much more powerful, well suited to test a gdb with a 129gdbserver, tests can verify specifically some output without being 130impacted by other output, allow to test Valgrind gdbserver with the 131all of the gdb test suite (but a lot of tests will rather test gdb 132than Valgrind gdbserver). 133 134Disadvantages: not an easy beast to understand and master, running the 135whole gdb testsuite with Valgrind gdbserver looks to be a challenge. 136 137Currently, tests are written with vg_regtest. Approach G will be looked at it 138again (e.g. to complement V) once a basic set of tests are available. 139 140 141Manual tests still to automate: 142------------------------------- 143 144Validate monitor commands abbreviation recognition 145*************************************************** 146mo v.info all_errors # to show all errors recorded so far 147mo v.i a # the same 148mo v # must give an error: v can match v.set v.info 149mo v # the same v 150mo v. # the same v. 151 152test of gdb detaching or dying 153****************************** 154valgrind --vgdb=yes --vgdb-error=0 --vgdb-poll=500 ./t 155 156in another window 157 158gdb ./t 159set remotetimeout 100 160target remote|vgdb 161detach valgrind continues 162target remote|vgdb reattach 163detach valgrind continues 164target remote|vgdb reattach 165monitor v.wait no effect 166 167 168 169test of valgrind/gdb output redirection 170*************************************** 171valgrind --vgdb=yes --vgdb-error=1 --vgdb-poll=500 ./t 172 173in another window 174 175**** command to type*** ****** expected behaviour 176gdb ./t 177set remotetimeout 1000 178target remote | vgdb 179mo v.set vgdb-error 1000 # so that valgrind does stop only at error 1000 and after 180mo v.set gdb_output # to have further valgrind errors output in gdb 181c # continue, some errors will appear 182C-c # interrupt program 183mo v.set log_output # to set back the valgrind output to normal process log output 184mo l # leak output to appear in log of process 185mo v.set mixed_output 186mo l # leak output to appear in gdb 187 188 189 190test with a big executable: firefox 191*********************************** 192valgrind --vgdb=yes --vgdb-error=1000 --vgdb-poll=50000 --trace-children=yes firefox 2>&1 | tee f.out 193 194wait for some messages from the "big" firefox executable to appear. 195Then: 196 197gdb /usr/lib/firefox-3.5/firefox 198target remote | vgdb 199... then you can do various next/print/bt/bt full/break/... to see it is working 200 201bulk test with the above 202************************ 203to verify there is no race condition/no reentrance problem 204between gdbserver code and valgrind: 205start firefox like in the previous test. 206In another window, do: 207 while true 208 do 209 vgdb leak 210 sleep 1 211 done 212 213NB: this will make firefox run extremely slow, as it will do a leak 214search every second. 215 216 217Test of "jump" functionality 218---------------------------- 219... to be done : put two breaks, jump over one. 220... same but when error is encountered 221 222 223* test with --max-invoke-ms=0 224----------------------------- 225valgrind --vgdb=yes ./t 226... wait till you see "petachounok sleeping 4 of 15 227then try to gdb it 228 229!!!! this often causes gdb to report a protocol timeout. 230use gdb set remotetimeout <a big time> to avoid that. 231The symptoms of a timeout are: 232 (gdb) tar rem|vgdb --max-invoke-ms=0 233 Remote debugging using |vgdb --max-invoke-ms=0 234 relaying data between gdb and process 2930 235 Ignoring packet error, continuing... 236 warning: unrecognized item "timeout" in "qSupported" response 237 238* tests of shadow registers 239---------------------------- 240Show/modify shadow registers 241