1Test suite runner for V8, including support for distributed running. 2==================================================================== 3 4 5Local usage instructions: 6========================= 7 8Run the main script with --help to get detailed usage instructions: 9 10$ tools/run-tests.py --help 11 12The interface is mostly the same as it was for the old test runner. 13You'll likely want something like this: 14 15$ tools/run-tests.py --nonetwork --arch ia32 --mode release 16 17--nonetwork is the default on Mac and Windows. If you don't specify --arch 18and/or --mode, all available values will be used and run in turn (e.g., 19omitting --mode from the above example will run ia32 in both Release and Debug 20modes). 21 22 23Networked usage instructions: 24============================= 25 26Networked running is only supported on Linux currently. Make sure that all 27machines participating in the cluster are binary-compatible (e.g. mixing 28Ubuntu Lucid and Precise doesn't work). 29 30Setup: 31------ 32 331.) Copy tools/test-server.py to a new empty directory anywhere on your hard 34 drive (preferably not inside your V8 checkout just to keep things clean). 35 Please do create a copy, not just a symlink. 36 372.) Navigate to the new directory and let the server setup itself: 38 39$ ./test-server.py setup 40 41 This will install PIP and UltraJSON, create a V8 working directory, and 42 generate a keypair. 43 443.) Swap public keys with someone who's already part of the networked cluster. 45 46$ cp trusted/`cat data/mypubkey`.pem /where/peers/can/see/it/myname.pem 47$ ./test-server.py approve /wherever/they/put/it/yourname.pem 48 49 50Usage: 51------ 52 531.) Start your server: 54 55$ ./test-server.py start 56 572.) (Optionally) inspect the server's status: 58 59$ ./test-server.py status 60 613.) From your regular V8 working directory, run tests: 62 63$ tool/run-tests.py --arch ia32 --mode debug 64 654.) (Optionally) enjoy the speeeeeeeeeeeeeeeed 66 67 68Architecture overview: 69====================== 70 71Code organization: 72------------------ 73 74This section is written from the point of view of the tools/ directory. 75 76./run-tests.py: 77 Main script. Parses command-line options and drives the test execution 78 procedure from a high level. Imports the actual implementation of all 79 steps from the testrunner/ directory. 80 81./test-server.py: 82 Interface to interact with the server. Contains code to setup the server's 83 working environment and can start and stop server daemon processes. 84 Imports some stuff from the testrunner/server/ directory. 85 86./testrunner/local/*: 87 Implementation needed to run tests locally. Used by run-tests.py. Inspired by 88 (and partly copied verbatim from) the original test.py script. 89 90./testrunner/objects/*: 91 A bunch of data container classes, used by the scripts in the various other 92 directories; serializable for transmission over the network. 93 94./testrunner/network/*: 95 Equivalents and extensions of some of the functionality in ./testrunner/local/ 96 as required when dispatching tests to peers on the network. 97 98./testrunner/network/network_execution.py: 99 Drop-in replacement for ./testrunner/local/execution that distributes 100 test jobs to network peers instead of running them locally. 101 102./testrunner/network/endpoint.py: 103 Receiving end of a network distributed job, uses the implementation 104 in ./testrunner/local/execution.py for actually running the tests. 105 106./testrunner/server/*: 107 Implementation of the daemon that accepts and runs test execution jobs from 108 peers on the network. Should ideally have no dependencies on any of the other 109 directories, but that turned out to be impractical, so there are a few 110 exceptions. 111 112./testrunner/server/compression.py: 113 Defines a wrapper around Python TCP sockets that provides JSON based 114 serialization, gzip based compression, and ensures message completeness. 115 116 117Networking architecture: 118------------------------ 119 120The distribution stuff is designed to be a layer between deciding which tests 121to run on the one side, and actually running them on the other. The frontend 122that the user interacts with is the same for local and networked execution, 123and the actual test execution and result gathering code is the same too. 124 125The server daemon starts four separate servers, each listening on another port: 126- "Local": Communication with a run-tests.py script running on the same host. 127 The test driving script e.g. needs to ask for available peers. It then talks 128 to those peers directly (one of them will be the locally running server). 129- "Work": Listens for test job requests from run-tests.py scripts on the network 130 (including localhost). Accepts an arbitrary number of connections at the 131 same time, but only works on them in a serialized fashion. 132- "Status": Used for communication with other servers on the network, e.g. for 133 exchanging trusted public keys to create the transitive trust closure. 134- "Discovery": Used to detect presence of other peers on the network. 135 In contrast to the other three, this uses UDP (as opposed to TCP). 136 137 138Give us a diagram! We love diagrams! 139------------------------------------ 140 . 141 Machine A . Machine B 142 . 143+------------------------------+ . 144| run-tests.py | . 145| with flag: | . 146|--nonetwork --network | . 147| | / | | . 148| | / | | . 149| v / v | . 150|BACKEND / distribution | . 151+--------- / --------| \ ------+ . 152 / | \_____________________ 153 / | . \ 154 / | . \ 155+----- v ----------- v --------+ . +---- v -----------------------+ 156| LocalHandler | WorkHandler | . | WorkHandler | LocalHandler | 157| | | | . | | | | 158| | v | . | v | | 159| | BACKEND | . | BACKEND | | 160|------------- +---------------| . |---------------+--------------| 161| Discovery | StatusHandler <----------> StatusHandler | Discovery | 162+---- ^ -----------------------+ . +-------------------- ^ -------+ 163 | . | 164 +---------------------------------------------------------+ 165 166Note that the three occurrences of "BACKEND" are the same code 167(testrunner/local/execution.py and its imports), but running from three 168distinct directories (and on two different machines). 169