1Layout test analyzer script. 2 3* Prerequisite: 4 5This script requires PySVN (http://pysvn.tigris.org/) to be installed on the 6machine. 7 8* How to execute 9 10python layouttest_analyzer.py (Please run with '-h' for available command-line 11 options) 12 13Example: python layouttest_analyzer.py -r imasaki@chromium.org 14-t /var/www/analyzer/graph.html 15 16* File/Directory structure 17 18layout_tests/ 19 README: this file. 20 *.py: python scripts. 21 tmp/: temp files. 22 test_data/: data for unit tests. 23 graph/: the default location for graph files. 24 anno/: the default location for bug annotation files. 25 result/: the default location for the analyzer results. 26 testname/: the default location for the test group definition files in CSV. 27 28* Execution overview. 29 301) using PySVN, the script collects layout test of interest 31(specified in command-line parameter and testnames/*.csv file) from 32Webkit SVN repository as well as test description. Then, it gets the 33latest test expectation file and does the simple parsing. The 34parsed test expectation is joined with the layout tests using test name 35as its join key. The tests are classified into 'whole', 'skip', 36'nonskip' test groups. 'Whole' contains all tests, 'skip' contains 37tests that are skipped as specified in the test expectation file, 'nonskip' 38tests are in the test expectation file and not skipped (these are tests 39that need attention). 40 412) the script reads the previous latest analyzer results, then, 42compares it with current results for each test group ('whole', 'skip', 43and 'nonskip'). It also looks into the SVN test expectation history 44diff for the time period for the test names of interest. 45 463) all obtained information is sent out to a email list specified 47in the parameters in pretty HTML format. 48 494) the trend graph is updated using the current results. 50 51* Components 52 53** LayoutTests: 54 55A class to store test names in layout tests. The test names (including 56regular expression patterns) are read from a CSV file and used for 57getting layout test names from Webkit SVN as well as test description. 58 59** TestExpectations: 60 61A class to model the content of test expectation file for analysis. 62The location of the test expectations file can be found in 63|TEST_EXPECTATIONS_LOCATIONS|. It is necessary to parse this 64file and store meaningful information for the analysis (joining with 65existing layout tests using a test name). Instance variable 66|all_test_expectation_info| is used. A test name such as 67'media/video-source-type.html' is used for the key to store 68information. However, a test name can appear multiple times in the 69test expectation file. So, the map should keep all the occurrence 70information. For example, the current test expectation file has the 71following two entries: 72 73BUGWK58587 LINUX DEBUG GPU : media/video-zoom.html = IMAGE 74BUGCR86714 MAC GPU : media/video-zoom.html = CRASH IMAGE 75 76In this case, all_test_expectation_info['media/video-zoom.html'] will 77have a list with two elements, each of which is the map of the test 78expectation information. 79 80** TestExpectationsHistory: 81 82A class to represent history of the test expectation file. The history 83is obtained by calling PySVN.log()/diff() APIs using the specified time 84period. 85 86** TrendGraph: 87 88A class to manage trend graph which is using Google Visualization 89 APIs. Google Visualization API 90 (http://code.google.com/apis/chart/interactive/docs/gallery/annotatedtimeline.html) 91 is used to present the historical analyzer result. Currently, data 92 is directly written to JavaScript file using file in-place 93 replacement for simplicity. 94