• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Copyright (c) 2013 The Chromium Authors. All rights reserved.
2# Use of this source code is governed by a BSD-style license that can be
3# found in the LICENSE file.
4
5"""Config file for Run Performance Test Bisect Tool
6
7This script is intended for use by anyone that wants to run a remote bisection
8on a range of revisions to look for a performance regression. Modify the config
9below and add the revision range, performance command, and metric. You can then
10run a git try <bot>.
11
12Changes to this file should never be submitted.
13
14Args:
15  'command': This is the full command line to pass to the
16      bisect-perf-regression.py script in order to execute the test.
17  'good_revision': An svn or git revision where the metric hadn't regressed yet.
18  'bad_revision': An svn or git revision sometime after the metric had
19      regressed.
20  'metric': The name of the metric to parse out from the results of the
21      performance test. You can retrieve the metric by looking at the stdio of
22      the performance test. Look for lines of the format:
23
24      RESULT <graph>: <trace>= <value> <units>
25
26      The metric name is "<graph>/<trace>".
27  'repeat_count': The number of times to repeat the performance test.
28  'max_time_minutes': The script will attempt to run the performance test
29      "repeat_count" times, unless it exceeds "max_time_minutes".
30  'truncate_percent': Discard the highest/lowest % values from performance test.
31
32Sample config:
33
34config = {
35  'command': './tools/perf/run_measurement --browser=release blink_perf third_party/WebKit/PerformanceTests/Layout/floats_50_100.html',
36  'good_revision': '233015',
37  'bad_revision': '233115',
38  'metric': 'floats_50_100/floats_50_100',
39  'repeat_count': '20',
40  'max_time_minutes': '20',
41  'truncate_percent': '25',
42}
43
44On Windows:
45  - If you're calling a python script you will need to add "python" to
46the command:
47
48config = {
49  'command': 'python tools/perf/run_measurement -v --browser=release kraken',
50  'good_revision': '185319',
51  'bad_revision': '185364',
52  'metric': 'Total/Total',
53  'repeat_count': '20',
54  'max_time_minutes': '20',
55  'truncate_percent': '25',
56}
57
58
59On ChromeOS:
60  - Script accepts either ChromeOS versions, or unix timestamps as revisions.
61  - You don't need to specify --identity and --remote, they will be added to
62    the command using the bot's BISECT_CROS_IP and BISECT_CROS_BOARD values.
63
64config = {
65  'command': './tools/perf/run_measurement -v '\
66      '--browser=cros-chrome-guest '\
67      'dromaeo tools/perf/page_sets/dromaeo/jslibstylejquery.json',
68  'good_revision': '4086.0.0',
69  'bad_revision': '4087.0.0',
70  'metric': 'jslib/jslib',
71  'repeat_count': '20',
72  'max_time_minutes': '20',
73  'truncate_percent': '25',
74}
75
76"""
77
78config = {
79  'command': '',
80  'good_revision': '',
81  'bad_revision': '',
82  'metric': '',
83  'repeat_count':'',
84  'max_time_minutes': '',
85  'truncate_percent':'',
86}
87
88# Workaround git try issue, see crbug.com/257689
89