• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Chrome Performance Dashboard Data Format
2
3## Recommended Format: Dashboard JSON v1
4
5The endpoint that accepts new points
6(`https://chromeperf.appspot.com/add_point`) accepts HTTP POST
7requests. With the POST request, there should be one parameter given,
8called "data", the value of which is JSON which contains all of the data
9being uploaded.
10
11Example:
12
13```javascript
14{
15  "master": "master.chromium.perf",
16  "bot": "linux-release",
17  "point_id": 123456,
18  "versions": {
19    "version type": "version string"
20  },
21  "supplemental": {
22    "field name": "supplemental data string",
23    "default_rev": "r_chrome_version"
24  },
25  "chart_data": {/*... as output by Telemetry; see below ...*/}
26}
27```
28
29Fields:
30
31 * `master` (string): Buildbot master name or top-level category for data.
32 * `bot` (string): Buildbot builder name, or another string that
33 represents platform type.
34 * `test_suite_name` (string): A string to use in the perf dashboard test
35 * path after master/bot. Can contain slashes.
36 * `format_version` (string): Allows dashboard to know how to process
37 the structure.
38 * `revisions` (dict): Maps repo name to revision.
39 * `supplemental` (dict): Unstructured key-value pairs which may be
40 displayed on the dashboard. Used to describe bot hardware, OS,
41 Chrome feature status, etc.
42 * `chart_data` (dict): The chart JSON as output by Telemetry.
43
44### Chart data:
45
46This contains all of the test results and any metadata that is stored with
47the test.
48
49```json
50{
51  "format_version": "1.0",
52  "benchmark_name": "page_cycler.typical_25",
53  "charts": {
54    "warm_times": {
55      "http://www.google.com/": {
56        "type": "list_of_scalar_values",
57        "values": [9, 9, 8, 9],
58      },
59      "http://www.yahoo.com/": {
60        "type": "list_of_scalar_values",
61        "values": [4, 5, 4, 4],
62      },
63      "summary": {
64        "type": "list_of_scalar_values",
65        "values": [13, 14, 12, 13],
66        "file": "gs://..."
67      },
68    }
69  }
70}
71```
72
73Fields:
74
75 * `charts`: [dict of string to dict] Maps a list of chart name strings
76 to their data dicts.
77 * `units`: [string] Units to display on the dashboard.
78 * `traces`: [dict of string to dict] Maps a list of trace name strings
79 to their trace dicts.
80 * `type`: [string] `"scalar"`, `"list_of_scalar_values"` or `"histogram"`,
81 which tells the dashboard how to interpret the rest of the fields.
82 * `improvement_direction` (string): Either `"bigger_is_better"`, or
83 `"smaller_is_better"`.
84 * `summary`: A special trace name which denotes the trace in a chart which does
85 not correspond to a specific page.
86
87## Legacy Format
88
89This format is deprecated and should not be used for new clients.
90
91In the format described below, the value of "data" in the HTTP POST
92should be a JSON encoding of a list of points to add. Each point is a
93map of property names to values for that point.
94
95Example 1:
96
97```json
98[
99  {
100    "master": "SenderType",
101    "bot": "platform-type",
102    "test": "my_test_suite/chart_name/trace_name",
103    "revision": 1234,
104    "value": 18.5
105  }
106]
107```
108
109Required fields:
110
111 * `master` (string), `bot` (string), `test` (string): These three
112 fields in combination specify a particular "test path". The master and
113 bot are supposed to be the Buildbot master name and slave `perf_id`,
114 respectively, but if the tests aren't being run by Buildbot, these
115 can be any descriptive strings which specify the test data origin
116 (note master and bot names can't contain slashes, and none of these
117 can contain asterisks).
118 * `revision` (int): The point ID, used to index the data point. It
119 doesn't actually have to be a "revision". Should be monotonically increasing
120 for data in each series.
121 * `value` (float): The Y-value for this point.
122
123Example 2 (including optional fields):
124
125```json
126[
127  {
128    "master": "ChromiumPerf",
129    "bot": "linux-release",
130    "test": "sunspider/string-unpack-code/ref",
131    "revision": 33241,
132    "value": "18.5",
133    "error": "0.5",
134    "units": "ms",
135    "masterid": "master.chromium.perf",
136    "buildername": "Linux Builder",
137    "buildnumber": 75,
138    "supplemental_columns": {
139      "r_webkit_rev": "167808",
140      "a_default_rev": "r_webkit_rev"
141    }
142  },
143  {
144    "master": "ChromiumPerf",
145    "bot": "linux-release",
146    "test": "sunspider/string-unpack-code",
147    "revision": 33241,
148    "value": "18.4",
149    "error": "0.489",
150    "units": "ms",
151    "masterid": "master.chromium.perf",
152    "buildername": "Linux Builder",
153    "buildnumber": 75,
154    "supplemental_columns": {
155      "r_webkit_rev": "167808",
156      "a_default_rev": "r_webkit_rev"
157    }
158  }
159]
160```
161
162Optional fields:
163
164 * `units` (string): The (y-axis) units for this point.
165 * `error` (float): A standard error or standard deviation value.
166 * `supplemental_columns`: A dictionary of other data associated with
167 this point.
168   * Properties starting with `r\_` are revision/version numbers.
169   * Properties starting with `d\_` are extra data numbers.
170   * Properties starting with `a\_` are extra metadata strings.
171     * `a_default_rev`: The name of a another supplemental property key
172     starting with "a_".
173     * `a_stdio_uri`: Link to stdio logs for the test run.
174 * `higher_is_better` (boolean). You can use this field to explicitly
175 define improvement direction.
176
177## Providing test and unit information
178
179Sending test descriptions are supported in with Dashboard JSON v1.
180Test descriptions for Telemetry tests are provided in code for the
181benchmarks, and are included by Telemetry in the chart JSON output.
182
183## Relevant code links
184
185Implementations of code that sends data to the dashboard:
186
187 * `chromium/build/scripts/slave/results_dashboard.py`
188 * `chromiumos/src/third_party/autotest/files/tko/perf_upload/perf_uploader.py`
189
190## Getting set up with new test results
191
192Once you're ready to start sending data to the real perf dashboard, there
193are a few more things you might want to do. Firstly, in order for the
194dashboard to accept the data, the IP of the sender must be whitelisted.
195
196If your data is not internal-only data, you can request that it be marked
197as such, again by filing an issue.
198
199Finally, if you want to monitor your the test results, you can decide
200which tests you want to be monitored, who should be receiving alerts, and
201whether you want to set any special thresholds for alerting.
202
203## Contact
204
205In general, for questions or requests you can email
206chrome-perf-dashboard-team@google.com.
207