• Home
Name Date Size #Lines LOC

..--

presubmit/04-Jul-2025-12893

DIR_METADATAD04-Jul-2025104 76

PRESUBMIT.pyD04-Jul-202518.5 KiB490357

README.mdD04-Jul-20257 KiB181148

fieldtrial_testing_config.jsonD04-Jul-2025817.8 KiB26,61626,615

README.md

1# Field Trial Testing Configuration
2
3This directory contains the `fieldtrial_testing_config.json` configuration file,
4which is used to ensure test coverage of active field trials.
5
6For each study, the first available experiment after platform filtering is used
7as the default experiment for Chromium builds. This experiment is also used for
8perf bots and various tests in the waterfall (browser tests, including those in
9browser_tests, components_browsertests, content_browsertests,
10extensions_browsertests, interactive_ui_tests, and sync_integration_tests, and
11[web platform tests](/docs/testing/web_platform_tests.md)). It is not used by
12unit test targets.
13
14> Note: This configuration applies specifically to Chromium developer and
15> [Chrome for Testing branded](https://goo.gle/chrome-for-testing) builds.
16> Chrome branded builds do not use these definitions by default. They can, however,
17> be enabled with the `--enable-field-trial-config` switch.
18> For Chrome branded Android builds, due to binary size constraints, the
19> configuration cannot be applied by this switch.
20
21> Note: Non-developer builds of Chromium (for example, non-Chrome browsers,
22> or Chromium builds provided by Linux distros) should disable the testing
23> config by either (1) specifying the GN flag `disable_fieldtrial_testing_config=true`,
24> (2) specifying the `--disable-field-trial-config` switch or (3) specifying a
25> custom variations server URL using the `--variations-server-url` switch.
26
27> Note: An experiment in the testing configuration file that enables/disables a
28> feature that is explicitly overridden (e.g. using the `--enable-features` or
29> `--disable-features` switches) will be skipped.
30
31## Config File Format
32
33```json
34{
35    "StudyName": [
36        {
37            "platforms": [Array of Strings of Valid Platforms for These Experiments],
38            "experiments": [
39                {
40                    "//0": "Comment Line 0. Lines 0-9 are supported.",
41                    "name": "ExperimentName",
42                    "params": {Dictionary of Params},
43                    "enable_features": [Array of Strings of Features],
44                    "disable_features": [Array of Strings of Features]
45                },
46                ...
47            ]
48        },
49        ...
50    ],
51    ...
52}
53```
54
55The config file is a dictionary at the top level mapping a study name to an
56array of *study configurations*. The study name in the configuration file
57**must** match the FieldTrial name used in the Chromium client code.
58
59> Note: Many newer studies do not use study names in the client code at all, and
60> rely on the [Feature List API][FeatureListAPI] instead. Nonetheless, if a
61> study has a server-side configuration, the study `name` specified here
62> must still match the name specified in the server-side configuration; this is
63> used to implement consistency checks on the server.
64
65### Study Configurations
66
67Each *study configuration* is a dictionary containing `platforms` and
68`experiments`.
69
70`platforms` is an array of strings, indicating the targetted platforms. The
71strings may be `android`, `android_weblayer`, `android_webview`, `chromeos`,
72`chromeos_lacros`, `ios`, `linux`, `mac`, or `windows`.
73
74`experiments` is an array containing the *experiments*.
75
76The converter uses the `platforms` array to determine which experiment to use
77for the study. The first experiment matching the active platform will be used.
78
79> Note: While `experiments` is defined as an array, currently only the first
80> entry is used*\**. We would love to be able to test all possible study
81> configurations, but don't currently have the buildbot resources to do so.
82> Hence, the current best-practice is to identify which experiment group is the
83> most likely candidate for ultimate launch, and to test that configuration. If
84> there is a server-side configuration for this study, it's typically
85> appropriate to copy/paste one of the experiment definitions into this file.
86>
87> *\**
88> <small>
89>   Technically, there is one exception: If there's a forcing_flag group
90>   specified in the config, that group will be used if there's a corresponding
91>   forcing_flag specified on the command line. You, dear reader, should
92>   probably not use this fancy mechanism unless you're <em>quite</em> sure you
93>   know what you're doing =)
94> </small>
95
96### Experiments (Groups)
97Each *experiment* is a dictionary that must contain the `name` key, identifying
98the experiment group name.
99
100> Note: Studies should typically use the [Feature List API][FeatureListAPI]. For
101> such studies, the experiment `name` specified in the testing config is still
102> required (for legacy reasons), but it is ignored. However, the lists of
103> `enable_features`, `disable_features`, and `params` **must** match the server
104> config. This is enforced via server-side Tricorder checks.
105>
106> For old-school studies that do check the actual experiment group name in the
107> client code, the `name` **must** exactly match the client code and the server
108> config.
109
110The remaining keys -- `enable_features`, `disable_features`, `min_os_version`,
111and `params` -- are optional.
112
113`enable_features` and `disable_features` indicate which features should be
114enabled and disabled, respectively, through the
115[Feature List API][FeatureListAPI].
116
117`min_os_version` indicates a minimum OS version level (e.g. "10.0.0") to apply
118the experiment. This string is decoded as a `base::Version`. The same version is
119applied to all platforms. If you need different versions for different
120platforms, you will need to use different studies.
121
122`params` is a dictionary mapping parameter name to parameter value.
123
124> Reminder: The variations framework does not actually fetch any field trial
125> definitions from the server for Chromium builds, so any feature enabling or
126> disabling must be configured here.
127
128[FeatureListAPI]: https://cs.chromium.org/chromium/src/base/feature_list.h
129
130#### Comments
131
132Each experiment may have up to 10 lines of comments. The comment key must be of
133the form `//N` where `N` is between 0 and 9.
134
135```json
136{
137    "AStudyWithExperimentComment": [
138        {
139            "platforms": ["chromeos", "linux", "mac", "windows"],
140            "experiments": [
141                {
142                    "//0": "This is the first comment line.",
143                    "//1": "This is the second comment line.",
144                    "name": "DesktopExperiment"
145                }
146            ]
147        }
148    ]
149}
150```
151
152### Specifying Different Experiments for Different Platforms
153Simply specify two different study configurations in the study:
154
155```json
156{
157    "DifferentExperimentsPerPlatform": [
158        {
159            "platforms": ["chromeos", "linux", "mac", "windows"],
160            "experiments": [{ "name": "DesktopExperiment" }]
161        },
162        {
163            "platforms": ["android", "ios"],
164            "experiments": [{ "name": "MobileExperiment" }]
165        }
166    ]
167}
168```
169
170## Formatting
171
172Run the following command to auto-format the `fieldtrial_testing_config.json`
173configuration file:
174
175```shell
176python3 testing/variations/PRESUBMIT.py testing/variations/fieldtrial_testing_config.json
177```
178
179The presubmit tool will also ensure that your changes follow the correct
180ordering and format.
181