• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Trace configuration
2
3Unlike many always-on logging systems (e.g. Linux's rsyslog, Android's logcat),
4in Perfetto all tracing data sources are idle by default and record data only
5when instructed to do so.
6
7Data sources record data only when one (or more) tracing sessions are active.
8A tracing session is started by invoking the `perfetto` cmdline client and
9passing a config (see QuickStart guide for
10[Android](/docs/quickstart/android-tracing.md) or
11[Linux](/docs/quickstart/linux-tracing.md)).
12
13A simple trace config looks like this:
14
15```protobuf
16duration_ms: 10000
17
18buffers {
19  size_kb: 65536
20  fill_policy: RING_BUFFER
21}
22
23data_sources {
24  config {
25    name: "linux.ftrace"
26    target_buffer: 0
27    ftrace_config {
28      ftrace_events: "sched_switch"
29      ftrace_events: "sched_wakeup"
30    }
31  }
32}
33
34````
35
36And is used as follows:
37
38```bash
39perfetto --txt -c config.pbtx -o trace_file.pftrace
40```
41
42TIP: Some more complete examples of trace configs can be found in the repo in
43[`/test/configs/`](/test/configs/).
44
45## TraceConfig
46
47The TraceConfig is a protobuf message
48([reference docs](/docs/reference/trace-config-proto.autogen)) that defines:
49
501. The general behavior of the whole tracing system, e.g.:
51    * The max duration of the trace.
52    * The number of in-memory buffers and their size.
53    * The max size of the output trace file.
54
552. Which data sources to enable and their configuration, e.g.:
56    * For the [kernel tracing data source](/docs/data-sources/cpu-scheduling.md)
57    , which ftrace events to enable.
58    * For the [heap profiler](/docs/data-sources/native-heap-profiler.md), the
59    target process name and sampling rate.
60
61    See the _data sources_ section of the docs for details on how to
62    configure the data sources bundled with Perfetto.
63
643. The `{data source} x {buffer}` mappings: which buffer each data
65    source should write into (see [buffers section](#buffers) below).
66
67The tracing service (`traced`) acts as a configuration dispatcher: it receives
68a config from the `perfetto` cmdline client (or any other
69[Consumer](/docs/concepts/service-model.md#consumer)) and forwards parts of the
70config to the various [Producers](/docs/concepts/service-model.md#producer)
71connected.
72
73When a tracing session is started by a consumer, the tracing service will:
74
75* Read the outer section of the TraceConfig (e.g. `duration_ms`, `buffers`) and
76  use that to determine its own behavior.
77* Read the list of data sources in the `data_sources` section. For each data
78  source listed in the config, if a corresponding name (`"linux.ftrace"` in the
79  example below) was registered, the service will ask the producer process to
80  start that data source, passing it the raw bytes of the
81  [`DataSourceConfig` subsection][dss] verbatim to the data source (See
82  backward/forward compat section below).
83
84![TraceConfig diagram](/docs/images/trace_config.png)
85
86[dss]: /docs/reference/trace-config-proto.autogen#DataSourceConfig
87
88## Buffers
89
90The buffer sections define the number, size and policy of the in-memory buffers
91owned by the tracing service. It looks as follows:
92
93```protobuf
94// Buffer #0
95buffers {
96  size_kb: 4096
97  fill_policy: RING_BUFFER
98}
99
100// Buffer #1
101buffers {
102  size_kb: 8192
103  fill_policy: DISCARD
104}
105```
106
107Each buffer has a fill policy which is either:
108
109* RING_BUFFER (default): the buffer behaves like a ring buffer and writes when
110  full will wrap over and replace the oldest trace data in the buffer.
111
112* DISCARD: the buffer stops accepting data once full. Further write attempts are
113  dropped.
114
115WARNING: DISCARD can have unexpected side-effect with data sources that commit
116data at the end of the trace.
117
118A trace config must define at least one buffer to be valid. In the simplest case
119all data sources will write their trace data into the same buffer.
120
121 While this is
122fine for most basic cases, it can be problematic in cases where different data
123sources write at significantly different rates.
124
125For instance, imagine a trace config that enables both:
126
1271. The kernel scheduler tracer. On a typical Android phone this records
128   ~10000 events/second, writing ~1 MB/s of trace data into the buffer.
129
1302. Memory stat polling. This data source writes the contents of /proc/meminfo
131   into the trace buffer and is configured to poll every 5 seconds, writing
132   ~100 KB per poll interval.
133
134If both data sources are configured to write into the same buffer and such
135buffer is set to 4MB, most traces will contain only one memory snapshot. There
136are very good chances that most traces won't contain any memory snapshot at all,
137even if the 2nd data sources was working perfectly.
138This is because during the 5 s. polling interval, the scheduler data source can
139end up filling the whole buffer, pushing the memory snapshot data out of the
140buffer.
141
142## Dynamic buffer mapping
143
144Data-source <> buffer mappings are dynamic in Perfetto.
145In the simplest case a tracing session can define only one buffer. By default,
146all data sources will record data into that one buffer.
147
148In cases like the example above, it might be preferable separating these data
149sources into different buffers.
150This can be achieved with the `target_buffer` field of the TraceConfig.
151
152![Buffer mapping](/docs/images/trace_config_buffer_mapping.png)
153
154Can be achieved with:
155
156```protobuf
157data_sources {
158  config {
159    name: "linux.ftrace"
160    target_buffer: 0       // <-- This goes into buffer 0.
161    ftrace_config { ... }
162  }
163}
164
165data_sources: {
166  config {
167      name: "linux.sys_stats"
168      target_buffer: 1     // <-- This goes into buffer 1.
169      sys_stats_config { ... }
170  }
171}
172
173data_sources: {
174  config {
175    name: "android.heapprofd"
176    target_buffer: 1       // <-- This goes into buffer 1 as well.
177    heapprofd_config { ... }
178  }
179}
180```
181
182## PBTX vs binary format
183
184There are two ways to pass the trace config when using the `perfetto` cmdline
185client format:
186
187#### Text format
188
189It is the preferred format for human-driven workflows and exploration. It
190allows to pass directly the text file in the PBTX (ProtoBuf TeXtual
191representation) syntax, for the schema defined in the
192[trace_config.proto](/protos/perfetto/config/trace_config.proto)
193(see [reference docs](/docs/reference/trace-config-proto.autogen))
194
195When using this mode pass the `--txt` flag to `perfetto` to indicate the config
196should be interpreted as a PBTX file:
197
198```bash
199perfetto -c /path/to/config.pbtx --txt -o trace_file.pftrace
200```
201
202NOTE: The `--txt` option has been introduced only in Android 10 (Q). Older
203versions support only the binary format.
204
205WARNING: Do not use the text format for machine-to-machine interaction
206benchmark, scripts and tools) as it's more prone to breakages (e.g. if a field
207is renamed or an enum is turned into an integer)
208
209#### Binary format
210
211It is the preferred format for machine-to-machine (M2M) interaction. It involves
212passing the protobuf-encoded binary of the TraceConfig message.
213This can be obtained passing the PBTX in input to the protobuf's `protoc`
214compiler (which can be downloaded
215[here](https://github.com/protocolbuffers/protobuf/releases)).
216
217```bash
218cd ~/code/perfetto  # external/perfetto in the Android tree.
219
220protoc --encode=perfetto.protos.TraceConfig \
221        -I. protos/perfetto/config/perfetto_config.proto \
222        < config.txpb \
223        > config.bin
224```
225
226and then passing it to perfetto as follows, without the `--txt` argument:
227
228```bash
229perfetto -c config.bin -o trace_file.pftrace
230```
231
232## {#long-traces} Streaming long traces
233
234By default Perfetto keeps the full trace buffer(s) in memory and writes it into
235the destination file (the `-o` cmdline argument) only at the end of the tracing
236session. This is to reduce the perf-intrusiveness of the tracing system.
237This, however, limits the max size of the trace to the physical memory size of
238the device, which is often too limiting.
239
240In some cases (e.g., benchmarks, hard to repro cases) it is desirable to capture
241traces that are way larger than that, at the cost of extra I/O overhead.
242
243To achieve that, Perfetto allows to periodically write the trace buffers into
244the target file (or stdout) using the following TraceConfig fields:
245
246* `write_into_file (bool)`:
247When true periodically drains the trace buffers into the output
248file. When this option is enabled, the userspace buffers need to be just
249big enough to hold tracing data between two write periods.
250The buffer sizing depends on the activity of the device.
251The data rate of a typical trace is ~1-4 MB/s. So a 16MB in-memory buffer can
252hold for up write periods of ~4 seconds before starting to lose data.
253
254* `file_write_period_ms (uint32)`:
255Overrides the default drain period (5s). Shorter periods require a smaller
256userspace buffer but increase the performance intrusiveness of tracing. If
257the period given is less than 100ms, the tracing service will use a period
258of 100ms.
259
260* `max_file_size_bytes (uint64)`:
261If set, stops the tracing session after N bytes have been written. Used to
262cap the size of the trace.
263
264For a complete example of a working trace config in long-tracing mode see
265[`/test/configs/long_trace.cfg`](/test/configs/long_trace.cfg).
266
267Summary: to capture a long trace just set `write_into_file:true`, set a long
268         `duration_ms` and use an in-memory buffer size of 32MB or more.
269
270## Data-source specific config
271
272Alongside the trace-wide configuration parameters, the trace config also defines
273data-source-specific behaviors. At the proto schema level, this is defined in
274the `DataSourceConfig` section of `TraceConfig`:
275
276From [data_source_config.proto](/protos/perfetto/config/data_source_config.proto):
277
278```protobuf
279message TraceConfig {
280  ...
281  repeated DataSource data_sources = 2;  // See below.
282}
283
284message DataSource {
285  optional protos.DataSourceConfig config = 1;  // See below.
286  ...
287}
288
289message DataSourceConfig {
290  optional string name = 1;
291  ...
292  optional FtraceConfig ftrace_config = 100 [lazy = true];
293  ...
294  optional AndroidPowerConfig android_power_config = 106 [lazy = true];
295}
296```
297
298Fields like `ftrace_config`, `android_power_config` are examples of data-source
299specific configs. The tracing service will completely ignore the contents of
300those fields and route the whole DataSourceConfig object to any data source
301registered with the same name.
302
303The `[lazy=true]` marker has a special implication in the
304[protozero](/docs/design-docs/protozero.md) code generator. Unlike standard
305nested messages, it generates raw accessors (e.g.,
306`const std::string& ftrace_config_raw()` instead of
307`const protos::FtraceConfig& ftrace_config()`). This is to avoid injecting too
308many `#include` dependencies and avoiding binary size bloat in the code that
309implements data sources.
310
311#### A note on backwards/forward compatibility
312The tracing service will route the raw binary blob of the `DataSourceConfig`
313message to the data sources with a matching name, without attempting to decode
314and re-encode it. If the `DataSourceConfig` section of the trace config contains
315a new field that didn't exist at the time when the service was built, the
316service will still pass the `DataSourceConfig` through to the data source.
317This allows to introduced new data sources without needing the service to
318know anything about them upfront.
319
320TODO: we are aware of the fact that today extending the `DataSourceConfig` with
321a custom proto requires changing the `data_source_config.proto` in the Perfetto
322repo, which is unideal for external projects. The long-term plan is to reserve
323a range of fields for non-upstream extensions and provide generic templated
324accessors for client code. Until then, we accept patches upstream to introduce
325ad-hoc configurations for your own data sources.
326
327## Multi-process data sources
328
329Some data sources are singletons. E.g., in the case of scheduler tracing that
330Perfetto ships on Android, there is only data source for the whole system,
331owned by the `traced_probes` service.
332
333However, in the general case multiple processes can advertise the same data
334source. This is the case, for instance, when using the
335[Perfetto SDK](/docs/instrumentation/tracing-sdk.md) for userspace
336instrumentation.
337
338If this happens, when starting a tracing session that specifies that data
339source in the trace config, Perfetto by default will ask all processes that
340advertise that data source to start it.
341
342In some cases it might be desirable to further limit the enabling of the data
343source to a specific process (or set of processes). That is possible through the
344`producer_name_filter` and `producer_name_regex_filter`.
345
346NOTE: the typical Perfetto run-time model is: one process == one Perfetto
347      Producer; one Producer typically hosts multiple data sources.
348
349When those filters are set, the Perfetto tracing service will activate the data
350source only in the subset of producers matching the filter.
351
352Example:
353
354```protobuf
355buffers {
356  size_kb: 4096
357}
358
359data_sources {
360  config {
361    name: "track_event"
362
363    # Enable the data source only on Chrome and Chrome canary.
364    producer_name_filter: "com.android.chrome"
365    producer_name_filter: "com.google.chrome.canary"
366  }
367}
368```
369
370## Triggers
371
372In nominal conditions, a tracing session has a lifecycle that simply matches the
373invocation of the `perfetto` cmdline client: trace data recording starts when
374the TraceConfig is passed to `perfetto` and ends when either the
375`TraceConfig.duration_ms` has elapsed, or when the cmdline client terminates.
376
377Perfetto supports an alternative mode of either starting or stopping the trace
378which is based on triggers. The overall idea is to declare in the trace config
379itself:
380
381* A set of triggers, which are just free-form strings.
382* Whether a given trigger should cause the trace to be started or stopped, and
383  the start/stop delay.
384
385Why using triggers? Why can't one just start perfetto or kill(SIGTERM) it when
386needed? The rationale of all this is the security model: in most Perfetto
387deployments (e.g., on Android) only privileged entities (e.g., adb shell) can
388configure/start/stop tracing. Apps are unprivileged in this sense and they
389cannot control tracing.
390
391Triggers offer a way to unprivileged apps to control, in a limited fashion, the
392lifecycle of a tracing session. The conceptual model is:
393
394* The privileged Consumer (see
395  [_Service model_](/docs/concepts/service-model.md)), i.e. the entity
396  that is normally authorized to start tracing (e.g., adb shell in Android),
397  declares upfront what are the possible trigger names for the trace and what
398  they will do.
399* Unprivileged entities (any random app process) can activate those triggers.
400  Unprivileged entities don't get a say on what the triggers will do, they only
401  communicate that an event happened.
402
403Triggers can be signaled via the cmdline util
404
405```bash
406/system/bin/trigger_perfetto "trigger_name"
407```
408
409(or also by starting an independent trace session which uses only the
410`activate_triggers: "trigger_name"` field in the config)
411
412There are two types of triggers:
413
414#### Start triggers
415
416Start triggers allow activating a tracing session only after some significant
417event has happened. Passing a trace config that has `START_TRACING` trigger
418causes the tracing session to stay idle (i.e. not recording any data) until either
419the trigger is hit or the `duration_ms` timeout is hit.
420
421Example config:
422```protobuf
423// If no trigger is hit, the trace will end without having recorded any data
424// after 30s.
425duration_ms: 30000
426
427// If the "myapp_is_slow" is hit, the trace starts recording data and will be
428// stopped after 5s.
429trigger_config {
430  trigger_mode: START_TRACING
431  triggers {
432    name: "myapp_is_slow"
433    stop_delay_ms: 5000
434  }
435}
436
437// The rest of the config is as usual.
438buffers { ... }
439data_sources { ... }
440```
441
442#### Stop triggers
443
444STOP_TRACING triggers allow to prematurely finalize a trace when the trigger is
445hit. In this mode the trace starts immediately when the `perfetto` client is
446invoked (like in nominal cases). The trigger acts as a premature finalization
447signal.
448
449This can be used to use perfetto in flight-recorder mode. By starting a trace
450with buffers configured in `RING_BUFFER` mode and `STOP_TRACING` triggers,
451the trace will be recorded in a loop and finalized when the culprit event is
452detected. This is key for events where the root cause is in the recent past
453(e.g., the app detects a slow scroll or a missing frame).
454
455Example config:
456```protobuf
457// If no trigger is hit, the trace will end after 30s.
458duration_ms: 30000
459
460// If the "missed_frame" is hit, the trace is stopped after 1s.
461trigger_config {
462  trigger_mode: STOP_TRACING
463  triggers {
464    name: "missed_frame"
465    stop_delay_ms: 1000
466  }
467}
468
469// The rest of the config is as usual.
470buffers { ... }
471data_sources { ... }
472```
473
474## Other resources
475
476* [TraceConfig Reference](/docs/reference/trace-config-proto.autogen)
477* [Buffers and dataflow](/docs/concepts/buffers.md)
478