1# Overview 2 3The project provides an easy way to save before/after CPU traces from Microbenchmark runs, and compare them visually using Differential Flame Graphs. 4 5 6 7Areas where the code got slower are highlighted in red, while areas that are now faster are marked in blue; the intensity of the colour is proportional to the size of the difference. 8 9See also the [end-to-end demo (video)](https://drive.google.com/file/d/119nI_zlAMbTHzh-Rdzf8UuUVCGEKnKFQ/view?usp=drive_link&resourcekey=0-SRRmKgVZYfAlnkL4Hvh-cg). 10 11# Usage 12 13## Interacting with the script 14 15- Overview of all commands: `./bench-flame-diff.sh -h` 16- Help for a specific command: `./bench-flame-diff.sh <command> -h` 17 18## First usage 19 20On first usage, initialise all dependencies by running: `./bench-flame-diff.sh init` 21 22## General workflow 23 241. Run a specific Microbenchmark with CPU Stack sampling enabled (see below for instructions) 251. Save the trace as _base_ for comparison using `./bench-flame-diff.sh save`. It's worth picking a good names for the saved traces since you're likely going to e.g. re-use the _base_ while iterating on code changes. 261. Apply changes in your code and run the same benchmark as in step 1 271. Save the trace as _current_ `./bench-flame-diff.sh save` 281. Compare both traces using `./bench-flame-diff.sh diff` which will create and open a diff in a web browser 291. Toggle between graphs using the buttons on the top: 30 - `base`: flamegraph for the _base_ trace 31 - `base-vs-curr`: differential flame graph showing _base_ vs _current_ on the _base_ trace 32 - `curr`: flamegraph for the _current_ trace 33 - `curr-vs-base`: differential flame graph showing _base_ vs _current_ on the _current_ trace 341. You can later go back to generated diffs using `./bench-flame-diff.sh open` 35 36# Misc 37 38## Enabling stack sampling in Benchmark traces 39 40This can be done in CLI or by editing `build.gradle`. Full documentation is [here](https://developer.android.com/topic/performance/benchmarking/microbenchmark-profile). 41 42Quick CLI example: 43``` 44# pick a target benchmark 45tgt=:compose:foundation:foundation-benchmark:connectedCheck 46 47# create a regex that targets a specific benchmark (test) 48test_rx="androidx.compose.foundation.benchmark.lazy.LazyListScrollingBenchmark.scrollProgrammatically_noNewItems\[.*Row.*\]" 49 50# run the benchmark and gather a 5 second (default) stack sample at 1000 Hz (default) 51./gradlew $tgt -Pandroid.testInstrumentationRunnerArguments.tests_regex="$test_rx" \ 52 -P android.testInstrumentationRunnerArguments.androidx.benchmark.profiling.mode=StackSampling \ 53 -P android.testInstrumentationRunnerArguments.androidx.benchmark.profiling.sampleDurationSeconds=5 \ 54 -P android.testInstrumentationRunnerArguments.androidx.benchmark.profiling.sampleFrequency=1000 55``` 56 57## CLI completion 58 59Generate shell-specific completion files with `./generate-completion.sh`. 60 61Then, source in your shell config, e.g.: 62- For `bash`: `dst="$(pwd)/completion_bash.sh"; echo "source '$dst'" >> ~/.bashrc` 63- For `zsh`: `dst="$(pwd)/completion_zsh.sh"; echo "source '$dst'" >> ~/.zshrc` 64 65After restarting the shell session, you will be able to 'tab-autocomplete' commands and argument names. 66 67# Dependencies 68 69On top of dependencies discoverable with `./gradlew app:dependencies` the project depends on: 70- https://github.com/brendangregg/FlameGraph 71- https://android.googlesource.com/platform/system/extras/+/refs/heads/main/simpleperf/scripts 72 73Both are fetched from the network in the `init` command and pinned to known-good-revisions. 74 75# Reporting issues 76 77File an issue on Buganizer using [this link](https://b.corp.google.com/issues/new?component=1229612&hotlistIds=3622386&hotlistIds=5709693&assignee=jgielzak@google.com&title=bench-flame-diff:%20) or reach out directly to [jgielzak@](http://go/moma/chat?with=jgielzak). 78 79Known issues and future work items are tracked [here](https://b.corp.google.com/hotlists/5709693). 80
