Lines Matching full:benchmark
1 # benchmark chapter
2 …uild Status](https://travis-ci.org/google/benchmark.svg?branch=master)](https://travis-ci.org/goog…
3 …7t1tk7cpxs/branch/master?svg=true)](https://ci.appveyor.com/project/google/benchmark/branch/master)
4 …age Status](https://coveralls.io/repos/google/benchmark/badge.svg)](https://coveralls.io/r/google/…
9 [Discussion group](https://groups.google.com/d/forum/benchmark-discuss)
23 $ git clone https://github.com/google/benchmark.git
24 # Benchmark requires Google Test as a dependency. Add the source tree as a subdirectory.
25 $ git clone https://github.com/google/googletest.git benchmark/googletest
27 $ cmake -G <generator> [options] ../benchmark
32 Note that Google Benchmark requires Google Test to build and run the tests. This
35 * Checkout the Google Test sources into `benchmark/googletest` as above.
57 git clone https://github.com/google/benchmark.git
58 cd benchmark
80 [`v2` branch](https://github.com/google/benchmark/tree/v2). Users who wish
93 benchmark function using the `BENCHMARK` macro, and ensure an appropriate `main`
97 #include <benchmark/benchmark.h>
99 static void BM_StringCreation(benchmark::State& state) {
103 // Register the function as a benchmark
104 BENCHMARK(BM_StringCreation);
106 // Define another benchmark
107 static void BM_StringCopy(benchmark::State& state) {
112 BENCHMARK(BM_StringCopy);
117 Don't forget to inform your linker to add benchmark library e.g. through
122 The benchmark library will measure and report the timing for code within the
129 [issue #67](https://github.com/google/benchmark/issues/67) for more details. You
147 static void BM_memcpy(benchmark::State& state) {
158 BENCHMARK(BM_memcpy)->Arg(8)->Arg(64)->Arg(512)->Arg(1<<10)->Arg(8<<10);
163 the specified range and will generate a benchmark for each such argument.
166 BENCHMARK(BM_memcpy)->Range(8, 8<<10);
174 BENCHMARK(BM_memcpy)->RangeMultiplier(2)->Range(8, 8<<10);
178 You might have a benchmark that depends on two or more inputs. For example, the
183 static void BM_SetInsert(benchmark::State& state) {
193 BENCHMARK(BM_SetInsert)
206 product of the two specified ranges and will generate a benchmark for each such
210 BENCHMARK(BM_SetInsert)->Ranges({{1<<10, 8<<10}, {128, 512}});
215 benchmark. The following example enumerates a dense range on one parameter,
219 static void CustomArguments(benchmark::internal::Benchmark* b) {
224 BENCHMARK(BM_SetInsert)->Apply(CustomArguments);
233 static void BM_StringCompare(benchmark::State& state) {
237 benchmark::DoNotOptimize(s1.compare(s2));
241 BENCHMARK(BM_StringCompare)
242 ->RangeMultiplier(2)->Range(1<<10, 1<<18)->Complexity(benchmark::oN);
249 BENCHMARK(BM_StringCompare)
257 BENCHMARK(BM_StringCompare)->RangeMultiplier(2)
267 template <class Q> void BM_Sequential(benchmark::State& state) {
283 Three macros are provided for adding benchmark templates.
301 static void BM_Fast(benchmark::State &state) {
306 BENCHMARK(BM_Fast);
319 call benchmark::State::StartKeepRunning()
343 call benchmark::State::StartKeepRunning()
349 the benchmark loop should be preferred.
351 ## Passing arbitrary arguments to a benchmark
352 In C++11 it is possible to define a benchmark that takes an arbitrary number
354 macro creates a benchmark that invokes `func` with the `benchmark::State` as
356 The `test_case_name` is appended to the name of the benchmark and
361 void BM_takes_args(benchmark::State& state, ExtraArgs&&... extra_args) {
364 // Registers a benchmark named "BM_takes_args/int_string_test" that passes
369 avoid modifying global state inside of a benchmark.
376 pointer to a new benchmark with the specified `name` that invokes
377 `func(st, args...)` where `st` is a `benchmark::State` object.
379 Unlike the `BENCHMARK` registration macros, which can only be used at the global
381 benchmark tests to be registered programmatically.
384 as a benchmark. Including capturing lambdas and function objects.
388 auto BM_test = [](benchmark::State& st, auto Inputs) { /* ... */ };
392 benchmark::RegisterBenchmark(test_input.name(), BM_test, test_input);
393 benchmark::Initialize(&argc, argv);
394 benchmark::RunSpecifiedBenchmarks();
399 In a multithreaded test (benchmark invoked by multiple threads simultaneously),
401 the start of the benchmark loop, and all will have finished before any thread
402 exits the benchmark loop. (This behavior is also provided by the `KeepRunning()`
407 static void BM_MultiThreaded(benchmark::State& state) {
418 BENCHMARK(BM_MultiThreaded)->Threads(2);
426 BENCHMARK(BM_test)->Range(8, 8<<10)->UseRealTime();
434 that loop, every iteration, but without counting that time to the benchmark time.
438 static void BM_SetInsert_With_Timer_Control(benchmark::State& state) {
449 BENCHMARK(BM_SetInsert_With_Timer_Control)->Ranges({{1<<10, 8<<10}, {128, 512}});
458 `SetIterationTime` once per iteration of the benchmark loop to
468 static void BM_ManualTiming(benchmark::State& state) {
487 BENCHMARK(BM_ManualTiming)->Range(1, 1<<17)->UseManualTime();
492 the `benchmark::DoNotOptimize(...)` and `benchmark::ClobberMemory()`
496 static void BM_test(benchmark::State& state) {
500 benchmark::DoNotOptimize(x += i);
535 static void BM_vector_push_back(benchmark::State& state) {
539 benchmark::DoNotOptimize(v.data()); // Allow v.data() to be clobbered.
541 benchmark::ClobberMemory(); // Force 42 to be written to memory.
549 If a benchmark runs a few milliseconds it may be hard to visually compare the
554 BENCHMARK(BM_test)->Unit(benchmark::kMillisecond);
558 By default each benchmark is run once and that single result is reported.
561 benchmark.
563 The number of runs of each benchmark is specified globally by the
564 `--benchmark_repetitions` flag or on a per benchmark basis by calling
565 `Repetitions` on the registered benchmark object. When a benchmark is run more
580 registered benchmark object overrides the value of the appropriate flag for that
581 benchmark.
591 void BM_spin_empty(benchmark::State& state) {
594 benchmark::DoNotOptimize(x);
599 BENCHMARK(BM_spin_empty)
608 first defining a type that derives from `::benchmark::Fixture` and then
618 class MyFixture : public benchmark::Fixture {};
620 BENCHMARK_F(MyFixture, FooTest)(benchmark::State& st) {
626 BENCHMARK_DEFINE_F(MyFixture, BarTest)(benchmark::State& st) {
645 class MyFixture : public benchmark::Fixture {};
647 BENCHMARK_TEMPLATE_F(MyFixture, IntTest, int)(benchmark::State& st) {
653 BENCHMARK_TEMPLATE_DEFINE_F(MyFixture, DoubleTest, double)(benchmark::State& st) {
668 static void UserCountersExample1(benchmark::State& state) {
685 When the benchmark finishes, the counters from each thread will be summed;
686 the resulting sum is the value which will be shown for the benchmark.
692 `benchmark::Counter::OneK::kIs1000`), or 1024
693 (`benchmark::Counter::OneK::kIs1024`)?
700 // by the duration of the benchmark.
701 state.counters["FooRate"] = Counter(numFoos, benchmark::Counter::kIsRate);
705 state.counters["FooAvg"] = Counter(numFoos, benchmark::Counter::kAvgThreads);
708 state.counters["FooAvgRate"] = Counter(numFoos,benchmark::Counter::kAvgThreadsRate);
711 …ers["BytesProcessed"] = Counter(state.range(0), benchmark::Counter::kIsIterationInvariantRate, ben…
731 or where there are only a couple of lines per benchmark. Here's an example of
736 Benchmark Time CPU Iterations UserCounters...
755 passing the flag `--benchmark_counters_tabular=true` to the benchmark
757 a lot of lines per individual benchmark. Note that this will trigger a
764 Benchmark Time CPU Iterations Bar Bat Baz Foo
775 Benchmark Time CPU Iterations
789 Note above the additional header printed when the benchmark changes from
796 communication, occur within a benchmark the
798 of benchmark and report the error. Note that only future iterations of the
799 `KeepRunning()` are skipped. For the ranged-for version of the benchmark loop
801 Users may explicitly return to exit the benchmark immediately.
803 The `SkipWithError(...)` function may be used at any point within the benchmark,
804 including before and after the benchmark loop.
809 static void BM_test(benchmark::State& state) {
825 static void BM_test_ranged_fo(benchmark::State & state) {
843 Benchmark Time CPU Iterations
852 When the benchmark binary is executed, each benchmark function is run serially.
854 benchmark a few times and measuring the time taken and ensuring that the
855 ultimate result will be statistically stable. As such, faster benchmark
856 functions will be run for more iterations than slower benchmark functions, and
859 In all cases, the number of iterations for which the benchmark is run is
860 governed by the amount of time the benchmark takes. Concretely, the number of
863 set per benchmark by calling `MinTime` on the registered benchmark object.
867 option, or at registration time, the benchmark function will be run several
870 As well as the per-benchmark entries, a preamble in the report will include
882 Benchmark Time(ns) CPU(ns) Iterations
892 The `benchmarks` attribute contains a list of every benchmark run. Example json
942 The library supports writing the output of the benchmark to a file specified
952 By default, benchmark builds as a debug library. You will see a warning in the
973 Google Benchmark uses C++11 when building the library. As such we require
991 ***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will in…
993 you might want to disable the CPU frequency scaling while running the benchmark: