Lines Matching full:benchmark
25 [Custom Benchmark Name](#custom-benchmark-name)
53 [Using RegisterBenchmark](#using-register-benchmark)
76 Benchmark Time(ns) CPU(ns) Iterations
86 The `benchmarks` attribute contains a list of every benchmark run. Example json
141 Write benchmark results to a file with the `--benchmark_out=<filename>` option
146 [is not parsable](https://github.com/google/benchmark/issues/794) by csv
175 Benchmark Time CPU Iterations
202 pwd: /home/user/benchmark/
203 Benchmark Time CPU Iterations
212 benchmark::AddCustomContext("foo", "bar");
222 When the benchmark binary is executed, each benchmark function is run serially.
224 benchmark a few times and measuring the time taken and ensuring that the
225 ultimate result will be statistically stable. As such, faster benchmark
226 functions will be run for more iterations than slower benchmark functions, and
229 In all cases, the number of iterations for which the benchmark is run is
230 governed by the amount of time the benchmark takes. Concretely, the number of
233 set per benchmark by calling `MinTime` on the registered benchmark object.
237 option, or at registration time, the benchmark function will be run several
240 As well as the per-benchmark entries, a preamble in the report will include
247 Global setup/teardown specific to each benchmark can be done by
250 The setup/teardown callbacks will be invoked once for each benchmark.
251 If the benchmark is multi-threaded (will run in k threads), they will be invoked exactly once before
253 If the benchmark uses different size groups of threads, the above will be true for each size group.
258 static void DoSetup(const benchmark::State& state) {
261 static void DoTeardown(const benchmark::State& state) {
264 static void BM_func(benchmark::State& state) {...}
266 BENCHMARK(BM_func)->Arg(1)->Arg(3)->Threads(16)->Threads(32)->Setup(DoSetup)->Teardown(DoTeardown);
285 static void BM_memcpy(benchmark::State& state) {
296 BENCHMARK(BM_memcpy)->Arg(8)->Arg(64)->Arg(512)->Arg(1<<10)->Arg(8<<10);
301 the specified range and will generate a benchmark for each such argument.
304 BENCHMARK(BM_memcpy)->Range(8, 8<<10);
312 BENCHMARK(BM_memcpy)->RangeMultiplier(2)->Range(8, 8<<10);
318 example shows a method of defining a dense range. It is then used to benchmark
322 static void BM_DenseRange(benchmark::State& state) {
325 benchmark::DoNotOptimize(v.data());
326 benchmark::ClobberMemory();
329 BENCHMARK(BM_DenseRange)->DenseRange(0, 1024, 128);
334 You might have a benchmark that depends on two or more inputs. For example, the
339 static void BM_SetInsert(benchmark::State& state) {
349 BENCHMARK(BM_SetInsert)
362 product of the two specified ranges and will generate a benchmark for each such
367 BENCHMARK(BM_SetInsert)->Ranges({{1<<10, 8<<10}, {128, 512}});
373 benchmark input for each combination in the product of the supplied vectors.
377 BENCHMARK(BM_SetInsert)
379 // would generate the same benchmark arguments as
380 BENCHMARK(BM_SetInsert)
400 BENCHMARK(BM_SetInsert)
402 benchmark::CreateRange(8, 128, /*multi=*/2),
403 benchmark::CreateDenseRange(1, 4, /*step=*/1)
405 // would generate the same benchmark arguments as
406 BENCHMARK(BM_SetInsert)
415 benchmark. The following example enumerates a dense range on one parameter,
419 static void CustomArguments(benchmark::internal::Benchmark* b) {
424 BENCHMARK(BM_SetInsert)->Apply(CustomArguments);
427 ### Passing Arbitrary Arguments to a Benchmark
429 In C++11 it is possible to define a benchmark that takes an arbitrary number
431 macro creates a benchmark that invokes `func` with the `benchmark::State` as
433 The `test_case_name` is appended to the name of the benchmark and
438 void BM_takes_args(benchmark::State& state, ExtraArgs&&... extra_args) {
441 // Registers a benchmark named "BM_takes_args/int_string_test" that passes
447 avoid modifying global state inside of a benchmark.
458 static void BM_StringCompare(benchmark::State& state) {
462 benchmark::DoNotOptimize(s1.compare(s2));
466 BENCHMARK(BM_StringCompare)
467 ->RangeMultiplier(2)->Range(1<<10, 1<<18)->Complexity(benchmark::oN);
474 BENCHMARK(BM_StringCompare)
482 BENCHMARK(BM_StringCompare)->RangeMultiplier(2)
483 ->Range(1<<10, 1<<18)->Complexity([](benchmark::IterationCount n)->double{return n; });
486 <a name="custom-benchmark-name" />
488 ## Custom Benchmark Name
490 You can change the benchmark's name as follows:
493 BENCHMARK(BM_memcpy)->Name("memcpy")->RangeMultiplier(2)->Range(8, 8<<10);
496 The invocation will execute the benchmark as before using `BM_memcpy` but changes
507 template <class Q> void BM_Sequential(benchmark::State& state) {
523 // C++11 or newer, you can use the BENCHMARK macro with template parameters:
524 BENCHMARK(BM_Sequential<WaitQueue<int>>)->Range(1<<0, 1<<10);
528 Three macros are provided for adding benchmark templates.
532 #define BENCHMARK(func<...>) // Takes any number of parameters.
545 `::benchmark::Fixture` and then creating/registering the tests using the
555 class MyFixture : public benchmark::Fixture {
557 void SetUp(const ::benchmark::State& state) {
560 void TearDown(const ::benchmark::State& state) {
564 BENCHMARK_F(MyFixture, FooTest)(benchmark::State& st) {
570 BENCHMARK_DEFINE_F(MyFixture, BarTest)(benchmark::State& st) {
591 class MyFixture : public benchmark::Fixture {};
593 BENCHMARK_TEMPLATE_F(MyFixture, IntTest, int)(benchmark::State& st) {
599 BENCHMARK_TEMPLATE_DEFINE_F(MyFixture, DoubleTest, double)(benchmark::State& st) {
616 static void UserCountersExample1(benchmark::State& state) {
633 When the benchmark finishes, the counters from each thread will be summed;
634 the resulting sum is the value which will be shown for the benchmark.
640 is 1k a 1000 (default, `benchmark::Counter::OneK::kIs1000`), or 1024
641 (`benchmark::Counter::OneK::kIs1024`)?
648 // by the duration of the benchmark.
650 state.counters["FooRate"] = Counter(numFoos, benchmark::Counter::kIsRate);
653 // by the duration of the benchmark, and the result inverted.
655 …state.counters["FooInvRate"] = Counter(numFoos, benchmark::Counter::kIsRate | benchmark::Counter::…
659 state.counters["FooAvg"] = Counter(numFoos, benchmark::Counter::kAvgThreads);
662 state.counters["FooAvgRate"] = Counter(numFoos,benchmark::Counter::kAvgThreadsRate);
665 …ers["BytesProcessed"] = Counter(state.range(0), benchmark::Counter::kIsIterationInvariantRate, ben…
687 or where there are only a couple of lines per benchmark. Here's an example of
692 Benchmark Time CPU Iterations UserCounters...
711 passing the flag `--benchmark_counters_tabular=true` to the benchmark
713 a lot of lines per individual benchmark. Note that this will trigger a
720 Benchmark Time CPU Iterations Bar Bat Baz Foo
731 Benchmark Time CPU Iterations
746 Note above the additional header printed when the benchmark changes from
754 In a multithreaded test (benchmark invoked by multiple threads simultaneously),
756 the start of the benchmark loop, and all will have finished before any thread
757 exits the benchmark loop. (This behavior is also provided by the `KeepRunning()`
762 static void BM_MultiThreaded(benchmark::State& state) {
773 BENCHMARK(BM_MultiThreaded)->Threads(2);
781 BENCHMARK(BM_test)->Range(8, 8<<10)->UseRealTime();
791 If the benchmark itself uses threads internally, this measurement may not
804 static void BM_OpenMP(benchmark::State& state) {
810 // run the benchmark loop. Depending on the internal implementation detail may
813 BENCHMARK(BM_OpenMP)->Range(8, 8<<10);
817 // run the benchmark loop. This will always be meaningful, an will match the
820 BENCHMARK(BM_OpenMP)->Range(8, 8<<10)->UseRealTime();
823 // run the benchmark loop. This will always measure to no less than the
825 BENCHMARK(BM_OpenMP)->Range(8, 8<<10)->MeasureProcessCPUTime();
828 // wall clock to decide for how long to run the benchmark loop.
829 BENCHMARK(BM_OpenMP)->Range(8, 8<<10)->MeasureProcessCPUTime()->UseRealTime();
836 that loop, every iteration, but without counting that time to the benchmark time.
841 static void BM_SetInsert_With_Timer_Control(benchmark::State& state) {
852 BENCHMARK(BM_SetInsert_With_Timer_Control)->Ranges({{1<<10, 8<<10}, {128, 512}});
865 `SetIterationTime` once per iteration of the benchmark loop to
875 static void BM_ManualTiming(benchmark::State& state) {
894 BENCHMARK(BM_ManualTiming)->Range(1, 1<<17)->UseManualTime();
901 If a benchmark runs a few milliseconds it may be hard to visually compare the
906 BENCHMARK(BM_test)->Unit(benchmark::kMillisecond);
914 the `benchmark::DoNotOptimize(...)` and `benchmark::ClobberMemory()`
918 static void BM_test(benchmark::State& state) {
922 benchmark::DoNotOptimize(x += i);
957 static void BM_vector_push_back(benchmark::State& state) {
961 benchmark::DoNotOptimize(v.data()); // Allow v.data() to be clobbered.
963 benchmark::ClobberMemory(); // Force 42 to be written to memory.
974 By default each benchmark is run once and that single result is reported.
977 benchmark.
979 The number of runs of each benchmark is specified globally by the
980 `--benchmark_repetitions` flag or on a per benchmark basis by calling
981 `Repetitions` on the registered benchmark object. When a benchmark is run more
997 registered benchmark object overrides the value of the appropriate flag for that
998 benchmark.
1010 void BM_spin_empty(benchmark::State& state) {
1013 benchmark::DoNotOptimize(x);
1018 BENCHMARK(BM_spin_empty)
1029 void BM_spin_empty(benchmark::State& state) {
1032 benchmark::DoNotOptimize(x);
1037 BENCHMARK(BM_spin_empty)
1040 }, benchmark::StatisticUnit::Percentage)
1044 <a name="using-register-benchmark" />
1051 pointer to a new benchmark with the specified `name` that invokes
1052 `func(st, args...)` where `st` is a `benchmark::State` object.
1054 Unlike the `BENCHMARK` registration macros, which can only be used at the global
1056 benchmark tests to be registered programmatically.
1059 as a benchmark. Including capturing lambdas and function objects.
1063 auto BM_test = [](benchmark::State& st, auto Inputs) { /* ... */ };
1067 benchmark::RegisterBenchmark(test_input.name(), BM_test, test_input);
1068 benchmark::Initialize(&argc, argv);
1069 benchmark::RunSpecifiedBenchmarks();
1070 benchmark::Shutdown();
1079 communication, occur within a benchmark the
1081 of benchmark and report the error. Note that only future iterations of the
1082 `KeepRunning()` are skipped. For the ranged-for version of the benchmark loop
1084 Users may explicitly return to exit the benchmark immediately.
1086 The `SkipWithError(...)` function may be used at any point within the benchmark,
1087 including before and after the benchmark loop. Moreover, if `SkipWithError(...)`
1088 has been used, it is not required to reach the benchmark loop and one may return
1089 from the benchmark function early.
1094 static void BM_test(benchmark::State& state) {
1110 static void BM_test_ranged_fo(benchmark::State & state) {
1134 static void BM_Fast(benchmark::State &state) {
1139 BENCHMARK(BM_Fast);
1152 call benchmark::State::StartKeepRunning()
1176 call benchmark::State::StartKeepRunning()
1182 the benchmark loop should be preferred.
1191 ***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will in…
1194 you might want to disable the CPU frequency scaling while running the benchmark: