1# C++ benchmark测试指导 2 3基准测试(benchmarking)是一种测量和评估软件性能指标的活动,可以在某个时候通过基准测试建立一个已知的性能水平(称为基准线),当系统的软硬件环境发生变化之后再进行一次基准测试以确定那些变化对性能的影响。这是基准测试最常见的用途,其他用途包括测定某种负载水平下的性能极限、管理系统或环境的变化、发现可能导致性能问题的条件等。 4 5## 用例编写 6 7``` 8/* 9 * Copyright (c) 2021 XXXX Device Co., Ltd. 10 * Licensed under the Apache License, Version 2.0 (the "License"); 11 * you may not use this file except in compliance with the License. 12 * You may obtain a copy of the License at 13 * 14 * http://www.apache.org/licenses/LICENSE-2.0 15 * 16 * Unless required by applicable law or agreed to in writing, software 17 * distributed under the License is distributed on an "AS IS" BASIS, 18 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 19 * See the License for the specific language governing permissions and 20 * limitations under the License. 21 */ 22 23#include <benchmark/benchmark.h> 24#include <string> 25#include <vector> 26 27using namespace std; 28 29namespace { 30 /** 31 * @tc.name: BenchmarkTestExample 32 * @tc.desc: Testcase for testing 'SimpleExample' function. 33 * @tc.type: PERF 34 * @tc.require: Issue Number 35 */ 36 size_t SimpleExample() 37 { 38 string str = "benchmark test"; 39 return str.size(); 40 } 41 42 static void BenchmarkTestExample(benchmark::State &state) 43 { 44 for (auto _ : state) { 45 /* @tc.steps: step1.call SimpleExample in loop */ 46 SimpleExample(); 47 } 48 } 49 50 /* Register the function as a benchmark */ 51 BENCHMARK(BenchmarkTestExample); 52 /* Register benchmark and explicitly set the fix iterations */ 53 BENCHMARK(BenchmarkTestExample)->Iterations(1000); 54 55 /** 56 * @tc.name: BenchmarkTestVectorOperator 57 * @tc.desc: Testcase for testing "AccessVectorElementByOperator" 58 * function. 59 * @tc.type: PERF 60 * @tc.require: Issue Number 61 */ 62 void AccessVectorElementByOperator() 63 { 64 constexpr int testLen = 5; 65 std::vector<int> testVec(testLen, 0); 66 for (int i = 0; i < testLen; i++) { 67 testVec[i] = i * i; 68 } 69 } 70 71 static void BenchmarkTestVectorOperator(benchmark::State &state) 72 { 73 for (auto _ : state) { 74 /* @tc.steps: step1.call AccessVectorElementByOperator in loop */ 75 AccessVectorElementByOperator(); 76 } 77 } 78 79 /* 80 * Register the function as a benchmark, set iterations repetitions. 81 * And set "ReportAggregatesOnly", it will display the statistics Mean, 82 * Median and Standard Deviation of Repeated Benchmarks. 83 */ 84 BENCHMARK(BenchmarkTestVectorOperator)->Iterations(1000)->Repetitions(3)-> 85 ReportAggregatesOnly(); 86 87 /** 88 * @tc.name: BenchmarkTestVectorAt 89 * @tc.desc: Testcase for testing "AccessVectorElementByAt" 90 * function. 91 * @tc.type: PERF 92 * @tc.require: Issue Number 93 */ 94 void AccessVectorElementByAt() 95 { 96 constexpr int testLen = 5; 97 std::vector<int> testVec(testLen, 0); 98 for (int i = 0; i < testLen; i++) { 99 testVec.at(i) = i * i; 100 } 101 } 102 103 static void BenchmarkTestVectorAt(benchmark::State &state) 104 { 105 for (auto _ : state) { 106 /* @tc.steps: step1.call AccessVectorElementByAt in loop */ 107 AccessVectorElementByAt(); 108 } 109 } 110 111 BENCHMARK(BenchmarkTestVectorAt)->Iterations(1000)->Repetitions(3)-> 112 ReportAggregatesOnly(); 113 114 /** 115 * @tc.name: CalculatedAreaTestCase 116 * @tc.desc: Define a testcase that accesses a class member 117 * variable. 118 * @tc.type: PERF 119 * @tc.require: Issue Number 120 */ 121 class BenchmarkDemoTest : public benchmark::Fixture { 122 public: 123 void SetUp(const ::benchmark::State &state) 124 { 125 /* @tc.setup: width and height assigned */ 126 phoneWidth_ = 1080; /* 1080 is default width */ 127 phoneHeight_ = 2244; /* 2244 is default height */ 128 } 129 130 void TearDown(const ::benchmark::State &state) 131 { 132 } 133 134 int phoneWidth_; 135 int phoneHeight_; 136 }; 137 138 BENCHMARK_F(BenchmarkDemoTest, CalculatedAreaTestCase)( 139 benchmark::State &st) 140 { 141 long int area = 0; 142 for (auto _ : st) { 143 /* @tc.steps: step1.calculate area */ 144 area = phoneWidth_ * phoneHeight_; 145 } 146 } 147 148 BENCHMARK_REGISTER_F(BenchmarkDemoTest, CalculatedAreaTestCase); 149} 150 151// Run the benchmark 152BENCHMARK_MAIN(); 153``` 154 155详细内容介绍: 156 1571. 添加测试用例文件头注释信息 158 159 ``` 160 /* 161 * Copyright (c) 2021 XXXX Device Co., Ltd. 162 * Licensed under the Apache License, Version 2.0 (the "License"); 163 * you may not use this file except in compliance with the License. 164 * You may obtain a copy of the License at 165 * 166 * http://www.apache.org/licenses/LICENSE-2.0 167 * 168 * Unless required by applicable law or agreed to in writing, software 169 * distributed under the License is distributed on an "AS IS" BASIS, 170 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 171 * See the License for the specific language governing permissions and 172 * limitations under the License. 173 */ 174 ``` 175 1762. 引用测试框架头文件和命名空间 177 178 ``` 179 #include <benchmark/benchmark.h> 180 181 using namespace std; 182 ``` 183 1843. 添加测试套引用的头文件 185 186 ``` 187 #include <string> 188 #include <vector> 189 ``` 190 1914. 测试用例实现 192 193 ``` 194 /** 195 * @tc.name: BenchmarkTestExample 196 * @tc.desc: Testcase for testing 'SimpleExample' function. 197 * @tc.type: PERF 198 * @tc.require: Issue Number 199 */ 200 size_t SimpleExample() 201 { 202 string str = "benchmark test"; 203 return str.size(); 204 } 205 206 static void BenchmarkTestExample(benchmark::State &state) 207 { 208 for (auto _ : state) { 209 /* @tc.steps: step1.call SimpleExample in loop */ 210 SimpleExample(); 211 } 212 } 213 ``` 214 > **注意:** 215 > 测试用例名称不要过长,否则会造成测试报告无法生成。 216 2175. 向benchmark注册用例 218 219 ``` 220 /* Register the function as a benchmark */ 221 BENCHMARK(BenchmarkTestExample); 222 /* Register benchmark and explicitly set the fix iterations */ 223 BENCHMARK(BenchmarkTestExample)->Iterations(1000); 224 ``` 225 226 注册用例同时可以指定用例的执行参数,常用的参数如下表 227 228 | 参数 | 说明 | 229 | -------------------- | ------------------------ | 230 | Iterations | 测试用例迭代次数 | 231 | Repetitions | 测试用例重复执行次数 | 232 | ReportAggregatesOnly | 计算平均值、中值、标准差 | 233 | arg | 向用例函数传入参数 | 234 235 > **说明:** 236 > 237 > Iterations和Repetitions看似重复概念,实则存在差异,此处说明 238 > 239 > **Iterations**:benchmark注册的用例函数实现中均存在for循环,如demo用例BenchmarkTestExample中的for,**Iterations表示这个for循环迭代次数** 240 > 241 > **Repetitions**:表示注册的用例重复执行次数 242 > 243 > 因此若一个用例两个参数均指定,理论上用例中的函数实际执行次数为**Repetitions与Iterations的乘积** 244 245 benchmark还支持其他多种参数,具体介绍和使用参考[benchmark](https://gitee.com/openharmony/third_party_benchmark) 246 247 248## 用例编译 249 250``` 251# Copyright (c) 2021 XXXX Device Co., Ltd. 252# Licensed under the Apache License, Version 2.0 (the "License"); 253# you may not use this file except in compliance with the License. 254# You may obtain a copy of the License at 255# 256# http://www.apache.org/licenses/LICENSE-2.0 257# 258# Unless required by applicable law or agreed to in writing, software 259# distributed under the License is distributed on an "AS IS" BASIS, 260# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 261# See the License for the specific language governing permissions and 262# limitations under the License. 263 264import("//build/test.gni") 265 266module_output_path = "developer_test/calculator" 267 268ohos_benchmarktest("BenchmarkDemoTest") { 269 module_out_path = module_output_path 270 sources = [ "benchmark_demo_test.cpp" ] 271} 272 273group("benchmarktest") { 274 testonly = true 275 deps = [] 276 277 deps += [ 278 # deps file 279 ":BenchmarkDemoTest", 280 ] 281} 282``` 283 284详细内容如下: 285 2861. 添加文件头注释信息 287 288 ``` 289 # Copyright (c) 2021 XXXX Device Co., Ltd. 290 # Licensed under the Apache License, Version 2.0 (the "License"); 291 # you may not use this file except in compliance with the License. 292 # You may obtain a copy of the License at 293 # 294 # http://www.apache.org/licenses/LICENSE-2.0 295 # 296 # Unless required by applicable law or agreed to in writing, software 297 # distributed under the License is distributed on an "AS IS" BASIS, 298 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 299 # See the License for the specific language governing permissions and 300 # limitations under the License. 301 ``` 302 3032. 导入编译模板文件 304 305 ``` 306 import("//build/test.gni") 307 ``` 308 3093. 指定文件输出路径 310 311 ``` 312 module_output_path = "developer_test/calculator" 313 ``` 314 315> **说明:** 此处输出路径为部件/模块名。 316 3174. 指定测试套名称 318 319 ``` 320 ohos_benchmarktest("BenchmarkDemoTest") { 321 module_out_path = module_output_path 322 sources = [ "benchmark_demo_test.cpp" ] 323 } 324 ``` 325 3265. 对目标测试套进行分组 327 328 ``` 329 group("benchmarktest") { 330 testonly = true 331 deps = [] 332 333 deps += [ 334 # deps file 335 ":BenchmarkDemoTest", 336 ] 337 } 338 ``` 339 340 > **说明:** 进行条件分组的目的在于执行用例时可以选择性的执行某一种特定类型的用例。 341 342