1# Testing<a name="EN-US_TOPIC_0000001082440445"></a> 2 3- [Overview](#section7375710115617) 4- [Directory Structure](#section102031353175317) 5- [Constraints](#section87444710110) 6- [Installation](#section1347156474) 7- [Test Cases](#section125411936102918) 8- [Test Framework Usage](#section75882026185016) 9- [Test Result and Logs](#section414715805819) 10- [Repositories Involved](#section6299103515474) 11 12## Overview<a name="section7375710115617"></a> 13 14The test-driven development mode is used during the development process. You can develop new cases or modify existing cases to test new or enhanced system features. The test helps you develop high-quality code in the development phase. 15 16## Directory Structure<a name="section102031353175317"></a> 17 18``` 19test/ 20├── developertest # Developers test framework 21│ ├── aw # Static library of the test framework 22│ ├── config # Test framework configuration 23│ ├── examples # Test case examples 24│ ├── src # Source code of the test framework 25│ ├── third_party # Adaptation code for third-party modules on which the test framework depends 26│ ├── start.bat # Developers test entry for Windows 27│ ├── start.sh # Developers test entry for Linux 28│ └── BUILD.gn # Build entry of the test framework 29├── xdevice # Basic component of the test framework 30│ ├── config # Framework configuration file 31│ ├── extension # Extension for the basic component 32│ ├── resource # Test resources of the basic component 33│ └── src # Source code of the basic component 34└── xts # X test suite 35``` 36 37## Constraints<a name="section87444710110"></a> 38 39The test tool environment must meet the following requirements: 40 411. Python version: 3.7.5 or later 422. Paramiko version: 2.7.1 or later 433. Setuptools version: 40.8.0 or later 444. RSA version: 4.0 or later 455. NFS version: V4 or later \(required when device supports connection using the serial port but not the hdc\) 466. pySerial version: 3.3 or later \(required when the device supports connection using the serial port but not the hdc\) 477. OS version: Windows 10 or later; Ubuntu 18.04 48 49## Installation<a name="section1347156474"></a> 50 51The Python environment is required. 52 531. Run the following command to install the Linux extension component Readline: 54 55 ``` 56 sudo apt-get install libreadline-dev 57 ``` 58 59 If the installation is successful, the following prompts are displayed: 60 61 ``` 62 Reading package lists... Done 63 Building dependency tree 64 Reading state information... Done 65 libreadline-dev is already the newest version (7.0-3). 66 0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded. 67 ``` 68 692. Run the following command to install the plug-in Setuptools: 70 71 ``` 72 pip3 install setuptools 73 ``` 74 75 If the installation is successful, the following prompts are displayed: 76 77 ``` 78 Requirement already satisfied: setuptools in d:\programs\python37\lib\site-packages (41.2.0) 79 ``` 80 813. Run the following command to install the plug-in Paramiko: 82 83 ``` 84 pip3 install paramiko 85 ``` 86 87 If the installation is successful, the following prompts are displayed: 88 89 ``` 90 Installing collected packages: pycparser, cffi, pynacl, bcrypt, cryptography, paramiko 91 Successfully installed bcrypt-3.2.0 cffi-1.14.4 cryptography-3.3.1 paramiko-2.7.2 pycparser-2.20 pynacl-1.4.0 92 ``` 93 944. Run the following command to install the Python plug-in RSA: 95 96 ``` 97 pip3 install rsa 98 ``` 99 100 If the installation is successful, the following prompts are displayed: 101 102 ``` 103 Installing collected packages: pyasn1, rsa 104 Successfully installed pyasn1-0.4.8 rsa-4.7 105 ``` 106 1075. Run the following command to install the serial port plug-in pySerial for Python on the local PC: 108 109 ``` 110 pip3 install pyserial 111 ``` 112 113 If the installation is successful, the following prompts are displayed: 114 115 ``` 116 Requirement already satisfied: pyserial in d:\programs\python37\lib\site-packages\pyserial-3.4-py3.7.egg (3.4) 117 ``` 118 1196. If the device supports test result output only using the serial port, install the NFS server. 120 121 For example, to install haneWIN NFS Server 1.2.50 for Windows, download the installation package from https://www.hanewin.net/nfs-e.htm. 122 123 For Linux, run the following command: 124 125 ``` 126 sudo apt install nfs-kernel-server 127 ``` 128 129 If the installation is successful, the following prompts are displayed: 130 131 ``` 132 Reading package lists... Done 133 Building dependency tree 134 Reading state information... Done 135 nfs-kernel-server is already the newest version (1:1.3.4-2.1ubuntu5.3). 136 0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded. 137 ``` 138 139 140## Test Cases<a name="section125411936102918"></a> 141 142- Test case specifications 143 - Naming rules 144 145 The source file name of the test case must be consistent with the test suite content. A test suite can contain multiple test cases and has only one test source file that is globally unique. Source files are named in the \[Feature\]\_\[Function\]\_\[Subfunction 1\]\_\[Subfunction 1.1\] format. Subfunctions can be further divided. 146 147 A source file name consists of lowercase letters and underscores \(\_\), and must end with **test**, for example, **developertest/examples/calculator**. 148 149 - Test case coding specifications 150 151 The test cases must comply with the feature code coding specifications. In addition, necessary case description information must be added. For details, see [Test case template](#li2069415903917). 152 153 - Test case compilation and configuration specifications 154 155 The test cases are compiled in GN mode. The configuration must comply with the compilation guide of the open-source project. 156 157 158- <a name="li2069415903917"></a>Test case template 159 160 For details, see the test case **developertest/examples/calculator/test/unittest/common/calculator\_add\_test.cpp**. 161 162- Directories planned for test cases 163 164 ``` 165 subsystem # Subsystem and system module 166 ├── parts # Components 167 │ └── test # Module test 168 │ └── unittest # Unit test 169 │ ├── common # Common test cases 170 │ ├── phone # Test case of the smartphone form 171 │ └── ivi # Test case of the head unit form 172 │ └── liteos-a # Test case of the IP camera form 173 │ └── moduletest # Module test 174 │ ├── common 175 │ ├── phone 176 │ └── ivi 177 │ └── liteos-a 178 └── test # Subsystem test 179 └── resource # Test resources 180 ├── module 181 ├── common 182 ├── phone 183 ├── ivi 184 ├── liteos-a 185 └── systemtest # System test 186 ├── common 187 ├── phone 188 ├── ivi 189 ├── liteos-a 190 ``` 191 192 > **NOTE:** 193 >The **phone**, **ivi**, **liteos-a** test cases are used as examples only for different device forms. For the same feature on different development boards, if the test cases are the same, they are stored in the **common** directory. For the same feature, if the test cases are used to distinguish different device forms and may include kernel differences and chip platform differences, the test cases are distinguished by directory. 194 195- Writing a test case 196 1. Add comments to the test case header file. 197 2. Reference the **gtest** header file and **ext** namespace. 198 3. Add the header file to test. 199 4. Define test suites \(test classes\). 200 5. Implement specific test cases of the test suite, including test case comments and logic implementation. 201 6. Set the test case compilation configuration. 202 203 > **NOTE:** 204 >The following examples are provided for reference: 205 >For devices supporting the serial port only: **developertest/examples/lite/cxx\_demo/test/unittest/common/calc\_subtraction\_test.cpp**. 206 >For devices supporting the hdc: **developertest/examples/calculator/test/unittest/common/calculator\_add\_test.cpp**. 207 >- **SetUp** and **TearDown** are the processing logic before and after each test case in the test suite is executed. 208 >- **SetUpTestCase** and **TearDownTestCase** are the processing logic before and after all cases in the test suite are executed. 209 >- HWTEST usage: This method is applicable only to simple tests \(not depending on **Setup** and **Teardown**\). This method is not applicable to the scenario where multiple test scenarios require the same data configuration. The test cases may affect each other and are not independent. 210 >- Use the **printf** function to print logs. 211 212 213- Writing a test case compilation file 214 - Define test case compilation and building objectives. 215 1. Add comments to the test case compilation header file. 216 2. Import the test case compilation template file. 217 3. Specify the output path of the test case file. 218 4. Configure the directory contained in the test case compilation dependency. 219 5. Specify the file name generated by the test case compilation target. 220 6. Write a specific test case compilation script and add the source files, configurations, and dependencies involved in the compilation. 221 7. Group the target test case files by condition. The group name is fixed to **unittest/moduletest**. 222 223 - If there are multiple test suites, define the common compilation configuration. 224 - Add test cases to the build system. 225 226 > **NOTE:** 227 >The following examples are provided for reference: 228 >- Devices supporting serial port connection only 229 >Test case compilation configuration: **developertest/examples/lite/cxx\_demo/test/unittest/common/BUILD.gn** 230 >Compilation entry configuration: **developertest/examples/lite/BUILD.gn** 231 >- Devices supporting the hdc connection 232 >Test case compilation configuration: **developertest/examples/calculator/test/unittest/common/BUILD.gn** 233 >Compilation entry configuration: **developertest/examples/ohos.build** 234 235 236- Writing a test case resource file 237 1. Create the **resource** directory in the **test** directory of a component or module. 238 2. Create a directory for a device form, for example, **phone**, in the **resource** directory. 239 3. Create a folder named after the module in the device form directory, for example, **testmodule**. 240 4. Create the **ohos\_test.xml** file in the folder named after the module. The file content is in the following format: 241 242 ``` 243 <?xml version="1.0" encoding="UTF-8"?> 244 <configuration ver="2.0"> 245 <target name="DetectorFileTest"> 246 <preparer> 247 <option name="push" value="test.txt -> /data/test/resource" src="res"/> 248 </preparer> 249 </target> 250 </configuration> 251 ``` 252 253 5. Define **resource\_config\_file** in the compilation configuration file of the test case to specify the resource file **ohos\_test.xml**. 254 255 > **NOTE:** 256 >The resource file is used to push the **test.txt** file in the **resource** directory to the **/data/test/resource** directory of the device to test. To do so, run the **hdc push** command. 257 258 6. Configure the **ohos\_test.xml** file, which contains the following tags: 259 260 > **NOTE:** 261 >**target\_name**: name of the test unit, which is usually defined in the **BUILD.gn** file in the test directory 262 >**preparer**: action to take before the test unit is executed 263 >**cleaner**: action to take after the test unit is executed 264 >**src="res"** indicates that test resources are stored in the **resource** directory under the root directory of the subsystem. **src="out"** indicates that test resources are in the **out/release/$**_subsystem name_ directory. 265 266 267- Test case levels 268 - Basic \(level 1\) 269 - Major \(level 2\) 270 - Minor \(level 3\) 271 - Uncommon \(level 4\) 272 273 274## Test Framework Usage<a name="section75882026185016"></a> 275 276- \(Optional\) Install the XDevice module. 277 1. Open the **xdevice** installation directory, for example, **test/xdevice** in Windows. 278 2. Open the console and run the following command: 279 280 ``` 281 python setup.py install 282 ``` 283 284 The following figure is displayed when the installation is complete. 285 286 ``` 287 Installed d:\programs\python37\lib\site-packages\xdevice-0.0.0-py3.7.egg 288 Processing dependencies for xdevice==0.0.0 289 Finished processing dependencies for xdevice==0.0.0 290 ``` 291 292 293- Configure the developers test module. 294 295 Configuration file: **developertest/config/user\_config.xml** 296 297 1. Modify basic configuration parameters. 298 299 \[build\] \# Set build parameters of the test case. 300 301 ``` 302 <build> 303 <example>false</example> 304 <version>false</version> 305 <testcase>true</testcase> 306 ... ... 307 </build> 308 ``` 309 310 > **NOTE:** 311 >**example**: whether to build the test case example. The default value is **false**. 312 >**version**: whether to build the test version. The default value is **false**. 313 >**testcase**: whether to build the test case. The default value is **true**. 314 315 2. For devices that support the Harmony device connector \(hdc\), modify the configuration file as follows: 316 317 \[device\] \# Configure the device information with the **"usb-hdc"** attribute, including the test device IP address and the matched hdc port. 318 319 ``` 320 <device type="usb-hdc"> 321 <ip>192.168.1.1</ip> 322 <port>9111</port> 323 <sn></sn> 324 </device> 325 ``` 326 327 3. For devices that support serial port connection only, modify the configuration file as follows: 328 329 \[board\_info\] \# Configure development board information. 330 331 ``` 332 <board_info> 333 <board_series>hispark</board_series> 334 <board_type>taurus</board_type> 335 <board_product>ipcamera</board_product> 336 <build_command>hb build</build_command> 337 </board_info> 338 ``` 339 340 > **NOTE:** 341 >**board\_series**: development board series. The default value is **hispark**. 342 >**board\_type**: development board type. The default value is **taurus**. 343 >**board\_product**: target product. The default value is **ipcamera**. 344 >**build\_command**: command used for building the test version and test case. The default value is **hb build**. 345 346 \[device\] \# Configure the serial port information with the **"ipcamera"** attribute, including the COM port and baud rate. For example: 347 348 ``` 349 <device type="com" label="ipcamera"> 350 <serial> 351 <com>COM1</com> 352 <type>cmd</type> 353 <baud_rate>115200</baud_rate> 354 <data_bits>8</data_bits> 355 <stop_bits>1</stop_bits> 356 <timeout>1</timeout> 357 </serial> 358 </device> 359 ``` 360 361 362- Modify the configuration of the **developertest** component. 363 364 \(Optional\) If a test case has been compiled, specify the compilation output path of the test case. In this case, the test platform will not recompile the test case. 365 366 Configuration file: **config/user\_config.xml** 367 368 1. Specify the output path of the test case and the compilation output directory. Example: 369 370 ``` 371 <test_cases> 372 <dir>/home/source_code/out/release/tests</dir> 373 </test_cases> 374 ``` 375 376 2. For devices that support serial port connection only, specify the NFS directory for the PC \(**host\_dir**\) and the corresponding directory for the development board \(**board\_dir**\) inside the **<NFS\>** tags. For example: 377 378 ``` 379 <NFS> 380 <host_dir>D:\nfs</host_dir> 381 <board_dir>user</board_dir> 382 </NFS> 383 ``` 384 385 386- Prepare the test environment. Check that the test environment meets the following conditions if the tested device supports only serial ports: 387 - The system image and file system have been burnt into a development board and are running properly on the development board. For example, in system mode, the device prompt **OHOS\#** is displayed during shell login, indicating that the system is running properly. 388 - The development host has been connected to the serial port of the development board and the network port. 389 - The IP addresses of the development host and development board are in the same network segment and can ping each other. 390 - An empty directory is created on the development host for mounting test cases through NFS, and the NFS service is started properly. 391 392- Run test suites. 393 - Start the test framework and go to the **test/developertest** directory. 394 1. Run the following command to start the test framework in Windows. 395 396 ``` 397 start.bat 398 ``` 399 400 2. Run the following command to start the test framework in Linux. 401 402 ``` 403 ./strat.sh 404 ``` 405 406 407 - Select a device form. 408 409 Configure device forms based on the actual development board, for example, **developertest/config/framework\_config.xml**. 410 411 - Run the test command. 412 1. To query the subsystems, modules, product forms, and test types supported by test cases, run the **show** commands. 413 414 ``` 415 usage: 416 show productlist Querying supported product forms 417 show typelist Querying the supported test type 418 show subsystemlist Querying supported subsystems 419 show modulelist Querying supported modules 420 ``` 421 422 2. Run the following command to execute the test \(**-t** is mandatory, and **-ss** and **-tm** are optional\): 423 424 ``` 425 run -t ut -ss test -tm example 426 ``` 427 428 3. Specify the parameters that can be used to execute the test suite specific to a specified feature or module. 429 430 ``` 431 usage: run [-h] [-p PRODUCTFORM] [-t [TESTTYPE [TESTTYPE ...]]] 432 [-ss SUBSYSTEM] [-tm TESTMODULE] [-ts TESTSUIT] 433 [-tc TESTCASE] [-tl TESTLEVEL] 434 435 optional arguments: 436 -h, --help show this help message and exit 437 -p PRODUCTFORM, --productform PRODUCTFORM Specified product form 438 -t [TESTTYPE [TESTTYPE ...]], --testtype [TESTTYPE [TESTTYPE ...]] 439 Specify test type(UT,MST,ST,PERF,ALL) 440 -ss SUBSYSTEM, --subsystem SUBSYSTEM Specify test subsystem 441 -tm TESTMODULE, --testmodule TESTMODULE Specified test module 442 -ts TESTSUIT, --testsuite TESTSUIT Specify test suite 443 -tc TESTCASE, --testcase TESTCASE Specify test case 444 -tl TESTLEVEL, --testlevel TESTLEVEL Specify test level 445 ``` 446 447 448 449- View test framework help if needed. 450 451 Run the following command to query commands supported by the test platform: 452 453 ``` 454 help 455 ``` 456 457 458- Run the following command to exit the self-test platform: 459 460 ``` 461 quit 462 ``` 463 464 465## Test Result and Logs<a name="section414715805819"></a> 466 467- Test logs and test reports are generated after you execute the test commands. 468- Test result 469 - Reports are displayed on the console. The root directory of the test result is as follows: 470 471 ``` 472 reports/xxxx-xx-xx-xx-xx-xx 473 ``` 474 475 - Test case formatting result 476 477 ``` 478 result/ 479 ``` 480 481 - Test case logs 482 483 ``` 484 log/plan_log_xxxx-xx-xx-xx-xx-xx.log 485 ``` 486 487 - Report summary 488 489 ``` 490 summary_report.html 491 ``` 492 493 - Report details 494 495 ``` 496 details_report.html 497 ``` 498 499 500- Test framework logs 501 502 ``` 503 reports/platform_log_xxxx-xx-xx-xx-xx-xx.log 504 ``` 505 506- Latest test reports 507 508 ``` 509 reports/latest 510 ``` 511 512 513## Repositories Involved<a name="section6299103515474"></a> 514 515**Testing subsystem** 516 517test\_developertest 518 519test\_xdevice 520 521test\_xdevice\_extension 522 523
