README.md
1# Developers Test<a name="EN-US_TOPIC_0000001122338419"></a>
2
3- [Overview](#section7375710115617)
4- [Directory Structure](#section102031353175317)
5- [Constraints](#section87444710110)
6- [Installation](#section1347156474)
7- [Using Test Framework](#section75882026185016)
8- [Test Result and Log](#section414715805819)
9- [Repositories Involved](#section6299103515474)
10
11## Overview<a name="section7375710115617"></a>
12
13This module allows you to develop new test cases for new features, or modify existing test cases for modified features. The developers test framework helps you develop high-quality code.
14
15## Directory Structure<a name="section102031353175317"></a>
16
17```
18developertest/
19├── aw # Static libraries of the test framework
20│ ├── cxx # C++ libraries
21│ └── python # Python libraries
22├── config # Test framework configuration
23│ ├── build_config.xml # Test case build configuration
24│ ├── filter_config.xml # Test case filter configuration
25│ ├── framework_config.xml # Test type configuration
26│ └── user_config.xml # User configuration
27├── examples # Test case examples
28│ ├── calculator # Calculator
29│ └── test # Test resources
30├── src # Source code of the test framework
31│ ├── main # Entry function
32│ └── core # Core code of the test framework
33├── third_party # Adaptation code for third-party components on which the test framework relies
34│ └── lib # Compilation configuration of the static library
35├── BUILD.gn # Compilation entry of the test framework
36├── start.bat # Developers test entry for Windows
37└── start.sh # Developers test entry for Linux
38```
39
40## Constraints<a name="section87444710110"></a>
41
42The test tool environment must meet the following requirements:
43
441. Python version: 3.7.5 or later
452. Paramiko version: 2.7.1 or later
463. Setuptools version: 40.8.0 or later
474. RSA version: 4.0 or later
485. NFS version: V4 or later \(required when device supports connection using the serial port but not the hdc\)
496. pySerial version: 3.3 or later \(required when the device supports connection using the serial port but not the hdc\)
507. OS version: Windows 10 or later; Ubuntu 18.04
51
52## Installation<a name="section1347156474"></a>
53
54The Python environment is required.
55
561. Run the following command to install the Linux extension component Readline:
57
58 ```
59 sudo apt-get install libreadline-dev
60 ```
61
62 ```
63 Reading package lists... Done
64 Building dependency tree
65 Reading state information... Done
66 libreadline-dev is already the newest version (7.0-3).
67 0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.
68 ```
69
702. Run the following command to install the plug-in Setuptools:
71
72 ```
73 pip3 install setuptools
74 ```
75
76 If the installation is successful, the following prompts are displayed:
77
78 ```
79 Requirement already satisfied: setuptools in d:\programs\python37\lib\site-packages (41.2.0)
80 ```
81
823. Run the following command to install the plug-in Paramiko:
83
84 ```
85 pip3 install paramiko
86 ```
87
88 If the installation is successful, the following prompts are displayed:
89
90 ```
91 Installing collected packages: pycparser, cffi, pynacl, bcrypt, cryptography, paramiko
92 Successfully installed bcrypt-3.2.0 cffi-1.14.4 cryptography-3.3.1 paramiko-2.7.2 pycparser-2.20 pynacl-1.4.0
93 ```
94
954. Run the following command to install the Python plug-in RSA:
96
97 ```
98 pip3 install rsa
99 ```
100
101 If the installation is successful, the following prompts are displayed:
102
103 ```
104 Installing collected packages: pyasn1, rsa
105 Successfully installed pyasn1-0.4.8 rsa-4.7
106 ```
107
1085. Run the following command to install the serial port plug-in pySerial for Python on the local PC:
109
110 ```
111 pip3 install pyserial
112 ```
113
114 If the installation is successful, the following prompts are displayed:
115
116 ```
117 Requirement already satisfied: pyserial in d:\programs\python37\lib\site-packages\pyserial-3.4-py3.7.egg (3.4)
118 ```
119
1206. If the device supports test result output only using the serial port, install the NFS server.
121
122 For example, to install haneWIN NFS Server 1.2.50 for Windows, download the installation package from https://www.hanewin.net/nfs-e.htm.
123
124 For Linux, run the following command:
125
126 ```
127 sudo apt install nfs-kernel-server
128 ```
129
130 If the installation is successful, the following prompts are displayed:
131
132 ```
133 Reading package lists... Done
134 Building dependency tree
135 Reading state information... Done
136 nfs-kernel-server is already the newest version (1:1.3.4-2.1ubuntu5.3).
137 0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.
138 ```
139
140
141## Using Test Framework<a name="section75882026185016"></a>
142
143- \(Optional\) Install the XDevice module.
144 1. Go to the XDevice installation directory **test/xdevice**.
145 2. Open the console and run the following command:
146
147 ```
148 python setup.py install
149 ```
150
151 If the installation is successful, the following prompts are displayed:
152
153 ```
154 Installed d:\programs\python37\lib\site-packages\xdevice-0.0.0-py3.7.egg
155 Processing dependencies for xdevice==0.0.0
156 Finished processing dependencies for xdevice==0.0.0
157 ```
158
159
160- Configure the developers test module in the configuration file **developertest/config/user\_config.xml**.
161 1. Modify basic configuration parameters of the test framework.
162
163 \[build\] \# Configure build parameters of the test case.
164
165 ```
166 <build>
167 <example>false</example>
168 <version>false</version>
169 <testcase>true</testcase>
170 ...
171 </build>
172 ```
173
174 > **NOTE:**
175 >**example**: whether to build the test case example. The default value is **false**.
176 >**version**: whether to build the test version. The default value is **false**.
177 >**testcase**: whether to build the test case. The default value is **true**.
178
179 2. For devices that support the Harmony device connector \(hdc\), modify the configuration file as follows:
180
181 \[device\] \# Configure the device information with the **"usb-hdc"** attribute, including the test device IP address and the matched hdc port.
182
183 ```
184 <device type="usb-hdc">
185 <ip>192.168.1.1</ip>
186 <port>9111</port>
187 <sn></sn>
188 </device>
189 ```
190
191 3. For devices that only support the serial port connection, modify the configuration file as follows:
192
193 \[device\] \# Configure the serial port information with the **"ipcamera"** attribute, including the COM port and baud rate.
194
195 ```
196 <device type="com" label="ipcamera">
197 <serial>
198 <com>COM1</com>
199 <type>cmd</type>
200 <baud_rate>115200</baud_rate>
201 <data_bits>8</data_bits>
202 <stop_bits>1</stop_bits>
203 <timeout>1</timeout>
204 </serial>
205 </device>
206 ```
207
208
209- \(Optional\) Modify the configuration file **config/user\_config.xml** of the **developertest** module. If a test case has been compiled, specify the compilation output directory of the test case. In this case, the test platform will not recompile the test case.
210 1. Specify the output directory of the test case, that is, the compilation output directory between the **<test\_cases\>** tags.
211
212 ```
213 <test_cases>
214 <dir>/home/opencode/out/release/tests</dir>
215 </test_cases>
216 ```
217
218 2. For devices that only support serial port connection, specify the NFS directory on the PC \(**host\_dir**\) and the corresponding directory on the board \(**board\_dir**\) between the **<NFS\>** tags. For example:
219
220 ```
221 <NFS>
222 <host_dir>D:\nfs</host_dir>
223 <board_dir>user</board_dir>
224 </NFS>
225 ```
226
227
228- Prepare the test environment. \(Check that the test environment is ready if the tested device only supports serial port connection.\)
229 - The system image and file system have been burnt into a development board and are running properly on the development board. For example, in system mode, device prompt **OHOS\#** is displayed during shell login, indicating that the system is running properly.
230 - The development host has been connected to the serial port of the development board and the network port.
231 - The IP addresses of the development host and development board are in the same network segment and can ping each other.
232 - An empty directory is created on the development host for mounting test cases through the NFS, and the NFS service is started properly.
233
234- Run test suites.
235 - Start the test framework and go to the **test/developertest** directory.
236 1. Run the following command to start the test framework in Windows.
237
238 ```
239 start.bat
240 ```
241
242 2. Run the following command to start the test framework in Linux.
243
244 ```
245 ./strat.sh
246 ```
247
248
249 - Select a device form.
250
251 Configure device forms based on the actual development board, for example, **developertest/config/framework\_config.xml**.
252
253 - Run test commands.
254 1. To query the subsystems, modules, product forms, and test types supported by test cases, run the **show** commands.
255
256 ```
257 usage:
258 show productlist Querying Supported Product Forms
259 show typelist Querying the Supported Test Type
260 show subsystemlist Querying Supported Subsystems
261 show modulelist Querying Supported Modules
262 ```
263
264 2. Run the execution command, where **-t** is mandatory, and **-ss** and **-tm** are optional.
265
266 ```
267 run -t ut -ss test -tm example
268 ```
269
270 3. Specify the parameters that can be used to execute the test suite corresponding to a specific feature or module.
271
272 ```
273 usage: run [-h] [-p PRODUCTFORM] [-t [TESTTYPE [TESTTYPE ...]]]
274 [-ss SUBSYSTEM] [-tm TESTMODULE] [-ts TESTSUIT]
275 [-tc TESTCASE] [-tl TESTLEVEL]
276
277 optional arguments:
278 -h, --help show this help message and exit
279 -p PRODUCTFORM, --productform PRODUCTFORM Specified product form
280 -t [TESTTYPE [TESTTYPE ...]], --testtype [TESTTYPE [TESTTYPE ...]]
281 Specify test type(UT,MST,ST,PERF,ALL)
282 -ss SUBSYSTEM, --subsystem SUBSYSTEM Specify test subsystem
283 -tm TESTMODULE, --testmodule TESTMODULE Specified test module
284 -ts TESTSUIT, --testsuite TESTSUIT Specify test suite
285 -tc TESTCASE, --testcase TESTCASE Specify test case
286 -tl TESTLEVEL, --testlevel TESTLEVEL Specify test level
287 ```
288
289
290
291- See the test framework help if needed.
292 - Run the following command to query commands supported by the test platform:
293
294 ```
295 help
296 ```
297
298
299
300- Exit the self-test platform.
301 - Run the following command to exit the test platform:
302
303 ```
304 quit
305 ```
306
307
308
309## Test Result and Log<a name="section414715805819"></a>
310
311- Test logs and test reports are generated after you execute the test commands.
312- Test result
313 - Reports are displayed on the console. The root path of the test result is as follows:
314
315 ```
316 reports/xxxx-xx-xx-xx-xx-xx
317 ```
318
319 - Test case formatting result
320
321 ```
322 result/
323 ```
324
325 - Test case log
326
327 ```
328 log/plan_log_xxxx-xx-xx-xx-xx-xx.log
329 ```
330
331 - Report summary
332
333 ```
334 summary_report.html
335 ```
336
337 - Test report details
338
339 ```
340 details_report.html
341 ```
342
343
344- Test framework log
345
346 ```
347 reports/platform_log_xxxx-xx-xx-xx-xx-xx.log
348 ```
349
350- Latest test reports
351
352 ```
353 reports/latest
354 ```
355
356
357## Repositories Involved<a name="section6299103515474"></a>
358
359[testing subsystem](https://gitee.com/openharmony/docs/blob/master/en/readme/testing-subsystem.md)
360
361**test\_developertest**
362
363[test\_xdevice](https://gitee.com/openharmony/test_xdevice/blob/master/README.md)
364
README_zh.md
1# 开发者测试组件<a name="ZH-CN_TOPIC_0000001122338419"></a>
2
3- [简介](#section7375710115617)
4- [目录](#section102031353175317)
5- [约束](#section87444710110)
6- [安装](#section1347156474)
7- [使用测试框架](#section75882026185016)
8- [测试结果与日志](#section414715805819)
9- [涉及仓](#section6299103515474)
10
11## 简介<a name="section7375710115617"></a>
12
13开发者基于系统新增特性可以通过开发者自己开发用例保证，对于系统已有特性的修改，也可通过修改项目中原有的测试用例保证，开发者测试旨在帮助开发者在开发阶段就能开发出高质量代码
14
15## 目录<a name="section102031353175317"></a>
16
17```
18developertest/
19├── aw # 测试框架的静态库
20│ ├── cxx # 支持C++
21│ └── python # 支持Python
22├── config # 测试框架配置
23│ ├── build_config.xml # 用例编译配置
24│ ├── filter_config.xml # 用例筛选配置
25│ ├── framework_config.xml # 测试类型配置
26│ └── user_config.xml # 用户使用配置
27├── examples # 测试用例示例
28│ ├── calculator # 计算器示例
29│ └── test # 测试资源示例
30├── src # 测试框架源码
31│ ├── main # 入口函数
32│ └── core # 测试框架核心代码
33├── third_party # 测试框架依赖第三方组件适配
34│ └── lib # 静态库编译配置
35├── BUILD.gn # 测试框架编译入口
36├── start.bat # 开发者测试入口（Windows）
37└── start.sh # 开发者测试入口（Linux）
38```
39
40## 约束<a name="section87444710110"></a>
41
42测试工具环境依赖
43
441. python版本\>=3.7.5
452. paramiko版本\>=2.7.1
463. setuptools版本\>=40.8.0
474. rsa版本\>=4.0
485. NFS版本\>=V4，设备不支持hdc连接，支持串口时使用。
496. pyserial版本\>=3.3，设备不支持hdc连接，支持串口时使用。
507. 运行操作系统：Windows版本\>=Win10，Linux为Ubuntu18.04。
51
52## 安装<a name="section1347156474"></a>
53
54- 依赖python环境：
55 1. 安装Linux扩展组件readline。
56
57 执行如下命令如下：
58
59 ```
60 sudo apt-get install libreadline-dev
61 ```
62
63 安装成功提示
64
65 ```
66 Reading package lists... Done
67 Building dependency tree
68 Reading state information... Done
69 libreadline-dev is already the newest version (7.0-3).
70 0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.
71 ```
72
73 2. 安装setuptools插件，安装命令如下：
74
75 ```
76 pip3 install setuptools
77 ```
78
79 安装成功提示如下：
80
81 ```
82 Requirement already satisfied: setuptools in d:\programs\python37\lib\site-packages (41.2.0)
83 ```
84
85 3. 安装paramiko插件，安装命令如下：
86
87 ```
88 pip3 install paramiko
89 ```
90
91 安装成功提示如下：
92
93 ```
94 Installing collected packages: pycparser, cffi, pynacl, bcrypt, cryptography, paramiko
95 Successfully installed bcrypt-3.2.0 cffi-1.14.4 cryptography-3.3.1 paramiko-2.7.2 pycparser-2.20 pynacl-1.4.0
96 ```
97
98 4. 安装python的rsa插件，安装命令如下：
99
100 ```
101 pip3 install rsa
102 ```
103
104 安装成功截图如下：
105
106 ```
107 Installing collected packages: pyasn1, rsa
108 Successfully installed pyasn1-0.4.8 rsa-4.7
109 ```
110
111 5. 需要本地的python安装串口插件pyserial，安装命令如下：
112
113 ```
114 pip3 install pyserial
115 ```
116
117 安装成功提示如下：
118
119 ```
120 Requirement already satisfied: pyserial in d:\programs\python37\lib\site-packages\pyserial-3.4-py3.7.egg (3.4)
121 ```
122
123 6. 如果设备仅支持串口输出测试结果，则需要安装NFS Server
124
125 windows环境下安装，例如安装haneWIN NFS Server1.2.50，下载地址：https://www.hanewin.net/nfs-e.htm。
126
127 Linux环境下安装，安装命令如下：
128
129 ```
130 sudo apt install nfs-kernel-server
131 ```
132
133 安装成功提示如下：
134
135 ```
136 Reading package lists... Done
137 Building dependency tree
138 Reading state information... Done
139 nfs-kernel-server is already the newest version (1:1.3.4-2.1ubuntu5.3).
140 0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.
141 ```
142
143
144
145## 使用测试框架<a name="section75882026185016"></a>
146
147- 可选，安装xdevice组件。
148 1. 打开xdevice安装目录：test/xdevice。
149 2. 打开控制台窗口，执行如下命令：
150
151 ```
152 python setup.py install
153 ```
154
155 安装成功提示如下：
156
157 ```
158 Installed d:\programs\python37\lib\site-packages\xdevice-0.0.0-py3.7.egg
159 Processing dependencies for xdevice==0.0.0
160 Finished processing dependencies for xdevice==0.0.0
161 ```
162
163
164- developertest组件配置。
165
166 文件：developertest/config/user\_config.xml
167
168 1. 测试框架通用配置。
169
170 \[build\] \# 配置测试用例的编译参数，例如：
171
172 ```
173 <build>
174 <example>false</example>
175 <version>false</version>
176 <testcase>true</testcase>
177 ... ...
178 </build>
179 ```
180
181 > **说明：**
182 >测试用例的编译参数说明如下：
183 >example：是否编译测试用例示例，默认false。
184 >version：是否编译测试版本，默认false。
185 >testcase：是否编译测试用例，默认true。
186
187 2. 支持hdc连接的被测设备。
188
189 \[device\] \# 配置标签为usb-hdc的环境信息，测试设备的IP地址和hdc映射的端口号，例如：
190
191 ```
192 <device type="usb-hdc">
193 <ip>192.168.1.1</ip>
194 <port>9111</port>
195 <sn></sn>
196 </device>
197 ```
198
199 3. 仅支持串口的被测设备。
200
201 \[device\] \# 配置标签为ipcamera的串口信息，COM口和波特率，例如：
202
203 ```
204 <device type="com" label="ipcamera">
205 <serial>
206 <com>COM1</com>
207 <type>cmd</type>
208 <baud_rate>115200</baud_rate>
209 <data_bits>8</data_bits>
210 <stop_bits>1</stop_bits>
211 <timeout>1</timeout>
212 </serial>
213 </device>
214 ```
215
216
217- 修改developertest组件配置。可选，如果测试用例已完成编译，可以直接指定测试用例的编译输出路径，测试平台执行测试用例时即不会重新编译测试用例。
218
219 文件：config/user\_config.xml。
220
221 1. \[test\_cases\] \# 指定测试用例的输出路径，编译输出目录，例如：
222
223 ```
224 <test_cases>
225 <dir>/home/opencode/out/release/tests</dir>
226 </test_cases>
227 ```
228
229 2. \[NFS\] \# 被测设备仅支持串口时配置，指定NFS的映射路径，host\_dir为PC侧的NFS目录，board\_dir为板侧创建的目录，例如：
230
231 ```
232 <NFS>
233 <host_dir>D:\nfs</host_dir>
234 <board_dir>user</board_dir>
235 </NFS>
236 ```
237
238
239- 测试环境准备（当被测设备仅支持串口时，需要检查）。
240 - 系统镜像与文件系统已烧录进开发板，开发板上系统正常运行，在系统模式下，如shell登录时设备提示符OHOS\#。
241 - 开发主机和开发板串口连接正常，网口连接正常。
242 - 开发主机IP与开发板IP处在同一小网网段，相互可以ping通。
243 - 开发主机侧创建空目录用于开发板通过NFS挂载测试用例，并且NFS服务启动正常。
244
245- 运行测试套。
246 - 启动测试框架，打开test/developertest目录。
247 1. Windows环境启动测试框架。
248
249 ```
250 start.bat
251 ```
252
253 2. Linux环境启动测试框架。
254
255 ```
256 ./strat.sh
257 ```
258
259
260 - 设备形态选择。
261
262 根据实际的开发板选择，设备形态配置：developertest/config/framework\_config.xml。
263
264 - 执行测试指令。
265 1. 查询测试用例支持的子系统，模块，产品形态以及测试类型，使用show命令。
266
267 ```
268 usage:
269 show productlist Querying Supported Product Forms
270 show typelist Querying the Supported Test Type
271 show subsystemlist Querying Supported Subsystems
272 show modulelist Querying Supported Modules
273 ```
274
275 2. 执行测试指令示例，其中-t为必选，-ss和-tm为可选字段
276
277 ```
278 run -t ut -ss test -tm example
279 ```
280
281 3. 参数说明：指定参数可以执行特定特性、模块对应的测试套
282
283 ```
284 usage: run [-h] [-p PRODUCTFORM] [-t [TESTTYPE [TESTTYPE ...]]]
285 [-ss SUBSYSTEM] [-tm TESTMODULE] [-ts TESTSUIT]
286 [-tc TESTCASE] [-tl TESTLEVEL]
287
288 optional arguments:
289 -h, --help show this help message and exit
290 -p PRODUCTFORM, --productform PRODUCTFORM Specified product form
291 -t [TESTTYPE [TESTTYPE ...]], --testtype [TESTTYPE [TESTTYPE ...]]
292 Specify test type(UT,MST,ST,PERF,ALL)
293 -ss SUBSYSTEM, --subsystem SUBSYSTEM Specify test subsystem
294 -tm TESTMODULE, --testmodule TESTMODULE Specified test module
295 -ts TESTSUIT, --testsuite TESTSUIT Specify test suite
296 -tc TESTCASE, --testcase TESTCASE Specify test case
297 -tl TESTLEVEL, --testlevel TESTLEVEL Specify test level
298 ```
299
300
301
302- 测试框架帮助。
303 - 帮助指令，用于查询测试平台支持哪些测试指令。
304
305 ```
306 help
307 ```
308
309
310
311- 退出自测试平台。
312 - 退出自测试平台，使用如下命令退出测试平台。
313
314 ```
315 quit
316 ```
317
318
319
320## 测试结果与日志<a name="section414715805819"></a>
321
322- 通过在测试框架中执行测试指令，即可以生成测试日志和测试报告。
323- 测试结果
324 - 测试用例的结果会直接显示在控制台上，执行一次的测试结果根路径如下：
325
326 ```
327 reports/xxxx-xx-xx-xx-xx-xx
328 ```
329
330 - 测试用例格式化结果。
331
332 ```
333 result/
334 ```
335
336 - 测试用例日志。
337
338 ```
339 log/plan_log_xxxx-xx-xx-xx-xx-xx.log
340 ```
341
342 - 测试报告汇总。
343
344 ```
345 summary_report.html
346 ```
347
348 - 测试报告详情。
349
350 ```
351 details_report.html
352 ```
353
354
355- 测试框架日志
356
357 ```
358 reports/platform_log_xxxx-xx-xx-xx-xx-xx.log
359 ```
360
361- 最新测试报告
362
363 ```
364 reports/latest
365 ```
366
367
368## 涉及仓<a name="section6299103515474"></a>
369
370[测试子系统](https://gitee.com/openharmony/docs/blob/master/zh-cn/readme/%E6%B5%8B%E8%AF%95%E5%AD%90%E7%B3%BB%E7%BB%9F.md)
371
372**test\_developertest**
373
374[test\_xdevice](https://gitee.com/openharmony/test_xdevice/blob/master/README_zh.md)
375
