• Home
Name Date Size #Lines LOC

..--

ability/12-May-2024-11,79910,366

common/12-May-2024-325262

communication/12-May-2024-24,04417,944

distributeddatamgr/12-May-2024-12,0639,944

distributedhardware/12-May-2024-6,3394,628

figures/12-May-2024-

filemanagement/12-May-2024-4,7733,534

multimedia/12-May-2024-2,4101,902

notification/12-May-2024-577435

testtools/12-May-2024-1,018805

BUILD.gnD12-May-2024924 3126

LICENSED12-May-202410.1 KiB177150

OAT.xmlD12-May-20244 KiB6512

README.mdD12-May-202429.2 KiB511405

README_ZH.mdD12-May-202429.3 KiB556430

build.shD12-May-20244.8 KiB129104

bundle.jsonD12-May-2024684 3130

runtest.shD12-May-20245.9 KiB188155

test_packages.gniD12-May-20243.1 KiB8773

README.md

1# XTS<a name="EN-US_TOPIC_0000001126156429"></a>
2
3-   [Introduction](#section465982318513)
4-   [System Types](#section125090457443)
5-   [Directory Structure](#section161941989596)
6-   [Constraints](#section119744591305)
7-   [Usage Guidelines](#section137768191623)
8-   [Test Case Development Guidelines](#section3695134065513)
9    -   [C-based Test Case Development and Compilation \(for the Mini System\)](#section198193336544)
10    -   [C-based Test Case Execution \(for the Mini System\)](#section13820233175418)
11    -   [C++-based Test Case Development and Compilation \(for Standard and Small Systems\)](#section3822123311540)
12    -   [C++-based Test Case Execution \(for Standard and Small Systems\)](#section128222336544)
13
14
15## Introduction<a name="section465982318513"></a>
16
17The X test suite \(XTS\) subsystem contains a set of OpenHarmony certification test suites, including the currently supported distributed compatibility test suite \(DCTS\).
18
19This subsystem contains the DCTS and  **tools**  software package.
20
21-   The  **dcts**  directory stores the source code and configuration files of DCTS test cases. The DCTS helps device vendors detect the distributed scenario incompatibility as early as possible and ensures that the software is compatible to OpenHarmony during the entire development process.
22-   The  **tools**  software package stores the test case development framework related to  **dcts**.
23
24## System Types<a name="section125090457443"></a>
25
26OpenHarmony supports the following system types:
27
28-   Mini system
29
30    A mini system runs on the devices whose memory is greater than or equal to 128 KiB and that are equipped with MCU processors such as ARM Cortex-M and 32-bit RISC-V. This system provides multiple lightweight network protocols and graphics frameworks, and a wide range of read/write components for the IoT bus. Typical products include connection modules, sensors, and wearables for smart home.
31
32-   Small system
33
34    A small system runs on the devices whose memory is greater than or equal to 1 MiB and that are equipped with application processors such as ARM Cortex-A. This system provides higher security capabilities, standard graphics frameworks, and video encoding and decoding capabilities. Typical products include smart home IP cameras, electronic cat eyes, and routers, and event data recorders \(EDRs\) for smart travel.
35
36-   Standard system
37
38    A standard system runs on the devices whose memory is greater than or equal to 128 MiB and that are equipped with application processors such as ARM Cortex-A. This system provides a complete application framework supporting the enhanced interaction, 3D GPU, hardware composer, diverse components, and rich animations. This system applies to high-end refrigerator displays.
39
40
41## Directory Structure<a name="section161941989596"></a>
42
43```
44/test/xts
45├── dcts                # Test code
46│   └── subsystem       # Source code of subsystem test cases for the standard system
47│   └── subsystem_lite  # Source code of subsystems test cases for mini and small systems
48│   └── common          # Source code of Test cases rely on shared memory for mini and small systems
49│   └── BUILD.gn        # Build configuration of test cases for the standard system
50│   └── build_lite
51│       └── BUILD.gn    # Build configuration of test cases for mini and small systems
52└── tools               # Test tool code
53```
54
55## Constraints<a name="section119744591305"></a>
56
57Test cases for the mini system must be developed based on C, the small system must be developed based on C++, and the standard system must be developed based on C++ or js.
581. Currently the DCTS supports the standard system.
592. Screenless devices do not support PIN authentication networking and do not support DCTS test。
60
61## Usage Guidelines<a name="section137768191623"></a>
62
63**Table  1**  Test case levels
64
65<a name="table12856113719456"></a>
66<table><thead align="left"><tr id="row88561737114517"><th class="cellrowborder" valign="top" width="33.333333333333336%" id="mcps1.2.4.1.1"><p id="p54711456452"><a name="p54711456452"></a><a name="p54711456452"></a>Level</p>
67</th>
68<th class="cellrowborder" valign="top" width="33.283328332833285%" id="mcps1.2.4.1.2"><p id="p114834544518"><a name="p114834544518"></a><a name="p114834544518"></a>Definition</p>
69</th>
70<th class="cellrowborder" valign="top" width="33.383338333833386%" id="mcps1.2.4.1.3"><p id="p348154574518"><a name="p348154574518"></a><a name="p348154574518"></a>Scope</p>
71</th>
72</tr>
73</thead>
74<tbody><tr id="row78561137134514"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p17481345124512"><a name="p17481345124512"></a><a name="p17481345124512"></a>Level0</p>
75</td>
76<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p1548945204514"><a name="p1548945204514"></a><a name="p1548945204514"></a>Smoke</p>
77</td>
78<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p174824517456"><a name="p174824517456"></a><a name="p174824517456"></a>Verifies basic functionalities of key features and basic DFX attributes with the most common input. The pass result indicates that the features are runnable.</p>
79</td>
80</tr>
81<tr id="row28563372451"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p448445144512"><a name="p448445144512"></a><a name="p448445144512"></a>Level1</p>
82</td>
83<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p44813452452"><a name="p44813452452"></a><a name="p44813452452"></a>Basic</p>
84</td>
85<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p648184584514"><a name="p648184584514"></a><a name="p648184584514"></a>Verifies basic functionalities of key features and basic DFX attributes with common input. The pass result indicates that the features are testable.</p>
86</td>
87</tr>
88<tr id="row12856137204514"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p164834520453"><a name="p164834520453"></a><a name="p164834520453"></a>Level2</p>
89</td>
90<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p148245184511"><a name="p148245184511"></a><a name="p148245184511"></a>Major</p>
91</td>
92<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p348945124510"><a name="p348945124510"></a><a name="p348945124510"></a>Verifies basic functionalities of key features and basic DFX attributes with common input and errors. The pass result indicates that the features are functional and ready for beta testing.</p>
93</td>
94</tr>
95<tr id="row48571137204514"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p7481545184513"><a name="p7481545184513"></a><a name="p7481545184513"></a>Level3</p>
96</td>
97<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p1748184514455"><a name="p1748184514455"></a><a name="p1748184514455"></a>Regular</p>
98</td>
99<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p8481245104519"><a name="p8481245104519"></a><a name="p8481245104519"></a>Verifies functionalities of all key features, and all DFX attributes with common and uncommon input combinations or normal and abnormal preset conditions.</p>
100</td>
101</tr>
102<tr id="row14857133712451"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p1248144515456"><a name="p1248144515456"></a><a name="p1248144515456"></a>Level4</p>
103</td>
104<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p94804514519"><a name="p94804514519"></a><a name="p94804514519"></a>Rare</p>
105</td>
106<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p1248445164516"><a name="p1248445164516"></a><a name="p1248445164516"></a>Verifies functionalities of key features under extremely abnormal presets and uncommon input combinations.</p>
107</td>
108</tr>
109</tbody>
110</table>
111
112**Table  2**  Test case granularities
113
114<a name="table1690733521014"></a>
115<table><thead align="left"><tr id="row9907143516105"><th class="cellrowborder" valign="top" width="19.591959195919593%" id="mcps1.2.4.1.1"><p id="p1141312416102"><a name="p1141312416102"></a><a name="p1141312416102"></a>Test Scale</p>
116</th>
117<th class="cellrowborder" valign="top" width="35.17351735173517%" id="mcps1.2.4.1.2"><p id="p34131416109"><a name="p34131416109"></a><a name="p34131416109"></a>Test Objects</p>
118</th>
119<th class="cellrowborder" valign="top" width="45.23452345234523%" id="mcps1.2.4.1.3"><p id="p12413141171010"><a name="p12413141171010"></a><a name="p12413141171010"></a>Test Environment</p>
120</th>
121</tr>
122</thead>
123<tbody><tr id="row2907335181017"><td class="cellrowborder" valign="top" width="19.591959195919593%" headers="mcps1.2.4.1.1 "><p id="p6414134117109"><a name="p6414134117109"></a><a name="p6414134117109"></a>LargeTest</p>
124</td>
125<td class="cellrowborder" valign="top" width="35.17351735173517%" headers="mcps1.2.4.1.2 "><p id="p20414104113101"><a name="p20414104113101"></a><a name="p20414104113101"></a>Service functionalities, all-scenario features, and mechanical power environment (MPE) and scenario-level DFX</p>
126</td>
127<td class="cellrowborder" valign="top" width="45.23452345234523%" headers="mcps1.2.4.1.3 "><p id="p5414841121013"><a name="p5414841121013"></a><a name="p5414841121013"></a>Devices close to real devices</p>
128</td>
129</tr>
130<tr id="row10907635131015"><td class="cellrowborder" valign="top" width="19.591959195919593%" headers="mcps1.2.4.1.1 "><p id="p19414134117104"><a name="p19414134117104"></a><a name="p19414134117104"></a>MediumTest</p>
131</td>
132<td class="cellrowborder" valign="top" width="35.17351735173517%" headers="mcps1.2.4.1.2 "><p id="p114141241141018"><a name="p114141241141018"></a><a name="p114141241141018"></a>Modules, subsystem functionalities after module integration, and DFX</p>
133</td>
134<td class="cellrowborder" valign="top" width="45.23452345234523%" headers="mcps1.2.4.1.3 "><p id="p14414941111015"><a name="p14414941111015"></a><a name="p14414941111015"></a>Single device that is actually used. You can perform message simulation, but do not mock functions.</p>
135</td>
136</tr>
137<tr id="row1990713353100"><td class="cellrowborder" valign="top" width="19.591959195919593%" headers="mcps1.2.4.1.1 "><p id="p641414110109"><a name="p641414110109"></a><a name="p641414110109"></a>SmallTest</p>
138</td>
139<td class="cellrowborder" valign="top" width="35.17351735173517%" headers="mcps1.2.4.1.2 "><p id="p54141541181018"><a name="p54141541181018"></a><a name="p54141541181018"></a>Modules, classes, and functions</p>
140</td>
141<td class="cellrowborder" valign="top" width="45.23452345234523%" headers="mcps1.2.4.1.3 "><p id="p1841494131013"><a name="p1841494131013"></a><a name="p1841494131013"></a>Local PC. Use a large number of mocks to replace dependencies with other modules.</p>
142</td>
143</tr>
144</tbody>
145</table>
146
147**Table  3**  Test types
148
149<a name="table18811230124619"></a>
150<table><thead align="left"><tr id="row48163019466"><th class="cellrowborder" valign="top" width="19.040000000000003%" id="mcps1.2.3.1.1"><p id="p1791784418465"><a name="p1791784418465"></a><a name="p1791784418465"></a>Type</p>
151</th>
152<th class="cellrowborder" valign="top" width="80.96%" id="mcps1.2.3.1.2"><p id="p2917164444614"><a name="p2917164444614"></a><a name="p2917164444614"></a>Definition</p>
153</th>
154</tr>
155</thead>
156<tbody><tr id="row682630104615"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p1091784414469"><a name="p1091784414469"></a><a name="p1091784414469"></a>Function</p>
157</td>
158<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p16917344104618"><a name="p16917344104618"></a><a name="p16917344104618"></a>Tests the correctness of both service and platform functionalities provided by the tested object for end users or developers.</p>
159</td>
160</tr>
161<tr id="row1821930154611"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p591714474612"><a name="p591714474612"></a><a name="p591714474612"></a>Performance</p>
162</td>
163<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p15917154410463"><a name="p15917154410463"></a><a name="p15917154410463"></a>Tests the processing capability of the tested object under specific preset conditions and load models. The processing capability is measured by the service volume that can be processed in a unit time, for example, call per second, frame per second, or event processing volume per second.</p>
164</td>
165</tr>
166<tr id="row13821030104616"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p691711440467"><a name="p691711440467"></a><a name="p691711440467"></a>Power</p>
167</td>
168<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p159171544104616"><a name="p159171544104616"></a><a name="p159171544104616"></a>Tests the power consumption of the tested object in a certain period of time under specific preset conditions and load models.</p>
169</td>
170</tr>
171<tr id="row6821330114618"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p13917164454612"><a name="p13917164454612"></a><a name="p13917164454612"></a>Reliability</p>
172</td>
173<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p11917194416465"><a name="p11917194416465"></a><a name="p11917194416465"></a>Tests the service performance of the tested object under common and uncommon input conditions, or specified service volume pressure and long-term continuous running pressure. The test covers stability, pressure handling, fault injection, and Monkey test times.</p>
174</td>
175</tr>
176<tr id="row11821930184612"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p1691774474618"><a name="p1691774474618"></a><a name="p1691774474618"></a>Security</p>
177</td>
178<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><a name="ul112421652121513"></a><a name="ul112421652121513"></a><ul id="ul112421652121513"><li>Tests the capability of defending against security threats, including but not limited to unauthorized access, use, disclosure, damage, modification, and destruction, to ensure information confidentiality, integrity, and availability.</li><li>Tests the privacy protection capability to ensure that the collection, use, retention, disclosure, and disposal of users' private data comply with laws and regulations.</li><li>Tests the compliance with various security specifications, such as security design, security requirements, and security certification of the Ministry of Industry and Information Technology (MIIT).</li></ul>
179</td>
180</tr>
181<tr id="row16825307467"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p129188444462"><a name="p129188444462"></a><a name="p129188444462"></a>Global</p>
182</td>
183<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p179186444465"><a name="p179186444465"></a><a name="p179186444465"></a>Tests the internationalized data and localization capabilities of the tested object, including multi-language display, various input/output habits, time formats, and regional features, such as currency, time, and culture taboos.</p>
184</td>
185</tr>
186<tr id="row08211308464"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p191814447465"><a name="p191814447465"></a><a name="p191814447465"></a>Compatibility</p>
187</td>
188<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><a name="ul186961827111612"></a><a name="ul186961827111612"></a><ul id="ul186961827111612"><li>Tests backward compatibility of an application with its own data, the forward and backward compatibility with the system, and the compatibility with different user data, such as audio file content of the player and smart SMS messages.</li><li>Tests system backward compatibility with its own data and the compatibility of common applications in the ecosystem.</li><li>Tests software compatibility with related hardware.</li></ul>
189</td>
190</tr>
191<tr id="row1782730124618"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p13918144134614"><a name="p13918144134614"></a><a name="p13918144134614"></a>User</p>
192</td>
193<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p1291844494611"><a name="p1291844494611"></a><a name="p1291844494611"></a>Tests user experience of the object in real user scenarios. All conclusions and comments should come from the users, which are all subjective evaluation in this case.</p>
194</td>
195</tr>
196<tr id="row58243024617"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p1291884474614"><a name="p1291884474614"></a><a name="p1291884474614"></a>Standard</p>
197</td>
198<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p0918124424614"><a name="p0918124424614"></a><a name="p0918124424614"></a>Tests the compliance with industry and company-specific standards, protocols, and specifications. The standards here do not include any security standards that should be classified into the security test.</p>
199</td>
200</tr>
201<tr id="row382830124619"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p119181044164619"><a name="p119181044164619"></a><a name="p119181044164619"></a>Safety</p>
202</td>
203<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p1291818443468"><a name="p1291818443468"></a><a name="p1291818443468"></a>Tests the safety property of the tested object to avoid possible hazards to personal safety, health, and the object itself.</p>
204</td>
205</tr>
206<tr id="row1083153014465"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p39187441469"><a name="p39187441469"></a><a name="p39187441469"></a>Resilience</p>
207</td>
208<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p891815444462"><a name="p891815444462"></a><a name="p891815444462"></a>Tests the resilience property of the tested object to ensure that it can withstand and maintain the defined running status (including downgrading) when being attacked, and recover from and adapt defense to the attacks to approach mission assurance.</p>
209</td>
210</tr>
211</tbody>
212</table>
213
214## Test Case Development Guidelines<a name="section3695134065513"></a>
215
216You should select the appropriate programming language and your target test framework to develop test cases.
217
218**Table  4**  Test frameworks and test case languages for different systems
219
220<a name="table4418343171415"></a>
221<table><thead align="left"><tr id="row34183435145"><th class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.2.4.1.1"><p id="p941874311148"><a name="p941874311148"></a><a name="p941874311148"></a>System</p>
222</th>
223<th class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.2.4.1.2"><p id="p1841804341413"><a name="p1841804341413"></a><a name="p1841804341413"></a>Test Framework</p>
224</th>
225<th class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.2.4.1.3"><p id="p2418104311148"><a name="p2418104311148"></a><a name="p2418104311148"></a>Language</p>
226</th>
227</tr>
228</thead>
229<tbody><tr id="row8419164319148"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.1 "><p id="p7419194312143"><a name="p7419194312143"></a><a name="p7419194312143"></a>Mini</p>
230</td>
231<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.2 "><p id="p10419124312145"><a name="p10419124312145"></a><a name="p10419124312145"></a>HCTest</p>
232</td>
233<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.3 "><p id="p11419643191410"><a name="p11419643191410"></a><a name="p11419643191410"></a>C</p>
234</td>
235</tr>
236<tr id="row141915438147"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.1 "><p id="p441911436141"><a name="p441911436141"></a><a name="p441911436141"></a>Small</p>
237</td>
238<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.2 "><p id="p541916432142"><a name="p541916432142"></a><a name="p541916432142"></a>HCPPTest</p>
239</td>
240<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.3 "><p id="p54191643131416"><a name="p54191643131416"></a><a name="p54191643131416"></a>C++</p>
241</td>
242</tr>
243<tr id="row4419134341417"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.1 "><p id="p341964313143"><a name="p341964313143"></a><a name="p341964313143"></a>Standard</p>
244</td>
245<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.2 "><p id="p10419443171416"><a name="p10419443171416"></a><a name="p10419443171416"></a>HJSUnit and HCPPTest</p>
246</td>
247<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.3 "><p id="p9419143181414"><a name="p9419143181414"></a><a name="p9419143181414"></a>JavaScript and C++</p>
248</td>
249</tr>
250</tbody>
251</table>
252
253### C-based Test Case Development and Compilation \(for the Mini System\)<a name="section198193336544"></a>
254
255**Developing test cases for the mini system**
256
257The HCTest framework is used to support test cases developed with the C language. HCTest is enhanced and adapted based on the open-source test framework Unity.
258
2591.  Access the  **test/xts/dcts**  repository where the test cases will be stored.
260
261    ```
262    ├── dcts
263    │ └──subsystem_lite
264    │ │ └── module_hal
265    │ │ │ └── BUILD.gn
266    │ │ │ └── src
267    │ └──build_lite
268    │ │ └── BUILD.gn
269    ```
270
2712.  Write the test case in the  **src**  directory.
272
273    1 Import the test framework header file.
274
275    ```
276    #include "hctest.h"
277    ```
278
279    2. Use the  **LITE\_TEST\_SUIT**  macro to define names of the subsystem, module, and test suite.
280
281    ```
282    /**
283    * @brief  Registers a test suite named IntTestSuite.
284    * @param  test Subsystem name
285    * @param  example Module name
286    * @param  IntTestSuite Test suite name
287    */
288    LITE_TEST_SUIT(test, example, IntTestSuite);
289    ```
290
291    3. Define Setup and TearDown.
292
293    Format: Test suite name+Setup, Test suite name+TearDown.
294
295    The Setup and TearDown functions must exist, but function bodies can be empty.
296
297    4. Use the  **LITE\_TEST\_CASE**  macro to write the test case.
298
299    Three parameters are involved: test suite name, test case name, and test case properties \(including type, granularity, and level\).
300
301    ```
302    LITE_TEST_CASE(IntTestSuite, TestCase001, Function | MediumTest | Level1)
303    {
304      // Do something
305    };
306    ```
307
308    5. Use the  **RUN\_TEST\_SUITE**  macro to register the test suite.
309
310    ```
311    RUN_TEST_SUITE(IntTestSuite);
312    ```
313
3143.  Create the configuration file \(**BUILD.gn**\) of the test module.
315
316    Create a  **BUILD.gn**  \(example\) build file in each test module directory. Specify the name of the built static library and its dependent header file and library in the build file. The format is as follows:
317
318    ```
319    import("//test/xts/tools/lite/build/suite_lite.gni")
320    hctest_suite("DctsDemoTest") {
321        suite_name = "dcts"
322        sources = [
323            "src/test_demo.c",
324        ]
325        include_dirs = [ ]
326        cflags = [ "-Wno-error" ]
327    }
328    ```
329
3304.  Add build options to the  **BUILD.gn**  file in the  **dcts**  directory.
331
332    You need to add the test module to the  **test/xts/dcts/build\_lite/BUILD.gn**  script in the  **dcts**  directory.
333
334    ```
335    lite_component("dcts") {
336        ...
337        if(board_name == "liteos_m") {
338            features += [
339                ...
340                "//xts/dcts/subsystem_lite/module_hal:DctsDemoTest"
341            ]
342        }
343    }
344    ```
345
3465.  Run build commands.
347
348    Test suites are built along with version build. The DCTS is built together with the debug version.
349
350    >![](figures/icon-note.gif) **NOTE:**
351    >The DCTS build middleware is a static library, which will be linked to the image.
352
353
354### C-based Test Case Execution \(for the Mini System\)<a name="section13820233175418"></a>
355
356**Executing test cases for the mini system**
357
358Burn the image into the development board.
359
360**Executing the test**
361
3621.  Use a serial port tool to log in to the development board and save information about the serial port.
3632.  Restart the device and view serial port logs.
364
365**Analyzing the test result**
366
367View the serial port logs, whose format is as follows:
368
369The log for each test suite starts with  **Start to run test suite:**  and ends with  **xx Tests xx Failures xx Ignored**.
370
371### C++-based Test Case Development and Compilation \(for Standard and Small Systems\)<a name="section3822123311540"></a>
372
373**Developing test cases for small-system devices**  \(For examples of the standard system, go to the  **global/i18n\_standard directory**.\)
374
375The HCPPTest framework is enhanced and adapted based on the open-source framework Googletest.
376
3771.  Access the  **test/xts/dcts**  repository where the test cases will be stored.
378
379    ```
380    ├── dcts
381    │ └──subsystem_lite
382    │ │ └── module_posix
383    │ │ │ └── BUILD.gn
384    │ │ │ └── src
385    │ └──build_lite
386    │ │ └── BUILD.gn
387    ```
388
3892.  Write the test case in the  **src**  directory.
390
391    1. Import the test framework header file.
392
393    The following statement includes  **gtest.h**.
394
395    ```
396    #include "gtest/gtest.h"
397    ```
398
399    2. Define Setup and TearDown.
400
401    ```
402    using namespace std;
403    using namespace testing::ext;
404    class TestSuite: public testing::Test {
405    protected:
406    // Preset action of the test suite, which is executed before the first test case
407    static void SetUpTestCase(void){
408    }
409    // Test suite cleanup action, which is executed after the last test case
410    static void TearDownTestCase(void){
411    }
412    // Preset action of the test case
413    virtual void SetUp()
414    {
415    }
416    // Cleanup action of the test case
417    virtual void TearDown()
418    {
419    }
420    };
421    ```
422
423    3. Use the  **HWTEST**  or  **HWTEST\_F**  macro to write the test case.
424
425    **HWTEST**: definition of common test cases, including the test suite name, test case name, and case annotation.
426
427    **HWTEST\_F**: definition of SetUp and TearDown test cases, including the test suite name, test case name, and case annotation.
428
429    Three parameters are involved: test suite name, test case name, and test case properties \(including type, granularity, and level\).
430
431    ```
432    HWTEST_F(TestSuite, TestCase_0001, Function | MediumTest | Level1) {
433    // Do something
434    }
435    ```
436
4373.  Create a configuration file \(**BUILD.gn**\) of the test module.
438
439    Create a  **BUILD.gn**  build file in each test module directory. Specify the name of the built static library and its dependent header file and library in the build file. Each test module is independently built into a  **.bin**  executable file, which can be directly pushed to the development board for testing.
440
441    Example:
442
443    ```
444    import("//test/xts/tools/lite/build/suite_lite.gni")
445    hcpptest_suite("DctsDemoTest") {
446        suite_name = "dcts"
447        sources = [
448            "src/TestDemo.cpp"
449        ]
450
451        include_dirs = [
452            "src",
453            ...
454        ]
455        deps = [
456            ...
457        ]
458        cflags = [ "-Wno-error" ]
459    }
460    ```
461
4624.  Add build options to the  **BUILD.gn**  file in the  **dcts**  directory.
463
464    Add the test module to the  **test/xts/dcts/build\_lite/BUILD.gn**  script in the  **dcts**  directory.
465
466    ```
467     lite_component("dcts") {
468    ...
469    else if(board_name == "liteos_a") {
470            features += [
471                ...
472                "//xts/dcts/subsystem_lite/module_posix:DctsDemoTest"
473            ]
474        }
475    }
476    ```
477
4785.  Run build commands.
479
480    Test suites are built along with the version build. The DCTS is built together with the debug version.
481
482    >![](figures/icon-note.gif) **NOTE:**
483    >The DCTS for the small system is independently built to an executable file \(.bin\) and archived in the  **suites\\dcts**  directory of the build result.
484
485
486### C++-based Test Case Execution \(for Standard and Small Systems\)<a name="section128222336544"></a>
487
488**Executing test cases for the small system**
489
490Currently, test cases are shared by the NFS and mounted to the development board for execution.
491
492**Setting up the environment**
493
4941.  Use a network cable or wireless network to connect the development board to your PC.
4952.  Configure the IP address, subnet mask, and gateway for the development board. Ensure that the development board and the PC are in the same network segment.
4963.  Install and register the NFS server on the PC and start the NFS service.
4974.  Run the  **mount**  command for the development board to ensure that the development board can access NFS shared files on the PC.
498
499    Format:  **mount** _NFS server IP address_**:/**_NFS shared directory_ **/**_development board directory_ **nfs**
500
501    Example:
502
503    ```
504    mount 192.168.1.10:/nfs /nfs nfs
505    ```
506
507
508**Executing test cases**
509
510Execute  **DctsDemoTest.bin**  to trigger test case execution, and analyze serial port logs generated after the execution is complete.
511

README_ZH.md

1# XTS子系统<a name="ZH-CN_TOPIC_0000001117409671"></a>
2
3-   [简介](#section465982318513)
4-   [系统类型](#section125090457443)
5-   [目录](#section161941989596)
6-   [约束](#section119744591305)
7-   [使用说明](#section137768191623)
8-   [用例开发指导](#section3695134065513)
9    -   [C语言用例开发编译指导(适用于轻量系统产品用例开发)](#section198193336544)
10    -   [C语言用例执行指导(适用于轻量系统产品用例开发)](#section13820233175418)
11    -   [C++语言用例开发编译指导(适用于小型系统、标准系统用例开发)](#section3822123311540)
12    -   [C++语言用例执行指导(适用于小型系统、标准系统用例开发)](#section128222336544)
13
14-   [相关仓](#section1371113476307)
15
16## 简介<a name="section465982318513"></a>
17
18XTS子系统是OpenHarmony生态认证测试套件的集合,当前包括dcts(distributed compatibility test suite)分布式兼容性测试套件。
19
20XTS子系统当前包括dcts与tools软件包:
21
22-   dcts,存放dcts相关测试用例源码与配置文件,其目的是帮助终端设备厂商尽早发现在分布式场景下与OpenHarmony的不兼容性,确保软件在整个开发过程中满足OpenHarmony的兼容性要求。
23-   tools,存放dcts相关测试用例开发框架。
24
25## 系统类型<a name="section125090457443"></a>
26
27OpenHarmony支持如下几种系统类型:
28
29-   轻量系统(mini system)
30
31    面向MCU类处理器例如Arm Cortex-M、RISC-V 32位的设备,硬件资源极其有限,支持的设备最小内存为128KiB,可以提供多种轻量级网络协议,轻量级的图形框架,以及丰富的IOT总线读写部件等。可支撑的产品如智能家居领域的连接类模组、传感器设备、穿戴类设备等。
32
33-   小型系统(small system)
34
35    面向应用处理器例如Arm Cortex-A的设备,支持的设备最小内存为1MiB,可以提供更高的安全能力、标准的图形框架、视频编解码的多媒体能力。可支撑的产品如智能家居领域的IP Camera、电子猫眼、路由器以及智慧出行领域的行车记录仪等。
36
37-   标准系统(standard system)
38
39    面向应用处理器例如Arm Cortex-A的设备,支持的设备最小内存为128MiB,可以提供增强的交互能力、3D GPU以及硬件合成能力、更多控件以及动画效果更丰富的图形能力、完整的应用框架。可支撑的产品如高端的冰箱显示屏。
40
41
42## 目录<a name="section161941989596"></a>
43
44```
45/test/xts
46├── dcts                # 测试代码存放目录
47│   └── subsystem       # 标准系统子系统测试用例源码存放目录
48│   └── subsystem_lite  # 轻量系统、小型系统子系统测试用例源码存放目录
49│   └── common          # 测试用例依赖共享内存源码存放目录
50│   └── BUILD.gn        # 标准系统测试用例编译配置
51│   └── build_lite      # 轻量系统、小型系统测试用例编译配置存放目录
52│       └── BUILD.gn    # 轻量系统、小型系统测试用例编译配置
53└── tools               # 测试工具代码存放目录
54```
55
56## 约束<a name="section119744591305"></a>
57
58轻量系统用例开发语言是C,小型系统用例开发语言是C++,标准系统用例开发语言是C++/js。
591. 当前DCTS支持标准系统。
602. 无屏设备不支持PIN码认证组网,暂不支持DCTS测试。
61
62## 使用说明<a name="section137768191623"></a>
63
64**表 1**  用例级别说明
65
66<a name="table12856113719456"></a>
67<table><thead align="left"><tr id="row88561737114517"><th class="cellrowborder" valign="top" width="33.333333333333336%" id="mcps1.2.4.1.1"><p id="p54711456452"><a name="p54711456452"></a><a name="p54711456452"></a>级别名称</p>
68</th>
69<th class="cellrowborder" valign="top" width="33.283328332833285%" id="mcps1.2.4.1.2"><p id="p114834544518"><a name="p114834544518"></a><a name="p114834544518"></a>基本定义</p>
70</th>
71<th class="cellrowborder" valign="top" width="33.383338333833386%" id="mcps1.2.4.1.3"><p id="p348154574518"><a name="p348154574518"></a><a name="p348154574518"></a>测试范围</p>
72</th>
73</tr>
74</thead>
75<tbody><tr id="row78561137134514"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p17481345124512"><a name="p17481345124512"></a><a name="p17481345124512"></a>Level0</p>
76</td>
77<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p1548945204514"><a name="p1548945204514"></a><a name="p1548945204514"></a>冒烟</p>
78</td>
79<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p174824517456"><a name="p174824517456"></a><a name="p174824517456"></a>验证关键功能点基本功能/最基本DFX属性在最常见输入下的表现,通过表示功能基本可运行。</p>
80</td>
81</tr>
82<tr id="row28563372451"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p448445144512"><a name="p448445144512"></a><a name="p448445144512"></a>Level1</p>
83</td>
84<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p44813452452"><a name="p44813452452"></a><a name="p44813452452"></a>基本</p>
85</td>
86<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p648184584514"><a name="p648184584514"></a><a name="p648184584514"></a>验证各功能点基本功能/基本DFX属性在常见输入下的表现,通过表示功能基本可测试。</p>
87</td>
88</tr>
89<tr id="row12856137204514"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p164834520453"><a name="p164834520453"></a><a name="p164834520453"></a>Level2</p>
90</td>
91<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p148245184511"><a name="p148245184511"></a><a name="p148245184511"></a>重要</p>
92</td>
93<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p348945124510"><a name="p348945124510"></a><a name="p348945124510"></a>验证各功能点的基本功能/基本DFX属性在常规输入/常见异常情况下的表现,通过表示功能基本正常可用,可开展Beta。</p>
94</td>
95</tr>
96<tr id="row48571137204514"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p7481545184513"><a name="p7481545184513"></a><a name="p7481545184513"></a>Level3</p>
97</td>
98<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p1748184514455"><a name="p1748184514455"></a><a name="p1748184514455"></a>一般</p>
99</td>
100<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p8481245104519"><a name="p8481245104519"></a><a name="p8481245104519"></a>验证各功能点的全部功能/全部DFX属性在各种常规/非常规输入组合下,或各种正常/异常预置条件组合下的表现。</p>
101</td>
102</tr>
103<tr id="row14857133712451"><td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.2.4.1.1 "><p id="p1248144515456"><a name="p1248144515456"></a><a name="p1248144515456"></a>Level4</p>
104</td>
105<td class="cellrowborder" valign="top" width="33.283328332833285%" headers="mcps1.2.4.1.2 "><p id="p94804514519"><a name="p94804514519"></a><a name="p94804514519"></a>生僻</p>
106</td>
107<td class="cellrowborder" valign="top" width="33.383338333833386%" headers="mcps1.2.4.1.3 "><p id="p1248445164516"><a name="p1248445164516"></a><a name="p1248445164516"></a>验证关键功能点在极端异常预置条件下、用户难以触及的异常输入组合下的表现。</p>
108</td>
109</tr>
110</tbody>
111</table>
112
113**表 2**  用例粒度说明
114
115<a name="table1690733521014"></a>
116<table><thead align="left"><tr id="row9907143516105"><th class="cellrowborder" valign="top" width="19.591959195919593%" id="mcps1.2.4.1.1"><p id="p1141312416102"><a name="p1141312416102"></a><a name="p1141312416102"></a>用例规模</p>
117</th>
118<th class="cellrowborder" valign="top" width="35.17351735173517%" id="mcps1.2.4.1.2"><p id="p34131416109"><a name="p34131416109"></a><a name="p34131416109"></a>被测试对象</p>
119</th>
120<th class="cellrowborder" valign="top" width="45.23452345234523%" id="mcps1.2.4.1.3"><p id="p12413141171010"><a name="p12413141171010"></a><a name="p12413141171010"></a>测试环境</p>
121</th>
122</tr>
123</thead>
124<tbody><tr id="row2907335181017"><td class="cellrowborder" valign="top" width="19.591959195919593%" headers="mcps1.2.4.1.1 "><p id="p6414134117109"><a name="p6414134117109"></a><a name="p6414134117109"></a>LargeTest</p>
125</td>
126<td class="cellrowborder" valign="top" width="35.17351735173517%" headers="mcps1.2.4.1.2 "><p id="p20414104113101"><a name="p20414104113101"></a><a name="p20414104113101"></a>业务功能/全场景特性/整机及场景级DFX</p>
127</td>
128<td class="cellrowborder" valign="top" width="45.23452345234523%" headers="mcps1.2.4.1.3 "><p id="p5414841121013"><a name="p5414841121013"></a><a name="p5414841121013"></a>尽量使用贴近真实的环境设备</p>
129</td>
130</tr>
131<tr id="row10907635131015"><td class="cellrowborder" valign="top" width="19.591959195919593%" headers="mcps1.2.4.1.1 "><p id="p19414134117104"><a name="p19414134117104"></a><a name="p19414134117104"></a>MediumTest</p>
132</td>
133<td class="cellrowborder" valign="top" width="35.17351735173517%" headers="mcps1.2.4.1.2 "><p id="p114141241141018"><a name="p114141241141018"></a><a name="p114141241141018"></a>模块/子系统集成至设备后的功能/DFX</p>
134</td>
135<td class="cellrowborder" valign="top" width="45.23452345234523%" headers="mcps1.2.4.1.3 "><p id="p14414941111015"><a name="p14414941111015"></a><a name="p14414941111015"></a>使用真实的单设备进行验证,可进行消息模拟,尽量不对函数进行MOCK</p>
136</td>
137</tr>
138<tr id="row1990713353100"><td class="cellrowborder" valign="top" width="19.591959195919593%" headers="mcps1.2.4.1.1 "><p id="p641414110109"><a name="p641414110109"></a><a name="p641414110109"></a>SmallTest</p>
139</td>
140<td class="cellrowborder" valign="top" width="35.17351735173517%" headers="mcps1.2.4.1.2 "><p id="p54141541181018"><a name="p54141541181018"></a><a name="p54141541181018"></a>模块/类/函数</p>
141</td>
142<td class="cellrowborder" valign="top" width="45.23452345234523%" headers="mcps1.2.4.1.3 "><p id="p1841494131013"><a name="p1841494131013"></a><a name="p1841494131013"></a>在开发者个人环境进行测试,尽量不依赖其他模块,存在大量的MOCK</p>
143</td>
144</tr>
145</tbody>
146</table>
147
148**表 3**  测试类型说明
149
150<a name="table18811230124619"></a>
151<table><thead align="left"><tr id="row48163019466"><th class="cellrowborder" valign="top" width="19.040000000000003%" id="mcps1.2.3.1.1"><p id="p1791784418465"><a name="p1791784418465"></a><a name="p1791784418465"></a>测试类型名称</p>
152</th>
153<th class="cellrowborder" valign="top" width="80.96%" id="mcps1.2.3.1.2"><p id="p2917164444614"><a name="p2917164444614"></a><a name="p2917164444614"></a>测试类型定义</p>
154</th>
155</tr>
156</thead>
157<tbody><tr id="row682630104615"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p1091784414469"><a name="p1091784414469"></a><a name="p1091784414469"></a>Function</p>
158</td>
159<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p16917344104618"><a name="p16917344104618"></a><a name="p16917344104618"></a>验证被测对象提供给用户的业务功能实现正确性的测试项,这里的“用户”可以是终端用户或开发者,功能包括业务功能及平台功能</p>
160</td>
161</tr>
162<tr id="row1821930154611"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p591714474612"><a name="p591714474612"></a><a name="p591714474612"></a>Performance</p>
163</td>
164<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p15917154410463"><a name="p15917154410463"></a><a name="p15917154410463"></a>验证被测对象在特定预置条件/负载模型下的处理能力的测试项,“处理能力”一般以单位时间内可处理的业务量来衡量,如呼叫/秒,帧率/秒,事件处理量/秒等</p>
165</td>
166</tr>
167<tr id="row13821030104616"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p691711440467"><a name="p691711440467"></a><a name="p691711440467"></a>Power</p>
168</td>
169<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p159171544104616"><a name="p159171544104616"></a><a name="p159171544104616"></a>验证被测对象在特定预置条件/负载模型下在一定时间内能源消耗量的测试项</p>
170</td>
171</tr>
172<tr id="row6821330114618"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p13917164454612"><a name="p13917164454612"></a><a name="p13917164454612"></a>Reliability</p>
173</td>
174<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p11917194416465"><a name="p11917194416465"></a><a name="p11917194416465"></a>验证被测对象在正常/异常输入情况下,或业务量压力和长时间连续运行压力情况下业务表现的测试项,含稳定性、压力、故障注入、Monkey测试项</p>
175</td>
176</tr>
177<tr id="row11821930184612"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p1691774474618"><a name="p1691774474618"></a><a name="p1691774474618"></a>Security</p>
178</td>
179<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p1791784410463"><a name="p1791784410463"></a><a name="p1791784410463"></a>验证系统对恶意威胁的防护能力,威胁包括但不限于未授权访问、使用、泄露、破坏、修改、毁灭,以保障信息的机密性、完整性和可用性; 验证系统对用户隐私的保护能力,保障用户的隐私数据被收集、使用、保有、披露和处置符合法律规范,保障用户的隐私权; 验证对各类安全规范的遵从情况,如安全设计规范、安全红线、工信部安全认证规范等,保障安全相关法律法规的合规。</p>
180</td>
181</tr>
182<tr id="row16825307467"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p129188444462"><a name="p129188444462"></a><a name="p129188444462"></a>Global</p>
183</td>
184<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p179186444465"><a name="p179186444465"></a><a name="p179186444465"></a>验证被测对象在是否具有国际化数据支持和本地化能力的测试项,包括语言显示、输入/输出习惯、时间显示、区域特性如货币时间禁忌等等</p>
185</td>
186</tr>
187<tr id="row08211308464"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p191814447465"><a name="p191814447465"></a><a name="p191814447465"></a>Compatibility</p>
188</td>
189<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p99181344144613"><a name="p99181344144613"></a><a name="p99181344144613"></a>当被测对象为应用时,包括被测对象对于自身数据的后向兼容性、对于系统的前后向兼容性、对于不同用户数据(如播放器之音频文件格式/智能短信之用户短信内容)的兼容性测试项; 当被测对象为系统时,包括被测系统对于系统自身数据的后向兼容性、以及对于生态中常用应用的兼容性测试项;当被测对象为软件时,包括被测系统对于相关的硬件的兼容性;</p>
190</td>
191</tr>
192<tr id="row1782730124618"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p13918144134614"><a name="p13918144134614"></a><a name="p13918144134614"></a>User</p>
193</td>
194<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p1291844494611"><a name="p1291844494611"></a><a name="p1291844494611"></a>验证被测对象在真实用户场景下的用户体验感受的测试项,注意此种情况下没有客观的“正确”与“失败”,所有的结论及评价都应该来自于用户</p>
195</td>
196</tr>
197<tr id="row58243024617"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p1291884474614"><a name="p1291884474614"></a><a name="p1291884474614"></a>Standard</p>
198</td>
199<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p0918124424614"><a name="p0918124424614"></a><a name="p0918124424614"></a>验证被测对象对于行业及公司内标准/协议/规范的遵从情况的测试项,注意此处的“标准”不包含任何安全标准,针对安全标准的测试项划归为“安全测试”类型</p>
200</td>
201</tr>
202<tr id="row382830124619"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p119181044164619"><a name="p119181044164619"></a><a name="p119181044164619"></a>Safety</p>
203</td>
204<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p1291818443468"><a name="p1291818443468"></a><a name="p1291818443468"></a>验证被测对象的Safety属性,避免产品可能对人身安全、健康以及产品本身带来的危害。</p>
205</td>
206</tr>
207<tr id="row1083153014465"><td class="cellrowborder" valign="top" width="19.040000000000003%" headers="mcps1.2.3.1.1 "><p id="p39187441469"><a name="p39187441469"></a><a name="p39187441469"></a>Resilience</p>
208</td>
209<td class="cellrowborder" valign="top" width="80.96%" headers="mcps1.2.3.1.2 "><p id="p891815444462"><a name="p891815444462"></a><a name="p891815444462"></a>验证被测对象的韧性属性,确保系统受攻击时承受并保持在有定义的运行状态(包括降级)、恢复并适应攻击以保障Mission达成。</p>
210</td>
211</tr>
212</tbody>
213</table>
214
215## 用例开发指导<a name="section3695134065513"></a>
216
217根据测试系统选择测试框架和对应测试用例语言。
218
219**表 4**  系统和测试框架、开发语言对应关系
220
221<a name="table4418343171415"></a>
222<table><thead align="left"><tr id="row34183435145"><th class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.2.4.1.1"><p id="p941874311148"><a name="p941874311148"></a><a name="p941874311148"></a>系统</p>
223</th>
224<th class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.2.4.1.2"><p id="p1841804341413"><a name="p1841804341413"></a><a name="p1841804341413"></a>测试框架</p>
225</th>
226<th class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.2.4.1.3"><p id="p2418104311148"><a name="p2418104311148"></a><a name="p2418104311148"></a>语言</p>
227</th>
228</tr>
229</thead>
230<tbody><tr id="row8419164319148"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.1 "><p id="p7419194312143"><a name="p7419194312143"></a><a name="p7419194312143"></a>轻量系统</p>
231</td>
232<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.2 "><p id="p10419124312145"><a name="p10419124312145"></a><a name="p10419124312145"></a>hctest</p>
233</td>
234<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.3 "><p id="p11419643191410"><a name="p11419643191410"></a><a name="p11419643191410"></a>c</p>
235</td>
236</tr>
237<tr id="row141915438147"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.1 "><p id="p441911436141"><a name="p441911436141"></a><a name="p441911436141"></a>小型系统</p>
238</td>
239<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.2 "><p id="p541916432142"><a name="p541916432142"></a><a name="p541916432142"></a>hcpptest</p>
240</td>
241<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.3 "><p id="p54191643131416"><a name="p54191643131416"></a><a name="p54191643131416"></a>c++</p>
242</td>
243</tr>
244<tr id="row4419134341417"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.1 "><p id="p341964313143"><a name="p341964313143"></a><a name="p341964313143"></a>标准系统</p>
245</td>
246<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.2 "><p id="p10419443171416"><a name="p10419443171416"></a><a name="p10419443171416"></a>HJSUnit、hcpptest</p>
247</td>
248<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.2.4.1.3 "><p id="p9419143181414"><a name="p9419143181414"></a><a name="p9419143181414"></a>js、c++</p>
249</td>
250</tr>
251</tbody>
252</table>
253
254### C语言用例开发编译指导(适用于轻量系统产品用例开发)<a name="section198193336544"></a>
255
256**示例:轻量系统测试用例开发**
257
258当前使用的测试框架是hctest,hctest测试框架支持使用C语言编写测试用例,是在开源测试框架unity的基础上进行增强和适配。
259
2601.  用例目录规范:测试用例存储到test/xts/dcts仓中
261
262    ```
263    ├── dcts
264    │ └──subsystem_lite
265    │ │ └── module_hal
266    │ │ │ └── BUILD.gn
267    │ │ │ └── src
268    │ └──build_lite
269    │ │ └── BUILD.gn
270    ```
271
2722.  src目录下用例编写样例。
273
274    1.引用测试框架
275
276    ```
277    #include "hctest.h"
278    ```
279
280    2. 使用宏定义LITE\_TEST\_SUIT定义子系统、模块、测试套件名称
281
282    ```
283    /**
284    * @brief  register a test suit named "IntTestSuite"
285    * @param  test subsystem name
286    * @param  example module name
287    * @param  IntTestSuite test suit name
288    */
289    LITE_TEST_SUIT(test, example, IntTestSuite);
290    ```
291
292    3. 定义Setup与TearDown
293
294    命名方式:测试套件名称+Setup,测试套件名称+TearDown。
295
296    Setup与TearDown必须存在,可以为空函数。
297
298    4. 使用宏定义LITE\_TEST\_CASE写测试用例
299
300    包括三个参数:测试套件名称,测试用例名称,用例属性(测试类型、用例粒度、用例级别)。
301
302    ```
303    LITE_TEST_CASE(IntTestSuite, TestCase001, Function | MediumTest | Level1)
304    {
305      //do something
306    };
307    ```
308
309    5. 使用宏定义 RUN\_TEST\_SUITE注册测试套件
310
311    ```
312    RUN_TEST_SUITE(IntTestSuite);
313    ```
314
3153.  测试模块的配置文件(BUILD.gn)样例:
316
317    在每个测试模块目录下新建BUILD.gn编译文件,用于指定编译后静态库的名称、依赖的头文件、依赖的库等;具体写法如下:
318
319    ```
320    import("//test/xts/tools/lite/build/suite_lite.gni")
321    hctest_suite("DctsDemoTest") {
322        suite_name = "dcts"
323        sources = [
324            "src/test_demo.c",
325        ]
326        include_dirs = [ ]
327        cflags = [ "-Wno-error" ]
328    }
329    ```
330
3314.  dcts下BUILD.gn增加编译选项。
332
333    需要将测试模块加入到dcts目录下的编译脚本中,编译脚本路径:test/xts/dcts/build\_lite/BUILD.gn334
335    ```
336    lite_component("dcts") {
337        ...
338        if(board_name == "liteos_m") {
339            features += [
340                ...
341                "//xts/dcts/subsystem_lite/module_hal:DctsDemoTest"
342            ]
343        }
344    }
345    ```
346
3475.  测试套件编译命令。
348
349    随版本编译,debug版本编译时会同步编译dcts测试套件
350
351    >![](figures/icon-note.gif) **说明:**
352    >dcts测试套件编译中间件为静态库,最终链接到版本镜像中 。
353
354
355### C语言用例执行指导(适用于轻量系统产品用例开发)<a name="section13820233175418"></a>
356
357**示例:轻量系统测试用例执行**
358
359将版本镜像烧录进开发板。
360
361**测试步骤**
362
3631.  使用串口工具登录开发板,并保存串口打印信息。
3642.  重启设备,查看串口日志。
365
366**测试结果分析指导**
367
368基于串口打印日志进行分析;
369
370每个测试套件执行以Start to run test suite开始,以xx Tests xx Failures xx Ignored结束。
371
372### C++语言用例开发编译指导(适用于小型系统、标准系统用例开发)<a name="section3822123311540"></a>
373
374**示例:小型系统测试用例开发**(标准系统参考具体样例目录:global/i18n\_standard)
375
376当前使用的测试框架是hcpptest,hcpptest测试框架是在开源的googletest测试框架的基础上进行的增强和适配。
377
3781.  规范用例目录:测试用例存储到test/xts/dcts仓中。
379
380    ```
381    ├── dcts
382    │ └──subsystem_lite
383    │ │ └── module_posix
384    │ │ │ └── BUILD.gn
385    │ │ │ └── src
386    │ └──build_lite
387    │ │ └── BUILD.gn
388    ```
389
3902.  测试模块src下用例编写样例:
391
392    1. 引用测试框架:
393
394    需要引用gtest.h  如:\#include "gtest/gtest.h"
395
396    ```
397    #include "gtest/gtest.h"
398    ```
399
400    2. 定义Setup与TearDown
401
402    ```
403    using namespace std;
404    using namespace testing::ext;
405    class TestSuite: public testing::Test {
406    protected:
407    // Preset action of the test suite, which is executed before the first test case
408    static void SetUpTestCase(void){
409    }
410    // Test suite cleanup action, which is executed after the last test case
411    static void TearDownTestCase(void){
412    }
413    // Preset action of the test case
414    virtual void SetUp()
415    {
416    }
417    // Cleanup action of the test case
418    virtual void TearDown()
419    {
420    }
421    };
422    ```
423
424    3. 使用宏定义HWTEST或HWTEST\_F写测试用例
425
426    普通测试用例的定义:HWTEST(测试套名称, 测试用例名称, 用例标注)。
427
428    包含SetUp和TearDown的测试用例的定义 :HWTEST\_F(测试套名称, 测试用例名称,用例标注)。
429
430    宏定义包括三个参数:测试套件名称,测试用例名称,用例属性(测试类型、用例粒度、用例级别)。
431
432    ```
433    HWTEST_F(TestSuite, TestCase_0001, Function | MediumTest | Level1) {
434    // do something
435    }
436    ```
437
4383.  测试模块下用例配置文件(BUILD.gn)样例:
439
440    每个测试模块目录下新建BUILD.gn编译文件,用于指定编译后可执行文件的名称、依赖的头文件、依赖的库等;具体写法如下。每个测试模块将独立编译成.bin可执行文件, 该文件可直接push到单板上进行测试。
441
442    举例:
443
444    ```
445    import("//test/xts/tools/lite/build/suite_lite.gni")
446    hcpptest_suite("DctsDemoTest") {
447        suite_name = "dcts"
448        sources = [
449            "src/TestDemo.cpp"
450        ]
451
452        include_dirs = [
453            "src",
454            ...
455        ]
456        deps = [
457            ...
458        ]
459        cflags = [ "-Wno-error" ]
460    }
461
462    ```
463
4644.  dcts目录下增加编译选项(BUILD.gn)样例:
465
466    将测试模块加入到dcts目录下的编译脚本中,编译脚本为:test/xts/dcts/build\_lite/BUILD.gn467
468    ```
469     lite_component("dcts") {
470    ...
471    else if(board_name == "liteos_a") {
472            features += [
473                ...
474                "//xts/dcts/subsystem_lite/module_posix:DctsDemoTest"
475            ]
476        }
477    }
478    ```
479
4805.  测试套件编译命令。
481
482    随版本编译,debug版本编译时会同步编译dcts测试套件
483
484    >![](figures/icon-note.gif) **说明:**
485    >小型系统dcts独立编译成可执行文件(bin格式), 在编译产物的suites\\dcts目录下归档。
486
487
488### C++语言用例执行指导(适用于小型系统、标准系统用例开发)<a name="section128222336544"></a>
489
490**示例:小型系统测试用例执行**
491
492目前的用例执行采用nfs共享的方式,mount到单板去执行。
493
494**环境搭建**
495
4961.  使用网线或无线网络将开发板与PC进行连接。
4972.  开发板配置IP、子网掩码、网关,确保开发板与PC处于同一个网段。
4983.  PC安装nfs服务器并完成注册,启动nfs服务。
4994.  开发板配置mount命令,确保开发板可以访问PC端的nfs共享文件。
500
501    格式:mount \[nfs服务器IP\]:\[/nfs共享目录\] \[/开发板目录\] nfs
502
503    举例:
504
505    ```
506    mount 192.168.1.10:/nfs /nfs nfs
507    ```
508
509
510**用例执行**
511
512测试套件执行 DctsDemoTest.bin 触发用例执行,基于串口打印日志进行分析。
513
514### 全量编译指导(适用于标准系统)<a name="section128222336544"></a>
515
516全量编译test/xts/dcts目录下执行编译命令: ./build.sh suite=dcts system_size=standard
517
518测试用例输出目录:out/release/suites/dcts/testcases
519
520测试框架&用例整体输出目录:out/release/suites/dcts(编译用例时会同步编译测试套执行框架)
521
522### 全量用例执行指导(适用于小型系统、标准系统)<a name="section128222336544"></a>
523
524搭建测试环境 Windows工作台下安装python3.7及以上版本,确保工作台和测试设备正常连接。
525
526测试执行目录(对应编译生成的out/release/suites/dcts目录)
527
528  ```
529├── testcase                       # 测试套文件存放目录
530│ └──xxx.hap                       # 测试套可执行hap文件
531│ └──xxx.json                      # 测试套对应执行配置文件
532├── tools                          # 测试框架工具目录
533├── run.bat                        # window平台测试套启动执行文件
534├── report                         # 测试报告生成目录
535
536  ```
537用例执行
538
539在Windows工作台上,找到从Linux服务器上拷贝下来的测试套件用例目录,在Windows命令窗口进入对应目录,直接执行dcts\run.bat540
541界面启动后,输入用例执行指令。
542
543全量执行:run dcts
544
545模块执行(具体模块可以查看\dcts\testcases):run –l DctsSamgrTest
546
547查看测试报告。 进入dcts\reports\,获取当前的执行记录,打开“summary_report.html”可以获取到测试报告。
548
549
550## 相关仓<a name="section1371113476307"></a>
551
552xts\_dcts
553
554xts\_tools
555
556