1# `toranj-ncp` test framework 2 3`toranj-ncp` is a test framework for OpenThread enabling testing of the combined behavior of OpenThread (in NCP mode), spinel interface, and `wpantund` driver on linux. 4 5`toranj` features: 6 7- It is developed in Python. 8- It can be used to simulate multiple nodes forming complex network topologies. 9- It allows testing of network interactions between many nodes (IPv6 traffic exchanges). 10- `toranj` in NCP mode runs `wpantund` natively with OpenThread in NCP mode on simulation platform (real-time). 11 12## Setup 13 14`toranj-ncp` requires `wpantund` to be installed. 15 16- Please follow [`wpantund` installation guide](https://github.com/openthread/wpantund/blob/master/INSTALL.md#wpantund-installation-guide). Note that `toranj` expects `wpantund` installed from latest master branch. 17- Alternative way to install `wpantund` is to use the same commands from git workflow [Simulation](https://github.com/openthread/openthread/blob/4b55284bd20f99a88e8e2c617ba358a0a5547f5d/.github/workflows/simulation.yml#L336-L341) for build target `toranj-test-framework`. 18 19To run all tests, `start` script can be used. This script will build OpenThread with proper configuration options and starts running all test. 20 21```bash 22 cd tests/toranj/ # from OpenThread repo root 23 TORANJ_CLI=0 ./start.sh 24``` 25 26The `toranj-ncp` tests are included in `tests/toranj/ncp` folder. Each test-case has its own script following naming model `test-nnn-name.py` (e.g., `test-001-get-set.py`). 27 28To run a specific test 29 30```bash 31 sudo python ncp/test-001-get-set.py 32``` 33 34## `toranj` Components 35 36`wpan` python module defines the `toranj` test components. 37 38### `wpan.Node()` Class 39 40`wpan.Node()` class creates a Thread node instance. It creates a sub-process to run `wpantund` and OpenThread, and provides methods to control the node. 41 42```python 43>>> import wpan 44>>> node1 = wpan.Node() 45>>> node1 46Node (index=1, interface_name=wpan1) 47>>> node2 = wpan.Node() 48>>> node2 49Node (index=2, interface_name=wpan2) 50``` 51 52Note: You may need to run as `sudo` to allow `wpantund` to create tunnel interface (i.e., use `sudo python`). 53 54### `wpan.Node` methods providing `wpanctl` commands 55 56`wpan.Node()` provides methods matching all `wpanctl` commands. 57 58- Get the value of a `wpantund` property, set the value, or add/remove value to/from a list based property: 59 60```python 61 node.get(prop_name) 62 node.set(prop_name, value, binary_data=False) 63 node.add(prop_name, value, binary_data=False) 64 node.remove(prop_name, value, binary_data=False) 65``` 66 67Example: 68 69```python 70>>> node.get(wpan.WPAN_NAME) 71'"test-network"' 72>>> node.set(wpan.WPAN_NAME, 'my-network') 73>>> node.get(wpan.WPAN_NAME) 74'"my-network"' 75>>> node.set(wpan.WPAN_KEY, '65F2C35C7B543BAC1F3E26BB9F866C1D', binary_data=True) 76>>> node.get(wpan.WPAN_KEY) 77'[65F2C35C7B543BAC1F3E26BB9F866C1D]' 78``` 79 80- Common network operations: 81 82```python 83 node.reset() # Reset the NCP 84 node.status() # Get current status 85 node.leave() # Leave the current network, clear all persistent data 86 87 # Form a network in given channel (if none given use a random one) 88 node.form(name, channel=None) 89 90 # Join a network with given info. 91 # node_type can be JOIN_TYPE_ROUTER, JOIN_TYPE_END_DEVICE, JOIN_TYPE_SLEEPY_END_DEVICE 92 node.join(name, channel=None, node_type=None, panid=None, xpanid=None) 93``` 94 95Example: 96 97```python 98>>> result = node.status() 99>>> print result 100wpan1 => [ 101 "NCP:State" => "offline" 102 "Daemon:Enabled" => true 103 "NCP:Version" => "OPENTHREAD/20170716-00460-ga438cef0c-dirty; NONE; Feb 12 2018 11:47:01" 104 "Daemon:Version" => "0.08.00d (0.07.01-191-g63265f7; Feb 2 2018 18:05:47)" 105 "Config:NCP:DriverName" => "spinel" 106 "NCP:HardwareAddress" => [18B4300000000001] 107] 108>>> 109>>> node.form("test-network", channel=12) 110'Forming WPAN "test-network" as node type "router"\nSuccessfully formed!' 111>>> 112>>> print node.status() 113wpan1 => [ 114 "NCP:State" => "associated" 115 "Daemon:Enabled" => true 116 "NCP:Version" => "OPENTHREAD/20170716-00460-ga438cef0c-dirty; NONE; Feb 12 2018 11:47:01" 117 "Daemon:Version" => "0.08.00d (0.07.01-191-g63265f7; Feb 2 2018 18:05:47)" 118 "Config:NCP:DriverName" => "spinel" 119 "NCP:HardwareAddress" => [18B4300000000001] 120 "NCP:Channel" => 12 121 "Network:NodeType" => "leader" 122 "Network:Name" => "test-network" 123 "Network:XPANID" => 0xA438CF5973FD86B2 124 "Network:PANID" => 0x9D81 125 "IPv6:MeshLocalAddress" => "fda4:38cf:5973:0:b899:3436:15c6:941d" 126 "IPv6:MeshLocalPrefix" => "fda4:38cf:5973::/64" 127] 128``` 129 130- Scan: 131 132```python 133 node.active_scan(channel=None) 134 node.energy_scan(channel=None) 135 node.discover_scan(channel=None, joiner_only=False, enable_filtering=False, panid_filter=None) 136 node.permit_join(duration_sec=None, port=None, udp=True, tcp=True) 137``` 138 139- On-mesh prefixes and off-mesh routes: 140 141```python 142 node.config_gateway(prefix, default_route=False) 143 node.add_route(route_prefix, prefix_len_in_bytes=None, priority=None) 144 node.remove_route(route_prefix, prefix_len_in_bytes=None, priority=None) 145``` 146 147A direct `wpanctl` command can be issued using `node.wpanctl(command)` with a given `command` string. 148 149`wpan` module provides variables for different `wpantund` properties. Some commonly used are: 150 151- Network/NCP properties: WPAN_STATE, WPAN_NAME, WPAN_PANID, WPAN_XPANID, WPAN_KEY, WPAN_CHANNEL, WPAN_HW_ADDRESS, WPAN_EXT_ADDRESS, WPAN_POLL_INTERVAL, WPAN_NODE_TYPE, WPAN_ROLE, WPAN_PARTITION_ID 152 153- IPv6 Addresses: WPAN_IP6_LINK_LOCAL_ADDRESS, WPAN_IP6_MESH_LOCAL_ADDRESS, WPAN_IP6_MESH_LOCAL_PREFIX, WPAN_IP6_ALL_ADDRESSES, WPAN_IP6_MULTICAST_ADDRESSES 154 155- Thread Properties: WPAN_THREAD_RLOC16, WPAN_THREAD_ROUTER_ID, WPAN_THREAD_LEADER_ADDRESS, WPAN_THREAD_LEADER_ROUTER_ID, WPAN_THREAD_LEADER_WEIGHT, WPAN_THREAD_LEADER_NETWORK_DATA, 156 157 WPAN_THREAD_CHILD_TABLE, WPAN_THREAD_CHILD_TABLE_ADDRESSES, WPAN_THREAD_NEIGHBOR_TABLE, 158 WPAN_THREAD_ROUTER_TABLE 159 160Method `join_node()` can be used by a node to join another node: 161 162```python 163 # `node1` joining `node2`'s network as a router 164 node1.join_node(node2, node_type=JOIN_TYPE_ROUTER) 165``` 166 167Method `allowlist_node()` can be used to add a given node to the allowlist of the device and enables allowlisting: 168 169```python 170 # `node2` is added to the allowlist of `node1` and allowlisting is enabled on `node1` 171 node1.allowlist_node(node2) 172``` 173 174#### Example (simple 3-node topology) 175 176Script below shows how to create a 3-node network topology with `node1` and `node2` being routers, and `node3` an end-device connected to `node2`: 177 178```python 179>>> import wpan 180>>> node1 = wpan.Node() 181>>> node2 = wpan.Node() 182>>> node3 = wpan.Node() 183 184>>> wpan.Node.init_all_nodes() 185 186>>> node1.form("test-PAN") 187'Forming WPAN "test-PAN" as node type "router"\nSuccessfully formed!' 188 189>>> node1.allowlist_node(node2) 190>>> node2.allowlist_node(node1) 191 192>>> node2.join_node(node1, wpan.JOIN_TYPE_ROUTER) 193'Joining "test-PAN" C474513CB487778D as node type "router"\nSuccessfully Joined!' 194 195>>> node3.allowlist_node(node2) 196>>> node2.allowlist_node(node3) 197 198>>> node3.join_node(node2, wpan.JOIN_TYPE_END_DEVICE) 199'Joining "test-PAN" C474513CB487778D as node type "end-device"\nSuccessfully Joined!' 200 201>>> print node2.get(wpan.WPAN_THREAD_NEIGHBOR_TABLE) 202[ 203 "EAC1672C3EAB30A4, RLOC16:9401, LQIn:3, AveRssi:-20, LastRssi:-20, Age:30, LinkFC:6, MleFC:0, IsChild:yes, RxOnIdle:yes, FTD:yes, SecDataReq:yes, FullNetData:yes" 204 "A2042C8762576FD5, RLOC16:dc00, LQIn:3, AveRssi:-20, LastRssi:-20, Age:5, LinkFC:21, MleFC:18, IsChild:no, RxOnIdle:yes, FTD:yes, SecDataReq:no, FullNetData:yes" 205] 206>>> print node1.get(wpan.WPAN_THREAD_NEIGHBOR_TABLE) 207[ 208 "960947C53415DAA1, RLOC16:9400, LQIn:3, AveRssi:-20, LastRssi:-20, Age:18, LinkFC:15, MleFC:11, IsChild:no, RxOnIdle:yes, FTD:yes, SecDataReq:no, FullNetData:yes" 209] 210 211``` 212 213### IPv6 Message Exchange 214 215`toranj` allows a test-case to define traffic patterns (IPv6 message exchange) between different nodes. Message exchanges (tx/rx) are prepared and then an async rx/tx operation starts. The success and failure of tx/rx operations can then be verified by the test case. 216 217`wpan.Node` method `prepare_tx()` prepares a UDP6 transmission from a node. 218 219```python 220 node1.prepare_tx(src, dst, data, count) 221``` 222 223- `src` and `dst` can be 224 225 - either a string containing an IPv6 address 226 - or a tuple (ipv6 address as string, port). if no port is given, a random port number is used. 227 228- `data` can be 229 230 - either a string containing the message to be sent, 231 - or an int indicating size of the message (a random message with the given length will be generated). 232 233- `count` gives number of times the message will be sent (default is 1). 234 235`prepare_tx` returns a `wpan.AsyncSender` object. The sender object can be used to check success/failure of tx operation. 236 237`wpan.Node` method `prepare_rx()` prepares a node to listen for UDP messages from a sender. 238 239```python 240 node2.prepare_rx(sender) 241``` 242 243- `sender` should be an `wpan.AsyncSender` object returned from previous `prepare_tx`. 244- `prepare_rx()` returns a `wpan.AsyncReceiver` object to help test to check success/failure of rx operation. 245 246After all exchanges are prepared, static method `perform_async_tx_rx()` should be used to start all previously prepared rx and tx operations. 247 248```python 249 wpan.Node.perform_async_tx_rx(timeout) 250``` 251 252- `timeout` gives amount of time (in seconds) to wait for all operations to finish. (default is 20 seconds) 253 254After `perform_async_tx_rx()` is done, the `AsyncSender` and `AsyncReceiver` objects can check if operations were successful (using property `was_successful`) 255 256#### Example 257 258Sending 10 messages containing `"Hello there!"` from `node1` to `node2` using their mesh-local addresses: 259 260```python 261# `node1` and `node2` are already joined and are part of the same Thread network. 262 263# Get the mesh local addresses 264>>> mladdr1 = node1.get(wpan.WPAN_IP6_MESH_LOCAL_ADDRESS)[1:-1] # remove `"` from start/end of string 265>>> mladdr2 = node2.get(wpan.WPAN_IP6_MESH_LOCAL_ADDRESS)[1:-1] 266 267>>> print (mladdr1, mladdr2) 268('fda4:38cf:5973:0:b899:3436:15c6:941d', 'fda4:38cf:5973:0:5836:fa55:7394:6d4b') 269 270# prepare a `sender` and corresponding `recver` 271>>> sender = node1.prepare_tx((mladdr1, 1234), (mladdr2, 2345), "Hello there!", 10) 272>>> recver = node2.prepare_rx(sender) 273 274# perform async message transfer 275>>> wpan.Node.perform_async_tx_rx() 276 277# check status of `sender` and `recver` 278>>> sender.was_successful 279True 280>>> recver.was_successful 281True 282 283# `sender` or `recver` can provide info about the exchange 284 285>>> sender.src_addr 286'fda4:38cf:5973:0:b899:3436:15c6:941d' 287>>> sender.src_port 2881234 289>>> sender.dst_addr 290'fda4:38cf:5973:0:5836:fa55:7394:6d4b' 291>>> sender.dst_port 2922345 293>>> sender.msg 294'Hello there!' 295>>> sender.count 29610 297 298# get all received msg by `recver` as list of tuples `(msg, (src_address, src_port))` 299>>> recver.all_rx_msg 300[('Hello there!', ('fda4:38cf:5973:0:b899:3436:15c6:941d', 1234)), ... ] 301``` 302 303### Logs and Verbose mode 304 305Every `wpan.Node()` instance will save its corresponding `wpantund` logs. By default the logs are saved in a file `wpantun-log<node_index>.log`. By setting `wpan.Node__TUND_LOG_TO_FILE` to `False` the logs are written to `stdout` as the test-cases are executed. 306 307When `start.sh` script is used to run all test-cases, if any test fails, to help with debugging of the issue, the last 30 lines of `wpantund` logs of every node involved in the test-case is dumped to `stdout`. 308 309A `wpan.Node()` instance can also provide additional logs and info as the test-cases are run (verbose mode). It can be enabled for a node instance when it is created: 310 311```python 312 node = wpan.Node(verbose=True) # `node` instance will provide extra logs. 313``` 314 315Alternatively, `wpan.Node._VERBOSE` settings can be changed to enable verbose logging for all nodes. The default value of `wpan.Node._VERBOSE` is determined from environment variable `TORANJ_VERBOSE` (verbose mode is enabled when env variable is set to any of `1`, `True`, `Yes`, `Y`, `On` (case-insensitive)), otherwise it is disabled. When `TORANJ_VERBOSE` is enabled, the OpenThread logging is also enabled (and collected in `wpantund-log<node_index>.log`files) on all nodes. 316 317Here is example of small test script and its corresponding log output with `verbose` mode enabled: 318 319```python 320node1 = wpan.Node(verbose=True) 321node2 = wpan.Node(verbose=True) 322 323wpan.Node.init_all_nodes() 324 325node1.form("toranj-net") 326node2.active_scan() 327 328node2.join_node(node1) 329verify(node2.get(wpan.WPAN_STATE) == wpan.STATE_ASSOCIATED) 330 331lladdr1 = node1.get(wpan.WPAN_IP6_LINK_LOCAL_ADDRESS)[1:-1] 332lladdr2 = node2.get(wpan.WPAN_IP6_LINK_LOCAL_ADDRESS)[1:-1] 333 334sender = node1.prepare_tx(lladdr1, lladdr2, 20) 335recver = node2.prepare_rx(sender) 336 337wpan.Node.perform_async_tx_rx() 338 339``` 340 341``` 342$ Node1.__init__() cmd: /usr/local/sbin/wpantund -o Config:NCP:SocketPath "system:../../examples/apps/ncp/ot-ncp-ftd 1" -o Config:TUN:InterfaceName wpan1 -o Config:NCP:DriverName spinel -o Daemon:SyslogMask "all -debug" 343$ Node2.__init__() cmd: /usr/local/sbin/wpantund -o Config:NCP:SocketPath "system:../../examples/apps/ncp/ot-ncp-ftd 2" -o Config:TUN:InterfaceName wpan2 -o Config:NCP:DriverName spinel -o Daemon:SyslogMask "all -debug" 344$ Node1.wpanctl('leave') -> 'Leaving current WPAN. . .' 345$ Node2.wpanctl('leave') -> 'Leaving current WPAN. . .' 346$ Node1.wpanctl('form "toranj-net"'): 347 Forming WPAN "toranj-net" as node type "router" 348 Successfully formed! 349$ Node2.wpanctl('scan'): 350 | PAN ID | Ch | XPanID | HWAddr | RSSI 351 ---+--------+----+------------------+------------------+------ 352 1 | 0x9DEB | 16 | 8CC6CFC810F23E1B | BEECDAF3439DC931 | -20 353$ Node1.wpanctl('get -v NCP:State') -> '"associated"' 354$ Node1.wpanctl('get -v Network:Name') -> '"toranj-net"' 355$ Node1.wpanctl('get -v Network:PANID') -> '0x9DEB' 356$ Node1.wpanctl('get -v Network:XPANID') -> '0x8CC6CFC810F23E1B' 357$ Node1.wpanctl('get -v Network:Key') -> '[BA2733A5D81EAB8FFB3C9A7383CB6045]' 358$ Node1.wpanctl('get -v NCP:Channel') -> '16' 359$ Node2.wpanctl('set Network:Key -d -v BA2733A5D81EAB8FFB3C9A7383CB6045') -> '' 360$ Node2.wpanctl('join "toranj-net" -c 16 -T r -p 0x9DEB -x 0x8CC6CFC810F23E1B'): 361 Joining "toranj-net" 8CC6CFC810F23E1B as node type "router" 362 Successfully Joined! 363$ Node2.wpanctl('get -v NCP:State') -> '"associated"' 364$ Node1.wpanctl('get -v IPv6:LinkLocalAddress') -> '"fe80::bcec:daf3:439d:c931"' 365$ Node2.wpanctl('get -v IPv6:LinkLocalAddress') -> '"fe80::ec08:f348:646f:d37d"' 366- Node1 sent 20 bytes (":YeQuNKjuOtd%H#ipM7P") to [fe80::ec08:f348:646f:d37d]:404 from [fe80::bcec:daf3:439d:c931]:12557 367- Node2 received 20 bytes (":YeQuNKjuOtd%H#ipM7P") on port 404 from [fe80::bcec:daf3:439d:c931]:12557 368 369``` 370