1Configuration Test 2------------------ 3To start the configuration test from the build directory do: 4 5./configuration-test -someip ../config/vsomeip-test.json 6 7The expected output is: 8 92015-02-10 08:47:31.503874 [info] Test "HOST ADDRESS" succeeded. 102015-02-10 08:47:31.507609 [info] Test "HAS CONSOLE" succeeded. 112015-02-10 08:47:31.507865 [info] Test "HAS FILE" succeeded. 122015-02-10 08:47:31.508001 [info] Test "HAS DLT" succeeded. 132015-02-10 08:47:31.508143 [info] Test "LOGFILE" succeeded. 142015-02-10 08:47:31.508315 [info] Test "LOGLEVEL" succeeded. 152015-02-10 08:47:31.508456 [info] Test "RELIABLE_TEST_1234_0022" succeeded. 162015-02-10 08:47:31.508593 [info] Test "UNRELIABLE_TEST_1234_0022" succeeded. 172015-02-10 08:47:31.508759 [info] Test "RELIABLE_TEST_1234_0023" succeeded. 182015-02-10 08:47:31.508896 [info] Test "UNRELIABLE_TEST_1234_0023" succeeded. 192015-02-10 08:47:31.509032 [info] Test "RELIABLE_TEST_2277_0022" succeeded. 202015-02-10 08:47:31.509185 [info] Test "UNRELIABLE_TEST_2277_0022" succeeded. 212015-02-10 08:47:31.509330 [info] Test "RELIABLE_TEST_4466_0321" succeeded. 222015-02-10 08:47:31.509467 [info] Test "UNRELIABLE_TEST_4466_0321" succeeded. 232015-02-10 08:47:31.509602 [info] Test "RELIABLE_TEST_2277_0022" succeeded. 242015-02-10 08:47:31.509771 [info] Test "UNRELIABLE_TEST_2277_0022" succeeded. 252015-02-10 08:47:31.509915 [info] Test "ADDRESS_TEST_1234_0022" succeeded. 262015-02-10 08:47:31.510049 [info] Test "MIN_INITIAL_DELAY_TEST_1234_0022" succeeded. 272015-02-10 08:47:31.510354 [info] Test "MAX_INITIAL_DELAY_TEST_1234_0022" succeeded. 282015-02-10 08:47:31.510610 [info] Test "REPETITION_BASE_DELAY_TEST_1234_0022" succeeded. 292015-02-10 08:47:31.513978 [info] Test "REPETITION_MAX_TEST_1234_0022" succeeded. 302015-02-10 08:47:31.514177 [info] Test "CYCLIC_OFFER_DELAY_TEST_1234_0022" succeeded. 312015-02-10 08:47:31.514280 [info] Test "CYCLIC_REQUEST_DELAY_TEST_1234_0022" succeeded. 322015-02-10 08:47:31.514397 [info] Test "MIN_INITIAL_DELAY_TEST_1234_0023" succeeded. 332015-02-10 08:47:31.514618 [info] Test "MAX_INITIAL_DELAY_TEST_1234_0023" succeeded. 342015-02-10 08:47:31.514754 [info] Test "REPETITION_BASE_DELAY_TEST_1234_0023" succeeded. 352015-02-10 08:47:31.514901 [info] Test "REPETITION_MAX_TEST_1234_0023" succeeded. 362015-02-10 08:47:31.515052 [info] Test "CYCLIC_OFFER_DELAY_TEST_1234_0023" succeeded. 372015-02-10 08:47:31.515186 [info] Test "CYCLIC_REQUEST_DELAY_TEST_1234_0023" succeeded. 382015-02-10 08:47:31.515325 [info] Test "MIN_INITIAL_DELAY_TEST_2277_0022" succeeded. 392015-02-10 08:47:31.515395 [info] Test "MAX_INITIAL_DELAY_TEST_2277_0022" succeeded. 402015-02-10 08:47:31.515536 [info] Test "REPETITION_BASE_DELAY_TEST_2277_0022" succeeded. 412015-02-10 08:47:31.515691 [info] Test "REPETITION_MAX_TEST_2277_0022" succeeded. 422015-02-10 08:47:31.515834 [info] Test "CYCLIC_OFFER_DELAY_TEST_2277_0022" succeeded. 432015-02-10 08:47:31.515971 [info] Test "CYCLIC_REQUEST_DELAY_TEST_2277_0022" succeeded. 442015-02-10 08:47:31.516109 [info] Test "MIN_INITIAL_DELAY_TEST_2266_0022" succeeded. 452015-02-10 08:47:31.516279 [info] Test "MAX_INITIAL_DELAY_TEST_2266_0022" succeeded. 462015-02-10 08:47:31.516380 [info] Test "REPETITION_BASE_DELAY_TEST_2266_0022" succeeded. 472015-02-10 08:47:31.516512 [info] Test "REPETITION_MAX_TEST_2266_0022" succeeded. 482015-02-10 08:47:31.516610 [info] Test "CYCLIC_OFFER_DELAY_TEST_2266_0022" succeeded. 492015-02-10 08:47:31.516736 [info] Test "CYCLIC_REQUEST_DELAY_TEST_2266_0022" succeeded. 502015-02-10 08:47:31.516874 [info] Test "ADDRESS_TEST_4466_0321" succeeded. 512015-02-10 08:47:31.516974 [info] Test "SERVICE DISCOVERY PROTOCOL" succeeded. 522015-02-10 08:47:31.517106 [info] Test "SERVICE DISCOVERY PORT" succeeded. 53 54 55Application test 56---------------- 57 58This test tests starting and stopping a vsomeip application in various ways. 59 60Automatic start from build directory: 61 62ctest -V -R application_test 63 64Manual start from sub folder test of build directory: 65 66./application_test_starter.sh 67 68 69Magic Cookies Test 70------------------ 71To run the magic cookies test you need two devices on the same network. The network addresses within 72the configuration files need to be adapted to match the devices addresses. 73 74To start the magic-cookies-test from the build-directory do: 75 76Automatic start from build directory: 77 78ctest -V -R magic_cookies_test 79 80Manual start from sub folder test of build directory: 81 82# On external host run 83./magic_cookies_test_service_start.sh 84 85# On local host run 86./magic_cookies_test_client_start.sh 87 88 89The expected result is an output like this on service side: 90 912015-02-10 08:42:07.317695 [info] Received a message with Client/Session [1343/0001] 922015-02-10 08:42:07.360105 [error] Detected Magic Cookie within message data. Resyncing. 932015-02-10 08:42:07.360298 [info] Received a message with Client/Session [1343/0003] 942015-02-10 08:42:07.360527 [error] Detected Magic Cookie within message data. Resyncing. 952015-02-10 08:42:07.360621 [error] Detected Magic Cookie within message data. Resyncing. 962015-02-10 08:42:07.360714 [info] Received a message with Client/Session [1343/0006] 972015-02-10 08:42:07.360850 [info] Received a message with Client/Session [1343/0007] 982015-02-10 08:42:07.361021 [error] Detected Magic Cookie within message data. Resyncing. 992015-02-10 08:42:07.361107 [error] Detected Magic Cookie within message data. Resyncing. 1002015-02-10 08:42:07.361191 [error] Detected Magic Cookie within message data. Resyncing. 1012015-02-10 08:42:07.361276 [info] Received a message with Client/Session [1343/000b] 1022015-02-10 08:42:07.361434 [info] Received a message with Client/Session [1343/000c] 1032015-02-10 08:42:07.361558 [info] Received a message with Client/Session [1343/000d] 1042015-02-10 08:42:07.361672 [error] Detected Magic Cookie within message data. Resyncing. 1052015-02-10 08:42:07.361761 [info] Received a message with Client/Session [1343/000f] 106 107Header Factory Tests 108-------------------- 109 110The following things are tested: 111a) create request 112 --> check "Protocol Version" / "Message Type" / "Return Type" fields 113b) create request, fill header, create response 114 --> compare header fields of request & response 115c) create notification 116 --> check "Protocol Version" / "Message Type" / "Return Type" fields 117d) create message, fill header (service/instance/method/interface version/message type) 118 --> send message 10 times 119 --> receive message and check client id / session id 120 121a) to c) are combined in one binary. d) is composed out of a client and service. 122 123To start the header factory tests from the build directory do: 124 125Automatic start from build directory: 126ctest -V -R header_factory_test 127 128Manual start from build directory: 129cd test 130./header_factory_test 131# Start client and service separately 132./header_factory_test_service_start.sh & 133./header_factory_test_client_start.sh 134# Alternatively start client and service with one script 135./header_factory_test_send_receive_starter.sh 136 137All tests should be marked as "passed". 138 139Routing Tests 140------------- 141 142The following things are tested: 143a) create a service instance 144 - check that it is accessible from a local client but invisible for an external client 145b) create a service instance, configure it to be externally visible 146 - check that it is accessible from a local client and from a external client 147 148a) and b) are composed out of a service each and one common client binary which is used 149with different configuration files. 150 151Automatic start from build directory: 152 153ctest -V -R local_routing_test 154 155A message will be shown when the external client should be started. 156 157Manual start from build directory: 158cd test 159# First part with local client 160# Start client and service with one script 161./local_routing_test_starter.sh 162 163# Alternatively start client and service separately 164# Warning some checks are done within the *_starter.sh script. 165# This should only be used for debugging 166# Start the service 167./local_routing_test_service_start.sh & 168# Start the client 169./local_routing_test_client_start.sh 170 171# Second part with external client 172# Start client and service with one script 173./external_local_routing_test_starter.sh 174# Start the external client from an external host when the message is displayed to start it 175./external_local_routing_test_client_external_start.sh 176 177# Alternatively start client and service separately 178# Warning some checks are done within the *_starter.sh script. 179# This should only be used for debugging 180# Start the service 181./external_local_routing_test_service_start.sh & 182# Start the client 183./local_routing_test_client_start.sh 184# Start the external client from an external host after local client has finished 185./external_local_routing_test_client_external_start.sh 186 187 188All tests should be marked as "passed". 189 190Payload Tests 191------------- 192 193The following things are tested: 194a) create a local service 195 - send messages with payloads of different size from a local client to the service 196 - check that the messages are received correctly 197 - measure the throughput 198b) create a service instance, configure it to be externally visible 199 - send messages with payloads of different size from a local client to the service 200 - check that the messages are received correctly 201 - measure the throughput 202c) create a service instance, configure it to be externally visible 203 - send messages with payloads of different size from an external client to the service 204 - check that the messages are received correctly 205 - measure the throughput 206d) create a service instance, configure it to be externally visible 207 - send messages with payloads of different size from a local client to the service 208 - send messages with payloads of different size from an external client to the service 209 - check that the messages are received correctly 210 - measure the throughput 211 212The tests a) to d) are composed out of a service and a client binary which are called 213with different configuration files and parameters. 214 215Automatic start from build directory: 216 217ctest -V -R payload_test 218 219A message will be shown when the external clients should be started. 220 221Manual start from build directory: 222cd test 223 224# First part with local client 225# start client and service with one script 226./local_payload_test_starter.sh 227 228# Alternatively start client and service separately 229# Warning some checks are done within the *_starter.sh script. 230# This should only be used for debugging 231./local_payload_test_service_start.sh & 232./local_payload_test_client_start.sh 233 234# Second part with external visible service and local client 235# start client and service with one script 236./external_local_payload_test_client_local_starter.sh 237 238# Alternatively start client and service separately 239# Warning some checks are done within the *_starter.sh script. 240# This should only be used for debugging 241./external_local_payload_test_service_start.sh & 242./external_local_payload_test_client_local_start.sh 243 244# Third part with external visible service and external client 245# start client and service with one script 246./external_local_payload_test_client_external_starter.sh 247# Start the external client from an external host if asked to 248./external_local_payload_test_client_external_start.sh 249 250# Alternatively start client and service separately 251# Warning some checks are done within the *_starter.sh script. 252# This should only be used for debugging 253./external_local_payload_test_service_client_external_start.sh 254# Start the external client from an external host 255./external_local_payload_test_client_external_start.sh 256 257# Fourth part with external visible service and local and external client 258# start client and service with one script 259./external_local_payload_test_client_local_and_external_starter.sh 260# Start the external client from an external host if asked to 261./external_local_payload_test_client_external_start.sh 262 263# Alternatively start client and service separately 264# Warning some checks are done within the *_starter.sh script. 265# This should only be used for debugging 266./external_local_payload_test_service_client_external_start.sh & 267# Start the local client 268VSOMEIP_APPLICATION_NAME=external_local_payload_test_client_local \ 269VSOMEIP_CONFIGURATION=external_local_payload_test_client_local.json \ 270./payload_test_client --dont-shutdown-service 271# Start the external client after the local client is finished from an 272# external host 273./external_local_payload_test_client_external_start.sh 274 275All tests should be marked as "passed". 276 277 278Big payload tests 279----------------- 280 281This test tests the possibility to sent messages with bigger payloads 282for local and TCP communication. 283 284The test will send a messages with 600k payload from a client to a service. 285The service will reply with a response containing 600k payload as well. 286This is repeated 10 times. 287There is a version for local and for TCP communication available. 288Additionally there are test versions available which sent up to 10MiB big 289messages and a version which tests the limitiation of message sizes configurable 290via json file. 291 292Automatic start from the build directory: 293 294ctest -V -R big_payload_test_local 295 296Manual start from sub folder test of build directory: 297 298./big_payload_test_service_local_start.sh & 299./big_payload_test_client_local_start.sh 300 301 302Automatic start of the TCP version from the build directory: 303 304ctest -V -R big_payload_test_external 305 306Manual start from sub folder test of build directory: 307 308./big_payload_test_client_start.sh 309 310# On external host run 311./big_payload_test_service_external_start.sh 312 313 314Client ID tests 315--------------- 316 317This tests tests communication over two nodes with multiple services on both 318nodes. 319 320The test setup is as followed: 321* There are six services with one method each. 322* Three of the services run on node 1. 323* Three of the services run on node 2. 324* Each of the services sends ten requests to the other services and waits 325 until it received a response for every request. 326* If all responses have been received, the service shutdown. 327 328Automatic start from the build directory: 329 330ctest -V -R client_id_test_diff_client_ids_diff_ports 331 332Manual start from sub folder test of build directory: 333 334./client_id_test_master_starter.sh client_id_test_diff_client_ids_diff_ports_master.json 335 336Second version where all services on one node use the same port: 337 338Automatic start from the build directory: 339 340ctest -V -R client_id_test_diff_client_ids_same_ports 341 342Manual start from sub folder test of build directory: 343 344./client_id_test_master_starter.sh client_id_test_diff_client_ids_same_ports_master.json 345 346 347Subscribe notify tests 348---------------------- 349This tests tests subscribe notify mechanism over two nodes with multiple services 350on both nodes. 351 352The test setup is as followed: 353* There are six services offering one event each. 354* Three of the services run on node 1. 355* Three of the services run on node 2. 356* Each of the services waits until all other services are available. 357* Each of the services subscribes to the offered event of all the other services. 358* Each of the services then waits until the other services have subscribed to 359 its event. 360* Each of the services then starts to sent out ten notifications for its event. 361* Each service waits until it received the correct amount of notifications from 362 all other services. 363* If all notifications have been received, the service shuts down. 364 365Automatic start from the build directory (example): 366 367ctest -V -R subscribe_notify_test_diff_client_ids_diff_ports_udp 368 369Manual start from sub folder test of build directory: 370 371./subscribe_notify_test_master_starter.sh UDP subscribe_notify_test_diff_client_ids_diff_ports_master.json 372 373There are multiple versions of this test which differ in the used subscription 374method and port setup (use ctest -N to see all). For manual start the desired 375description method has to be passed to the starter script as first parameter. 376 377The subscribe_notify_test_one_event_two_eventgroups_* tests are testing the 378requirement that for events which are member of multiple eventgroups initial 379events shall be sent per eventgroup. However normal updates of the event should 380be sent only once even if a remote subscriber is subscribed to multiple of the 381event's eventgroups (TR_SOMEIP_00570). 382 383 384CPU load test 385------------- 386This test does a increasing number of synchronous function calls to the same 387method of the service and measures CPU load for each batch of function calls. 388All method calls transport a payload of 40 Bytes. The responses don't transport 389any payload. 390 391The CPU load is measured thorugh the proc fs. 392If the test prints a message like: 393 394 Synchronously sent 0890 messages. CPU load [%]: 12.68 395 396This means that the test process consumed 12% of the jiffies consumed by 397complete system while doing 890 methodcalls. 398 399Automatic start from the build directory (example): 400 401ctest -V -R cpu_load_test 402 403 404Initial event tests 405---------------------- 406This tests tests initial event mechanism over two nodes with multiple services 407on both nodes. 408 409The test setup is as followed: 410* There are six services offering one event each. 411* Three of the services run on node 1. 412* Three of the services run on node 2. 413* All of the services initially set their event to their service id and notify 414 once 415* On each node there are 20 client applications which subscribe to all of the 416 services events which are started at different times 417* Each client waits until it received one notification (the initial one) from 418 all services and then exits. 419* If all clients exited, the services are killed as well 420 421Automatic start from the build directory (example): 422 423ctest -V -R initial_event_test_diff_client_ids_diff_ports_udp 424 425Manual start from sub folder test of build directory: 426./initial_event_test_master_starter.sh UDP initial_event_test_diff_client_ids_diff_ports_master.json 427 428There are multiple versions of this test which differ in the used subscription 429method and port setup (use ctest -N to see all). For manual start the desired 430description method has to be passed to the starter script as first parameter. 431 432Offer tests 433----------- 434This tests test various cases of offering a service and error recovery 435after an application became unresponsive 436 437* Rejecting offer of service instance whose hosting application is 438 still alive. 439* Rejecting offer of service instance whose hosting application is 440 still alive with daemon 441* Accepting offer of service instance whose hosting application 442 crashed with (send SIGKILL) 443* Accepting offer of service instance whose hosting application became 444 unresponsive (SIGSTOP) 445* Rejecting offers for which there is already a pending offer 446* Rejecting remote offer for which there is already a local offer 447* Rejecting a local offer for which there is already a remote offer 448 449Automatic start from the build directory (example): 450 451ctest -V -R offer_tests 452 453Manual start from sub folder test of build directory: 454./offer_test_local_starter 455./offer_test_external_master_starter.sh 456 457Tests in detail: 458Rejecting offer of service instance whose hosting application is still 459alive: 460* start application which offers service 461* start client which continuously exchanges messages with the service 462* start application which offers the same service again -> should be 463 rejected and an error message should be printed. 464* Message exchange with client application should not be interrupted. 465 466Rejecting offer of service instance whose hosting application is still 467alive with daemon 468* start daemon (needed as he has to ping the offering client) 469* start application which offers service 470* start client which continuously exchanges messages with the service 471* start application which offers the same service again -> should be 472 rejected and an error message should be printed. 473* Message exchange with client application should not be interrupted. 474 475Accepting offer of service instance whose hosting application crashed 476with (send SIGKILL) 477* start daemon 478* start application which offers service 479* start client which exchanges messages with the service 480* kill application with SIGKILL 481* start application which offers the same service again -> should be 482 accepted. 483* start another client which exchanges messages with the service 484* Client should now communicate with new offerer. 485 486Accepting offer of service instance whose hosting application became 487unresponsive (SIGSTOP) 488* start daemon 489* start application which offers service 490* Send a SIGSTOP to the service to make it unresponsive 491* start application which offers the same service again -> should be 492 marked as PENDING_OFFER and a ping should be sent to the paused 493 application. 494* After the timeout passed the new offer should be accepted. 495* start client which exchanges messages with the service 496* Client should now communicate with new offerer. 497 498Rejecting offers for which there is already a pending offer 499* start daemon 500* start application which offers service 501* Send a SIGSTOP to the service to make it unresponsive 502* start application which offers the same service again -> should be 503 marked as PENDING_OFFER and a ping should be sent to the paused 504 application. 505* start application which offers the same service again -> should be 506 rejected as there is already a PENDING_OFFER pending. 507* After the timeout passed the new offer should be accepted. 508* start client which exchanges messages with the service 509* Client should now communicate with new offerer. 510 511Rejecting a local offer for which there is already a remote offer: 512* start daemon 513* start application which offers service 514* start daemon remotely 515* start same application which offers the same service again remotely 516 -> should be rejected as there is already a service instance 517 running in the network 518 519Rejecting remote offer for which there is already a local offer 520* start application which offers service 521* send SD message trying to offer the same service instance as already 522 offered locally from a remote host -> should be rejected 523 524nPDU tests 525----------------- 526 527This test is intended to test the functionality of the so called nPDU 528feature. The test setup is as followed: 529 530* There are two nodes, one hosting the services, one hosting the clients. 531* On each of the nodes is a routing manager daemon (RMD) started whose only 532 purpose is to provide routing manager functionality and shutdown the clients 533 and services at the end. 534* There are four services created. Each of the services has four methods. 535* All services are listening on the same port. Therefore there only is: 536 * one server endpoint created in the RMD on service side 537 * one client endpoint created in the RMD on client side 538* There are four clients created. Each of the clients will: 539 * Create a thread for each service 540 * Create a thread for each method of each service 541 * Send multiple messages with increasing payload to each of the services' 542 methods from the corresponding thread. 543 * After sending the threads will sleep the correct amount of time to insure 544 applicative debounce > debounce time + max retention time. 545* After all messages have been sent to the services the clients will notify the 546 RMD that they're finished. The RMD then instructs the RMD on service side to 547 shutdown the services and exit afterwards. After that the RMD on client side 548 exits as well. 549* Upon receiving a method call the service will check if the debounce time 550 specified in the json file for this method was undershot and print out a 551 warning. 552* The test first runs in synchronous mode and waits for a response of the 553 service before sending the next message. 554* After that the test runs in a mode where no response from the service are 555 required (message type REQUEST_NO_RETURN) thus the clients send at maximum 556 allowed frequency. 557 558Automatic start from build directory: 559 560ctest -V -R npdu_test_UDP 561ctest -V -R npdu_test_TCP 562 563A message will be shown when the external clients should be started. 564 565Manual start: 566# Service side 567./npdu_test_service_npdu_start.sh 568 569# Client side UDP mode 570./npdu_test_client_npdu_start.sh UDP 571 572# Client side TCP mode 573./npdu_test_client_npdu_start.sh TCP 574