1# Unity Helper Scripts 2 3## With a Little Help From Our Friends 4 5Sometimes what it takes to be a really efficient C programmer is a little non-C. 6The Unity project includes a couple of Ruby scripts for making your life just a tad easier. 7They are completely optional. 8If you choose to use them, you'll need a copy of Ruby, of course. 9Just install whatever the latest version is, and it is likely to work. You can find Ruby at [ruby-lang.org][]. 10 11### `generate_test_runner.rb` 12 13Are you tired of creating your own `main` function in your test file? 14Do you keep forgetting to add a `RUN_TEST` call when you add a new test case to your suite? 15Do you want to use CMock or other fancy add-ons but don't want to figure out how to create your own `RUN_TEST` macro? 16 17Well then we have the perfect script for you! 18 19The `generate_test_runner` script processes a given test file and automatically creates a separate test runner file that includes ?main?to execute the test cases within the scanned test file. 20All you do then is add the generated runner to your list of files to be compiled and linked, and presto you're done! 21 22This script searches your test file for void function signatures having a function name beginning with "test" or "spec". 23It treats each of these functions as a test case and builds up a test suite of them. 24For example, the following includes three test cases: 25 26```C 27void testVerifyThatUnityIsAwesomeAndWillMakeYourLifeEasier(void) 28{ 29 ASSERT_TRUE(1); 30} 31void test_FunctionName_should_WorkProperlyAndReturn8(void) { 32 ASSERT_EQUAL_INT(8, FunctionName()); 33} 34void spec_Function_should_DoWhatItIsSupposedToDo(void) { 35 ASSERT_NOT_NULL(Function(5)); 36} 37``` 38 39You can run this script a couple of ways. 40The first is from the command line: 41 42```Shell 43ruby generate_test_runner.rb TestFile.c NameOfRunner.c 44``` 45 46Alternatively, if you include only the test file parameter, the script will copy the name of the test file and automatically append `_Runner` to the name of the generated file. 47The example immediately below will create TestFile_Runner.c. 48 49```Shell 50ruby generate_test_runner.rb TestFile.c 51``` 52 53You can also add a [YAML][] file to configure extra options. 54Conveniently, this YAML file is of the same format as that used by Unity and CMock. 55So if you are using YAML files already, you can simply pass the very same file into the generator script. 56 57```Shell 58ruby generate_test_runner.rb TestFile.c my_config.yml 59``` 60 61The contents of the YAML file `my_config.yml` could look something like the example below. 62If you're wondering what some of these options do, you're going to love the next section of this document. 63 64```YAML 65:unity: 66 :includes: 67 - stdio.h 68 - microdefs.h 69 :cexception: 1 70 :suite_setup: "blah = malloc(1024);" 71 :suite_teardown: "free(blah);" 72``` 73 74If you would like to force your generated test runner to include one or more header files, you can just include those at the command line too. 75Just make sure these are _after_ the YAML file, if you are using one: 76 77```Shell 78ruby generate_test_runner.rb TestFile.c my_config.yml extras.h 79``` 80 81Another option, particularly if you are already using Ruby to orchestrate your builds - or more likely the Ruby-based build tool Rake - is requiring this script directly. 82Anything that you would have specified in a YAML file can be passed to the script as part of a hash. 83Let's push the exact same requirement set as we did above but this time through Ruby code directly: 84 85```Ruby 86require "generate_test_runner.rb" 87options = { 88 :includes => ["stdio.h", "microdefs.h"], 89 :cexception => 1, 90 :suite_setup => "blah = malloc(1024);", 91 :suite_teardown => "free(blah);" 92} 93UnityTestRunnerGenerator.new.run(testfile, runner_name, options) 94``` 95 96If you have multiple files to generate in a build script (such as a Rakefile), you might want to instantiate a generator object with your options and call it to generate each runner afterwards. 97Like thus: 98 99```Ruby 100gen = UnityTestRunnerGenerator.new(options) 101test_files.each do |f| 102 gen.run(f, File.basename(f,'.c')+"Runner.c" 103end 104``` 105 106#### Options accepted by generate_test_runner.rb 107 108The following options are available when executing `generate_test_runner`. 109You may pass these as a Ruby hash directly or specify them in a YAML file, both of which are described above. 110In the `examples` directory, Example 3's Rakefile demonstrates using a Ruby hash. 111 112##### `:includes` 113 114This option specifies an array of file names to be `#include`'d at the top of your runner C file. 115You might use it to reference custom types or anything else universally needed in your generated runners. 116 117##### `:defines` 118 119This option specifies an array of definitions to be `#define`'d at the top of your runner C file. 120Each definition will be wrapped in an `#ifndef`. 121 122##### `:suite_setup` 123 124Define this option with C code to be executed _before any_ test cases are run. 125 126Alternatively, if your C compiler supports weak symbols, you can leave this option unset and instead provide a `void suiteSetUp(void)` function in your test suite. 127The linker will look for this symbol and fall back to a Unity-provided stub if it is not found. 128 129This option can also be specified at the command prompt as `--suite_setup=""` 130 131##### `:suite_teardown` 132 133Define this option with C code to be executed _after all_ test cases have finished. 134An integer variable `num_failures` is available for diagnostics. 135The code should end with a `return` statement; the value returned will become the exit code of `main`. 136You can normally just return `num_failures`. 137 138Alternatively, if your C compiler supports weak symbols, you can leave this option unset and instead provide a `int suiteTearDown(int num_failures)` function in your test suite. 139The linker will look for this symbol and fall back to a Unity-provided stub if it is not found. 140 141This option can also be specified at the command prompt as `--suite_teardown=""` 142 143##### `:enforce_strict_ordering` 144 145This option should be defined if you have the strict order feature enabled in CMock (see CMock documentation). 146This generates extra variables required for everything to run smoothly. 147If you provide the same YAML to the generator as used in CMock's configuration, you've already configured the generator properly. 148 149##### `:externc` 150 151This option should be defined if you are mixing C and CPP and want your test runners to automatically include extern "C" support when they are generated. 152 153This option can also be specified at the command prompt as `--externc` 154 155##### `:mock_prefix` and `:mock_suffix` 156 157Unity automatically generates calls to Init, Verify and Destroy for every file included in the main test file that starts with the given mock prefix and ends with the given mock suffix, file extension not included. 158By default, Unity assumes a `Mock` prefix and no suffix. 159 160##### `:plugins` 161 162This option specifies an array of plugins to be used (of course, the array can contain only a single plugin). 163This is your opportunity to enable support for CException support, which will add a check for unhandled exceptions in each test, reporting a failure if one is detected. 164To enable this feature using Ruby: 165 166```Ruby 167:plugins => [ :cexception ] 168``` 169 170Or as a yaml file: 171 172```YAML 173:plugins: 174 -:cexception 175``` 176 177If you are using CMock, it is very likely that you are already passing an array of plugins to CMock. 178You can just use the same array here. 179 180This script will just ignore the plugins that don't require additional support. 181 182This option can also be specified at the command prompt as `--cexception` 183 184##### `:include_extensions` 185 186This option specifies the pattern for matching acceptable header file extensions. 187By default it will accept hpp, hh, H, and h files. 188If you need a different combination of files to search, update this from the default `'(?:hpp|hh|H|h)'`. 189 190##### `:source_extensions` 191 192This option specifies the pattern for matching acceptable source file extensions. 193By default it will accept cpp, cc, C, c, and ino files. 194If you need a different combination of files to search, update this from the default `'(?:cpp|cc|ino|C|c)'`. 195 196##### `:use_param_tests` 197 198This option enables parameterized test usage. 199That tests accepts arguments from `TEST_CASE` and `TEST_RANGE` macros, 200that are located above current test definition. 201By default, Unity assumes, that parameterized tests are disabled. 202 203Few usage examples can be found in `/test/tests/test_unity_parameterized.c` file. 204 205You should define `UNITY_SUPPORT_TEST_CASES` macro for tests success compiling, 206if you enable current option. 207 208You can see list of supported macros list in the 209[Parameterized tests provided macros](#parameterized-tests-provided-macros) 210section that follows. 211 212This option can also be specified at the command prompt as `--use_param_tests=1` 213 214##### `:cmdline_args` 215 216When set to `true`, the generated test runner can accept a number of 217options to modify how the test(s) are run. 218 219Ensure Unity is compiled with `UNITY_USE_COMMAND_LINE_ARGS` defined or else 220the required functions will not exist. 221 222These are the available options: 223 224| Option | Description | 225| --------- | ------------------------------------------------- | 226| `-l` | List all tests and exit | 227| `-f NAME` | Filter to run only tests whose name includes NAME | 228| `-n NAME` | (deprecated) alias of -f | 229| `-h` | show the Help menu that lists these options | 230| `-q` | Quiet/decrease verbosity | 231| `-v` | increase Verbosity | 232| `-x NAME` | eXclude tests whose name includes NAME | 233 234##### `:setup_name` 235 236Override the default test `setUp` function name. 237 238This option can also be specified at the command prompt as `--setup_name=""` 239 240##### `:teardown_name` 241 242Override the default test `tearDown` function name. 243 244This option can also be specified at the command prompt as `--teardown_name=""` 245 246##### `:test_reset_name` 247 248Override the default test `resetTest` function name. 249 250This option can also be specified at the command prompt as `--test_reset_name=""` 251 252##### `:test_verify_name` 253 254Override the default test `verifyTest` function name. 255 256This option can also be specified at the command prompt as `--test_verify_name=""` 257 258##### `:main_name` 259 260Override the test's `main()` function name (from `main` to whatever is specified). 261The sentinel value `:auto` will use the test's filename with the `.c` extension removed prefixed 262with `main_` as the "main" function. 263 264To clarify, if `:main_name == :auto` and the test filename is "test_my_project.c", then the 265generated function name will be `main_test_my_project(int argc, char** argv)`. 266 267This option can also be specified at the command prompt as `--main_name=""` 268 269##### `main_export_decl` 270 271Provide any `cdecl` for the `main()` test function. Is empty by default. 272 273##### `:omit_begin_end` 274 275If `true`, the `UnityBegin` and `UnityEnd` function will not be called for 276Unity test state setup and cleanup. 277 278This option can also be specified at the command prompt as `--omit_begin_end` 279 280#### Parameterized tests provided macros 281 282Unity provides support for few param tests generators, that can be combined 283with each other. You must define test function as usual C function with usual 284C arguments, and test generator will pass what you tell as a list of arguments. 285 286Let's show how all of them works on the following test function definitions: 287 288```C 289/* Place your test generators here, usually one generator per one or few lines */ 290void test_demoParamFunction(int a, int b, int c) 291{ 292 TEST_ASSERT_GREATER_THAN_INT(a + b, c); 293} 294``` 295 296##### `TEST_CASE` 297 298Test case is a basic generator, that can be used for param testing. 299One call of that macro will generate only one call for test function. 300It can be used with different args, such as numbers, enums, strings, 301global variables, another preprocessor defines. 302 303If we use replace comment before test function with the following code: 304 305```C 306TEST_CASE(1, 2, 5) 307TEST_CASE(10, 7, 20) 308``` 309 310script will generate 2 test calls: 311 312```C 313test_demoParamFunction(1, 2, 5); 314test_demoParamFunction(10, 7, 20); 315``` 316 317That calls will be wrapped with `setUp`, `tearDown` and other 318usual Unity calls, as for independent unit tests. 319The following output can be generated after test executable startup: 320 321```Log 322tests/test_unity_parameterizedDemo.c:14:test_demoParamFunction(1, 2, 5):PASS 323tests/test_unity_parameterizedDemo.c:14:test_demoParamFunction(10, 7, 20):PASS 324``` 325 326##### `TEST_RANGE` 327 328Test range is an advanced generator. It single call can be converted to zero, 329one or few `TEST_CASE` equivalent commands. 330 331That generator can be used for creating numeric ranges in decimal representation 332only: integers & floating point numbers. It uses few formats for every parameter: 333 3341. `[start, stop, step]` is stop-inclusive format 3352. `<start, stop, step>` is stop-exclusive formats 336 337Format providers 1 and 2 accept only three arguments: 338 339* `start` is start number 340* `stop` is end number (can or cannot exists in result sequence for format 1, 341will be always skipped for format 2) 342* `step` is incrementing step: can be either positive or negative value. 343 344Let's use our `test_demoParamFunction` test for checking, what ranges 345will be generated for our single `TEST_RANGE` row: 346 347```C 348TEST_RANGE([3, 4, 1], [10, 5, -2], <30, 31, 1>) 349``` 350 351Tests execution output will be similar to that text: 352 353```Log 354tests/test_unity_parameterizedDemo.c:14:test_demoParamFunction(3, 10, 30):PASS 355tests/test_unity_parameterizedDemo.c:14:test_demoParamFunction(3, 8, 30):PASS 356tests/test_unity_parameterizedDemo.c:14:test_demoParamFunction(3, 6, 30):PASS 357tests/test_unity_parameterizedDemo.c:14:test_demoParamFunction(4, 10, 30):PASS 358tests/test_unity_parameterizedDemo.c:14:test_demoParamFunction(4, 8, 30):PASS 359tests/test_unity_parameterizedDemo.c:14:test_demoParamFunction(4, 6, 30):PASS 360``` 361 362As we can see: 363 364| Parameter | Format | Possible values | Total of values | Format number | 365|---|---|---|---|---| 366| `a` | `[3, 4, 1]` | `3`, `4` | 2 | Format 1 | 367| `b` | `[10, 5, -2]` | `10`, `8`, `6` | 3 | Format 1, negative step, end number is not included | 368| `c` | `<30, 31, 1>` | `30` | 1 | Format 2 | 369 370_Note_, that format 2 also supports negative step. 371 372We totally have 2 * 3 * 1 = 6 equal test cases, that can be written as following: 373 374```C 375TEST_CASE(3, 10, 30) 376TEST_CASE(3, 8, 30) 377TEST_CASE(3, 6, 30) 378TEST_CASE(4, 10, 30) 379TEST_CASE(4, 8, 30) 380TEST_CASE(4, 6, 30) 381``` 382 383##### `TEST_MATRIX` 384 385Test matix is an advanced generator. It single call can be converted to zero, 386one or few `TEST_CASE` equivalent commands. 387 388That generator will create tests for all cobinations of the provided list. Each argument has to be given as a list of one or more elements in the format `[<parm1>, <param2>, ..., <paramN-1>, <paramN>]`. 389 390All parameters supported by the `TEST_CASE` is supported as arguments: 391- Numbers incl type specifiers e.g. `<1>`, `<1u>`, `<1l>`, `<2.3>`, or `<2.3f>` 392- Strings incl string concatianion e.g. `<"string">`, or `<"partial" "string">` 393- Chars e.g. `<'c'>` 394- Enums e.g. `<ENUM_NAME>` 395- Elements of arrays e.g. `<data[0]>` 396 397Let's use our `test_demoParamFunction` test for checking, what ranges 398will be generated for our single `TEST_RANGE` row: 399 400```C 401TEST_MATRIX([3, 4, 7], [10, 8, 2, 1],[30u, 20.0f]) 402``` 403 404Tests execution output will be similar to that text: 405 406```Log 407tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(3, 10, 30u):PASS 408tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(3, 10, 20.0f):PASS 409tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(3, 8, 30u):PASS 410tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(3, 8, 20.0f):PASS 411tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(3, 2, 30u):PASS 412tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(3, 2, 20.0f):PASS 413tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(3, 1, 30u):PASS 414tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(3, 1, 20.0f):PASS 415tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(4, 10, 30u):PASS 416tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(4, 10, 20.0f):PASS 417tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(4, 8, 30u):PASS 418tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(4, 8, 20.0f):PASS 419tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(4, 2, 30u):PASS 420tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(4, 2, 20.0f):PASS 421tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(4, 1, 30u):PASS 422tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(4, 1, 20.0f):PASS 423tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(7, 10, 30u):PASS 424tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(7, 10, 20.0f):PASS 425tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(7, 8, 30u):PASS 426tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(7, 8, 20.0f):PASS 427tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(7, 2, 30u):PASS 428tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(7, 2, 20.0f):PASS 429tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(7, 1, 30u):PASS 430tests/test_unity_parameterizedDemo.c:18:test_demoParamFunction(7, 1, 20.0f):PASS 431``` 432 433As we can see: 434 435| Parameter | Format | Count of values | 436|---|---|---| 437| `a` | `[3, 4, 7]` | 2 | 438| `b` | `[10, 8, 2, 1]` | 4 | 439| `c` | `[30u, 20.0f]` | 2 | 440 441We totally have 2 * 4 * 2 = 16 equal test cases, that can be written as following: 442 443```C 444TEST_CASE(3, 10, 30u) 445TEST_CASE(3, 10, 20.0f) 446TEST_CASE(3, 8, 30u) 447TEST_CASE(3, 8, 20.0f) 448TEST_CASE(3, 2, 30u) 449TEST_CASE(3, 2, 20.0f) 450TEST_CASE(3, 1, 30u) 451TEST_CASE(3, 1, 20.0f) 452TEST_CASE(4, 10, 30u) 453TEST_CASE(4, 10, 20.0f) 454TEST_CASE(4, 8, 30u) 455TEST_CASE(4, 8, 20.0f) 456TEST_CASE(4, 2, 30u) 457TEST_CASE(4, 2, 20.0f) 458TEST_CASE(4, 1, 30u) 459TEST_CASE(4, 1, 20.0f) 460TEST_CASE(7, 10, 30u) 461TEST_CASE(7, 10, 20.0f) 462TEST_CASE(7, 8, 30u) 463TEST_CASE(7, 8, 20.0f) 464TEST_CASE(7, 2, 30u) 465TEST_CASE(7, 2, 20.0f) 466TEST_CASE(7, 1, 30u) 467TEST_CASE(7, 1, 20.0f) 468``` 469 470### `unity_test_summary.rb` 471 472A Unity test file contains one or more test case functions. 473Each test case can pass, fail, or be ignored. 474Each test file is run individually producing results for its collection of test cases. 475A given project will almost certainly be composed of multiple test files. 476Therefore, the suite of tests is comprised of one or more test cases spread across one or more test files. 477This script aggregates individual test file results to generate a summary of all executed test cases. 478The output includes how many tests were run, how many were ignored, and how many failed. In addition, the output includes a listing of which specific tests were ignored and failed. 479A good example of the breadth and details of these results can be found in the `examples` directory. 480Intentionally ignored and failing tests in this project generate corresponding entries in the summary report. 481 482If you're interested in other (prettier?) output formats, check into the [Ceedling][] build tool project that works with Unity and CMock and supports xunit-style xml as well as other goodies. 483 484This script assumes the existence of files ending with the extensions `.testpass` and `.testfail`. 485The contents of these files includes the test results summary corresponding to each test file executed with the extension set according to the presence or absence of failures for that test file. 486The script searches a specified path for these files, opens each one it finds, parses the results, and aggregates and prints a summary. 487Calling it from the command line looks like this: 488 489```Shell 490ruby unity_test_summary.rb build/test/ 491``` 492 493You can optionally specify a root path as well. 494This is really helpful when you are using relative paths in your tools' setup, but you want to pull the summary into an IDE like Eclipse for clickable shortcuts. 495 496```Shell 497ruby unity_test_summary.rb build/test/ ~/projects/myproject/ 498``` 499 500Or, if you're more of a Windows sort of person: 501 502```Shell 503ruby unity_test_summary.rb build\teat\ C:\projects\myproject\ 504``` 505 506When configured correctly, you'll see a final summary, like so: 507 508```Shell 509-------------------------- 510UNITY IGNORED TEST SUMMARY 511-------------------------- 512blah.c:22:test_sandwiches_should_HaveBreadOnTwoSides:IGNORE 513 514------------------------- 515UNITY FAILED TEST SUMMARY 516------------------------- 517blah.c:87:test_sandwiches_should_HaveCondiments:FAIL:Expected 1 was 0 518meh.c:38:test_soda_should_BeCalledPop:FAIL:Expected "pop" was "coke" 519 520-------------------------- 521OVERALL UNITY TEST SUMMARY 522-------------------------- 52345 TOTAL TESTS 2 TOTAL FAILURES 1 IGNORED 524``` 525 526How convenient is that? 527 528*Find The Latest of This And More at [ThrowTheSwitch.org][]* 529 530[ruby-lang.org]: https://ruby-lang.org/ 531[YAML]: http://www.yaml.org/ 532[Ceedling]: http://www.throwtheswitch.org/ceedling 533[ThrowTheSwitch.org]: https://throwtheswitch.org 534