1<a id="top"></a> 2# Test cases and sections 3 4**Contents**<br> 5[Tags](#tags)<br> 6[Tag aliases](#tag-aliases)<br> 7[BDD-style test cases](#bdd-style-test-cases)<br> 8[Type parametrised test cases](#type-parametrised-test-cases)<br> 9 10While Catch fully supports the traditional, xUnit, style of class-based fixtures containing test case methods this is not the preferred style. 11 12Instead Catch provides a powerful mechanism for nesting test case sections within a test case. For a more detailed discussion see the [tutorial](tutorial.md#test-cases-and-sections). 13 14Test cases and sections are very easy to use in practice: 15 16* **TEST_CASE(** _test name_ \[, _tags_ \] **)** 17* **SECTION(** _section name_ **)** 18 19_test name_ and _section name_ are free form, quoted, strings. The optional _tags_ argument is a quoted string containing one or more tags enclosed in square brackets. Tags are discussed below. Test names must be unique within the Catch executable. 20 21For examples see the [Tutorial](tutorial.md#top) 22 23## Tags 24 25Tags allow an arbitrary number of additional strings to be associated with a test case. Test cases can be selected (for running, or just for listing) by tag - or even by an expression that combines several tags. At their most basic level they provide a simple way to group several related tests together. 26 27As an example - given the following test cases: 28 29 TEST_CASE( "A", "[widget]" ) { /* ... */ } 30 TEST_CASE( "B", "[widget]" ) { /* ... */ } 31 TEST_CASE( "C", "[gadget]" ) { /* ... */ } 32 TEST_CASE( "D", "[widget][gadget]" ) { /* ... */ } 33 34The tag expression, ```"[widget]"``` selects A, B & D. ```"[gadget]"``` selects C & D. ```"[widget][gadget]"``` selects just D and ```"[widget],[gadget]"``` selects all four test cases. 35 36For more detail on command line selection see [the command line docs](command-line.md#specifying-which-tests-to-run) 37 38Tag names are not case sensitive and can contain any ASCII characters. This means that tags `[tag with spaces]` and `[I said "good day"]` are both allowed tags and can be filtered on. Escapes are not supported however and `[\]]` is not a valid tag. 39 40### Special Tags 41 42All tag names beginning with non-alphanumeric characters are reserved by Catch. Catch defines a number of "special" tags, which have meaning to the test runner itself. These special tags all begin with a symbol character. Following is a list of currently defined special tags and their meanings. 43 44* `[!hide]` or `[.]` - causes test cases to be skipped from the default list (i.e. when no test cases have been explicitly selected through tag expressions or name wildcards). The hide tag is often combined with another, user, tag (for example `[.][integration]` - so all integration tests are excluded from the default run but can be run by passing `[integration]` on the command line). As a short-cut you can combine these by simply prefixing your user tag with a `.` - e.g. `[.integration]`. Because the hide tag has evolved to have several forms, all forms are added as tags if you use one of them. 45 46* `[!throws]` - lets Catch know that this test is likely to throw an exception even if successful. This causes the test to be excluded when running with `-e` or `--nothrow`. 47 48* `[!mayfail]` - doesn't fail the test if any given assertion fails (but still reports it). This can be useful to flag a work-in-progress, or a known issue that you don't want to immediately fix but still want to track in your tests. 49 50* `[!shouldfail]` - like `[!mayfail]` but *fails* the test if it *passes*. This can be useful if you want to be notified of accidental, or third-party, fixes. 51 52* `[!nonportable]` - Indicates that behaviour may vary between platforms or compilers. 53 54* `[#<filename>]` - running with `-#` or `--filenames-as-tags` causes Catch to add the filename, prefixed with `#` (and with any extension stripped), as a tag to all contained tests, e.g. tests in testfile.cpp would all be tagged `[#testfile]`. 55 56* `[@<alias>]` - tag aliases all begin with `@` (see below). 57 58* `[!benchmark]` - this test case is actually a benchmark. This is an experimental feature, and currently has no documentation. If you want to try it out, look at `projects/SelfTest/Benchmark.tests.cpp` for details. 59 60## Tag aliases 61 62Between tag expressions and wildcarded test names (as well as combinations of the two) quite complex patterns can be constructed to direct which test cases are run. If a complex pattern is used often it is convenient to be able to create an alias for the expression. This can be done, in code, using the following form: 63 64 CATCH_REGISTER_TAG_ALIAS( <alias string>, <tag expression> ) 65 66Aliases must begin with the `@` character. An example of a tag alias is: 67 68 CATCH_REGISTER_TAG_ALIAS( "[@nhf]", "[failing]~[.]" ) 69 70Now when `[@nhf]` is used on the command line this matches all tests that are tagged `[failing]`, but which are not also hidden. 71 72## BDD-style test cases 73 74In addition to Catch's take on the classic style of test cases, Catch supports an alternative syntax that allow tests to be written as "executable specifications" (one of the early goals of [Behaviour Driven Development](http://dannorth.net/introducing-bdd/)). This set of macros map on to ```TEST_CASE```s and ```SECTION```s, with a little internal support to make them smoother to work with. 75 76* **SCENARIO(** _scenario name_ \[, _tags_ \] **)** 77 78This macro maps onto ```TEST_CASE``` and works in the same way, except that the test case name will be prefixed by "Scenario: " 79 80* **GIVEN(** _something_ **)** 81* **WHEN(** _something_ **)** 82* **THEN(** _something_ **)** 83 84These macros map onto ```SECTION```s except that the section names are the _something_s prefixed by "given: ", "when: " or "then: " respectively. 85 86* **AND_WHEN(** _something_ **)** 87* **AND_THEN(** _something_ **)** 88 89Similar to ```WHEN``` and ```THEN``` except that the prefixes start with "and ". These are used to chain ```WHEN```s and ```THEN```s together. 90 91When any of these macros are used the console reporter recognises them and formats the test case header such that the Givens, Whens and Thens are aligned to aid readability. 92 93Other than the additional prefixes and the formatting in the console reporter these macros behave exactly as ```TEST_CASE```s and ```SECTION```s. As such there is nothing enforcing the correct sequencing of these macros - that's up to the programmer! 94 95## Type parametrised test cases 96 97In addition to `TEST_CASE`s, Catch2 also supports test cases parametrised 98by types, in the form of `TEMPLATE_TEST_CASE` and 99`TEMPLATE_PRODUCT_TEST_CASE`. 100 101* **TEMPLATE_TEST_CASE(** _test name_ , _tags_, _type1_, _type2_, ..., _typen_ **)** 102 103_test name_ and _tag_ are exactly the same as they are in `TEST_CASE`, 104with the difference that the tag string must be provided (however, it 105can be empty). _type1_ through _typen_ is the list of types for which 106this test case should run, and, inside the test code, the current type 107is available as the `TestType` type. 108 109Because of limitations of the C++ preprocessor, if you want to specify 110a type with multiple template parameters, you need to enclose it in 111parentheses, e.g. `std::map<int, std::string>` needs to be passed as 112`(std::map<int, std::string>)`. 113 114Example: 115```cpp 116TEMPLATE_TEST_CASE( "vectors can be sized and resized", "[vector][template]", int, std::string, (std::tuple<int,float>) ) { 117 118 std::vector<TestType> v( 5 ); 119 120 REQUIRE( v.size() == 5 ); 121 REQUIRE( v.capacity() >= 5 ); 122 123 SECTION( "resizing bigger changes size and capacity" ) { 124 v.resize( 10 ); 125 126 REQUIRE( v.size() == 10 ); 127 REQUIRE( v.capacity() >= 10 ); 128 } 129 SECTION( "resizing smaller changes size but not capacity" ) { 130 v.resize( 0 ); 131 132 REQUIRE( v.size() == 0 ); 133 REQUIRE( v.capacity() >= 5 ); 134 135 SECTION( "We can use the 'swap trick' to reset the capacity" ) { 136 std::vector<TestType> empty; 137 empty.swap( v ); 138 139 REQUIRE( v.capacity() == 0 ); 140 } 141 } 142 SECTION( "reserving smaller does not change size or capacity" ) { 143 v.reserve( 0 ); 144 145 REQUIRE( v.size() == 5 ); 146 REQUIRE( v.capacity() >= 5 ); 147 } 148} 149``` 150 151* **TEMPLATE_PRODUCT_TEST_CASE(** _test name_ , _tags_, (_template-type1_, _template-type2_, ..., _template-typen_), (_template-arg1_, _template-arg2_, ..., _template-argm_) **)** 152 153_template-type1_ through _template-typen_ is list of template template 154types which should be combined with each of _template-arg1_ through 155 _template-argm_, resulting in _n * m_ test cases. Inside the test case, 156the resulting type is available under the name of `TestType`. 157 158To specify more than 1 type as a single _template-type_ or _template-arg_, 159you must enclose the types in an additional set of parentheses, e.g. 160`((int, float), (char, double))` specifies 2 template-args, each 161consisting of 2 concrete types (`int`, `float` and `char`, `double` 162respectively). You can also omit the outer set of parentheses if you 163specify only one type as the full set of either the _template-types_, 164or the _template-args_. 165 166 167Example: 168```cpp 169template< typename T> 170struct Foo { 171 size_t size() { 172 return 0; 173 } 174}; 175 176TEMPLATE_PRODUCT_TEST_CASE("A Template product test case", "[template][product]", (std::vector, Foo), (int, float)) { 177 TestType x; 178 REQUIRE(x.size() == 0); 179} 180``` 181 182You can also have different arities in the _template-arg_ packs: 183```cpp 184TEMPLATE_PRODUCT_TEST_CASE("Product with differing arities", "[template][product]", std::tuple, (int, (int, double), (int, double, float))) { 185 TestType x; 186 REQUIRE(std::tuple_size<TestType>::value >= 1); 187} 188``` 189 190_While there is an upper limit on the number of types you can specify 191in single `TEMPLATE_TEST_CASE` or `TEMPLATE_PRODUCT_TEST_CASE`, the limit 192is very high and should not be encountered in practice._ 193 194--- 195 196[Home](Readme.md#top) 197