1[/ 2 / Copyright (c) 2003 Boost.Test contributors 3 / 4 / Distributed under the Boost Software License, Version 1.0. (See accompanying 5 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) 6 /] 7 8[section Custom command line arguments] 9 10It is possible to pass custom command line arguments to the test module. The general format for passing custom 11arguments is the following: 12 13`` 14<boost_test_module> [<boost_test_arg1>...] [-- [<custom_parameter1>...] 15`` 16 17This means that everything that is passed after "`--`" is considered as a custom parameter and will not be intercepted nor interpreted 18by the __UTF__. This avoids any troubleshooting between the __UTF__ parameters and the custom ones. 19 20There are several use cases for accessing the arguments passed on the command line: 21 22* instantiating an object used in test cases and which is dependant on parameters external to the test-module: 23 the name of the graphic card, the credentials for a database connection, etc. The rest of the test module would check 24 that the functions in test are not sensitive to this type of parametrization. One can also imagine running this same 25 test module with different parameters (different graphic cards...) in a batched manner, 26* modifying the test tree by adding or parametrizing test cases: the arguments passed on the command line may contain 27 for instance a set of parameters that define test cases. 28 29In the first scenario, [link ref_consuming_cmd_test_case test cases] or fixtures, including 30[link ref_consuming_cmd_global_fixture global fixtures], may be used. Since those are part of the test tree, they can benefit from the __UTF__ rich set of assertions 31and controlled execution environment. 32 33In the second scenario, the command line argument interact directly with the content of the test tree: by passing specific 34arguments, different set of tests are created. There are mainly two options for achieving this: using a dedicated 35[link ref_consuming_cmd_init_function initialization function] or using [link ref_consuming_cmd_dataset data driven] test cases. 36The error handling of the command line parameters needs however to be adapted. 37 38[#ref_consuming_cmd_test_case][h4 Consuming custom arguments from a test case] 39The [link boost_test.tests_organization.test_tree.master_test_suite master test suite] collects the custom arguments 40passed to the test module in the following way: 41 42* `argv[0]`, usually set by the operating system as the executable name, remains unchanged 43* any argument interpreted by the test module is removed from `argv` 44* the empty token `--` is removed as well 45* any additional argument passed after the empty token is reported in `argv` starting at index `1` 46 47[bt_example runtime-configuration_1..Basic custom command line..run-fail] 48 49[#ref_consuming_cmd_global_fixture][h4 Consuming custom arguments from a global fixture] 50Another possibility for consuming the custom command line arguments would be from within a 51[link boost_test.tests_organization.fixtures.global global fixture]. This is especially useful 52when external parameters are needed for instantiating global objects used in the test module. 53 54The usage is the same as for test cases. The following example runs the test module twice with 55different arguments, and illustrate the feature. 56 57[tip The global fixture can check for the correctness of the custom arguments and may abort the full run 58 of the test module.] 59 60[bt_example runtime-configuration_2..Command line arguments interpreted in a global fixtures..run-fail] 61 62The above example instantiates a specific device through the `DeviceInterface::factory` member function. The 63name of the device to instantiate is passed via the command line argument `--device-name`, and the instantiated 64device is available through the global object `CommandLineDeviceInit::device`. 65The module requires `3` arguments on the command line: 66 67* `framework::master_test_suite().argv[0]` is the test module name as explained in the previous paragraph 68* `framework::master_test_suite().argv[1]` should be equal to `--device-name` 69* `framework::master_test_suite().argv[2]` should be the name of the device to instantiate 70 71As it can be seen in the shell outputs, any command line argument consumed by the __UTF__ is removed from 72`argc` / `argv`. Since global fixtures are running in the __UTF__ controlled environment, any fatal error reported 73by the fixture (through the __BOOST_TEST_REQUIRE__ assertion) aborts the test execution. Non fatal errors 74on the other hand do not abort the test-module and are reported as assertion failure, and would not prevent the execution 75of the test case `check_device_has_meaningful_name`. 76 77[note It is possible to have several global fixtures in a test module, spread over several compilation units. 78 Each of those fixture may in turn be accessing a specific part of the command line.] 79 80 81[#ref_consuming_cmd_init_function][h4 Parametrizing the test tree from command line in the initialization function] 82The initialization function are described in details in this [link boost_test.adv_scenarios.test_module_init_overview section]. 83The initialization function is called before any other test or fixture, and before entering the master test suite. The initialization 84function is not considered as a test-case, although it is called under the controlled execution 85environment of the __UTF__. This means that: 86 87* the errors will be properly handled, 88* loggers are not fully operational, 89* it is not possible to use the __UTF__ assertion macros like __BOOST_TEST__ as it is not a test-case. 90 91The following example shows how to use the command line arguments parsing described above to create/add new test cases 92to the test tree. It also shows very limited support to messages (does not work for all loggers), and error handling. 93 94[bt_example runtime-configuration_3..Init function parametrized from the command line..run-fail] 95 96As seen in this example, the error handling is quite different than a regular test-case: 97 98* For the /alternative/ initialization API (see 99 __BOOST_TEST_ALTERNATIVE_INIT_API__), the easiest way to indicate an error would be to return `false` 100 in case of failure. 101* For the /obsolete/ and /alternative/, raising an exception such as `std::runtime_error` or 102 [classref boost::unit_test::framework::setup_error] as above works as well. 103 104[#ref_consuming_cmd_dataset][h4 Data-driven test cases parametrized from the command line] 105It is possible to use the command line arguments to manipulate the dataset generated by a data-drive test case. 106 107By default, datasets are created before entering the `main` of the test module, and try to be efficient in the number 108of copies of their arguments. It is however possible 109to indicate a delay for the evaluation of the dataset by constructing the dataset with the `make_delayed` function. 110 111With the `make_delayed`, the construction of the dataset will happen at the same time as the construction of the 112test tree during the test module initialization, and not before. It is this way possible to access the 113[link boost_test.tests_organization.test_tree.master_test_suite master test suite] and its command line arguments. 114 115The example below shows a complex dataset generation from the content of an external file. The data contained 116in the file participates to the definition of the test case. 117 118[bt_example runtime-configuration_4..Dataset test case parametrized from the command line..run-fail] 119 120 121* Using `make_delayed`, the tests generated from a dataset are instantiated during the framework setup. This 122 let the dataset generator access the `argc` and `argv` of the master test suite. 123* The generation of the test-cases out of this dataset happens before the global fixture are reached (and before 124 any test cases), and after the initialization function. 125* The generator of the dataset is [*not] considered being a test case and the __UTF__ assertions are not accessible. 126 However, the __UTF__ will catch the exceptions raised during the generation of the test-cases by the dataset. 127 To report an error, a `std::logic_error` or [classref boost::unit_test::framework::setup_error] can be raised 128 and will be reported by the __UTF__. 129 130[/ 131[h4 Handling errors] 132The handling of errors that happen during the command line parsing has been discussed through the examples above. 133 134Some additional notes: 135 136* an exception occurring in a global fixture or the initialization function will be caught by the framework and will abort 137 the test module 138* a global fixture is attached to the [link boost_test.tests_organization.test_tree.master_test_suite master test suite], and 139 any failure there will be reported by the loggers properly. 140* A global fixture cannot manipulate the test tree, while the data-driven tests or custom initialization functions can. 141 142] 143 144[endsect] [/ Custom runtime parameters] 145