1[/ 2 / Copyright (c) 2003 Boost.Test contributors 3 / 4 / Distributed under the Boost Software License, Version 1.0. (See accompanying 5 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) 6 /] 7 8[section:log_formats Log formats] 9 10The __UTF__ supports several log formats: 11 12* [link boost_test.test_output.log_formats.log_human_readable_format HRF]: human readable format 13* [link boost_test.test_output.log_formats.log_xml_format XML]: an machine interpretable log format 14* [link boost_test.test_output.log_formats.log_junit_format JUNIT]: a standardized log format 15 understandable by automated tools such as Continuous Builds 16 17[h4 Design] 18 19The following functionalities are supported by the logging framework: 20 21* each logger manages its own log level. The rationale is that some log format are meant for 22 automated processing, and by design need to carry all the information that will later be digested 23 by a visualization tool. 24* several log format may be active at the same time. The rationale is that the user might want to 25 see a non-exhaustive log in his terminal using a human friendly format, while having a detailed 26 full log in a file with a format dedicated to automated processing. 27* each logger /indicates/ its default output stream. some logger may prefer to output to one of the standard 28 stream while other may prefer output to a file. 29 30[note The logger indicates the default output stream in case the default should be used. ] 31 32[h4 Defaults] 33By default the active log level threshold is set to 34 35* [link boost_test.test_output.log_formats.test_log_output "non fatal error messages"] and the test log output 36 is generated in [link boost_test.test_output.log_formats.log_human_readable_format human readable format]. 37* [link boost_test.test_output.log_formats.test_log_output "general information"] for 38 [link boost_test.test_output.log_formats.log_junit_format JUNIT] log format 39 40The active log level threshold and the output format can be configured 41at runtime during a test module invocation and at compile time from within a test module using the 42[link boost_test.test_output.logging_api test log public interfaces]. The behavior is logger specific though. 43 44 45[/ -------------------------------------------------------------------------------------------------- ] 46[section Test log output] 47 48The test log is produced during the test execution. All entries in the test log are assigned a particular log 49level. Only the entries with level that exceeds the ['active log level threshold] actually 50appear in the test log output. Log levels are arranged by the 'importance' of the log entries. Here is 51the list of all levels in order of increasing 'importance': 52 53[#test_log_output_table] 54[table:id_messages Messages 55 [ 56 [Notifications] 57 [Meaning] 58 ] 59 [ 60 [Success] 61 [This category includes messages that provide information on successfully passed assertions] 62 ] 63 [ 64 [Test tree traversal] 65 [This category includes messages that are produced by the __UTF__ core and indicate which test suites/cases are currently being executed or skipped] 66 ] 67 [ 68 [General information] 69 [This category includes general information messages produced in most cases by a test module author using the 70 macro __BOOST_TEST_MESSAGE__] 71 ] 72 [ 73 [Warning] 74 [This category includes messages produced by failed `WARNING` level assertions] 75 ] 76 [ 77 [Non fatal error] 78 [This category includes messages produced by failed `CHECK` level assertions] 79 ] 80 81 [ 82 [Uncaught C++ exceptions] 83 [This category includes messages that are produced by the __UTF__ and provide detailed information on the C++ 84 exceptions uncaught by the test case body. 85 ] 86 ] 87 [ 88 [Non-fatal system error] 89 [This category includes messages that are produced by the __UTF__ itself and provides information about caught 90 non-fatal system error. For example it includes messages produced in the case of test case timeout or if 91 floating point values calculation errors are caught. 92 ] 93 ] 94 [ 95 [Fatal system error] 96 [This category includes messages produced by failed require level assertions and by the __UTF__ itself in case of 97 abnormal test case termination.] 98 ] 99] 100 101[note 102 The active log level works namely as threshold, not as selector. For the given active log level threshold, all 103 test log entries with ['importance] higher than threshold are enabled and all test log entries with 104 ['importance] below threshold are disabled. 105] 106 107In addition to the levels described above the test log defines two special log levels. The current log level can 108be set to: 109 110* All messages[br] 111 If active log level threshold is set to this value, all test log entries appear in the output. In practice 112 this is equivalent to setting the active log level threshold to ['success information messages] 113* Nothing[br] 114 If the active log level threshold is set to this value, none of test log entries appear in the output. This log level 115 is used to execute a ['silent] test that doesn't produce any test log and only generates a result code indicating 116 whether test failed or passed. 117 118[endsect] [/ test log output] 119 120 121 122[section:log_human_readable_format HRF: Human readable log format] 123 124The human readable log format is designed to closely match an errors description produced by the Microsoft family 125of C++ compilers. This format allows jumping to the error location, if test module output is redirected into IDE 126output window. The rest of the log messages are designed to produce the most human friendly description of the 127events occurring in test module. This is a default format generated by test modules. 128 129Here the list of events along with corresponding message and the condition that has to be satisfied for it to appear 130in the output. 131 132[table 133 [ 134 [Event] 135 [Condition] 136 [Output] 137 ] 138 139 [ [On testing start] 140 [threshold != log_nothing] 141 [`Running <total number of test cases> test case(s) ...`] ] 142 143 [ [On testing start] 144 [threshold != log_nothing, show_build_info is set] 145 [ 146[pre Platform: $BOOST_PLATFORM 147Compiler: $BOOST_COMPILER 148STL : $BOOST_STDLIB 149Boost : $BOOST_VERSION]] ] 150 151 [ [On abnormal testing termination] 152 [threshold <= log_messages] 153 [`Test is aborted`] ] 154 155 [ [On test unit start] 156 [threshold <= log_test_units] 157 [`Entering test <test unit type> <test unit name>`] ] 158 159 [ [On test unit end] 160 [threshold <= log_test_units; testing time is reported only if elapsed time is more than 1 us.] 161 [`Leaving test <test unit type> <test unit name>; testing time <value>`] ] 162 163 [ [On skipped test unit] 164 [threshold <= log_test_units] 165 [`Test <test unit type> <test unit name> is skipped`] ] 166 167 [ [On uncaught C++ exception] 168 [threshold <= log_cpp_exception_errors. Checkpoint message is reported only if provided] 169 [`unknown location(0): fatal error in <test case name>: <explanation> <last checkpoint location>: last checkpoint: <checkpoint message>`] ] 170 171 [ [On resumable system error] 172 [threshold <= log_system_errors. Checkpoint message is reported only if provided] 173 [`unknown location(0): fatal error in <test case name>: <explanation> <last checkpoint location>: last checkpoint: <checkpoint message>`] ] 174 175 [ [On fatal system error] 176 [threshold <= log_fatal_errors. Checkpoint message is reported only if provided] 177 [`unknown location(0): fatal error in <test case name>: <explanation> <last checkpoint location>: last checkpoint: <checkpoint message>`] ] 178 179 [ [On passed test assertion] 180 [threshold <= log_successful_tests] 181 [`<assertion location>: info: check<assertion expression> passed`] ] 182 183 [ [On failed WARNING level test assertion] 184 [threshold <= log_warnings] 185 [`<assertion location>: warning in <test case name>: condition <assertion description> is not satisfied`]] 186 187 [ [On failed CHECK level test assertion] 188 [threshold <= log_all_errors] 189 [`<assertion location>: error in <test case name>: check <assertion description> failed`] ] 190 191 [ [On failed REQUIRE level test assertion] 192 [threshold <= log_fatal_errors] 193 [`<assertion location>: fatal error in <test case name>: critical check <assertion description> failed`]] 194 195 [ [On test log message] 196 [threshold <= log_messages] 197 [`<Message content>`]] 198] 199 200The level of details concerning the error message depends on the [link boost_test.testing_tools testing tool] producing the log entry. 201 202[endsect] [/ human readable report] 203 204[/ -------------------------------------------------------------------------------------------------- ] 205[section:log_xml_format XML log format] 206 207This log format is designed for automated test results processing. The test log output XML schema depends on the 208active log level threshold. 209 210[endsect] [/section:log_xml_format ] 211 212[/ -------------------------------------------------------------------------------------------------- ] 213[section:log_junit_format JUNIT log format] 214 215The [@http://junit.org/ JUNIT format] is log format supported by a wide range of Continuous Build/Integration tools. 216 217This format defaults its log level to [link test_log_output_table `General information`] and its default stream to a file named after 218[link boost_test.tests_organization.test_tree.master_test_suite master test suite]. 219The logger will attempt to not overwrite any existing output file, which is also usually understood by Continuous Build tools. 220 221This format is in fact both a log and a report format: most of the Continuous Build tools will summarize 222the content of a JUNIT file and show an overview of the failing/succeeding tests of a module (report format) 223while letting the user inspect the detailed logs (log format). 224 225[caution The minimal log-level for the JUnit logger is [link test_log_output_table `non fatal error`]. 226 Any set-up to higher log level will default to that minimal log-level.] 227 228[note Until Boost 1.64, the log level was previously defaulting to `success` and was causing a heavy load 229 on the logging part in some circumstances.] 230 231[endsect] [/section:log_junit_format ] 232 233 234[endsect] 235