1<html> 2<head> 3<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 4<title>Reported information</title> 5<link rel="stylesheet" href="../../boostbook.css" type="text/css"> 6<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> 7<link rel="home" href="../../index.html" title="Boost.Test"> 8<link rel="up" href="../testing_tools.html" title="Writing unit tests"> 9<link rel="prev" href="boost_test_universal_macro.html" title="BOOST_TEST: universal and general purpose assertions"> 10<link rel="next" href="extended_comparison.html" title="Extended comparisons support"> 11</head> 12<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> 13<table cellpadding="2" width="100%"><tr> 14<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../boost.png"></td> 15<td align="center"><a href="../../../../../../index.html">Home</a></td> 16<td align="center"><a href="../../../../../../libs/libraries.htm">Libraries</a></td> 17<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> 18<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> 19<td align="center"><a href="../../../../../../more/index.htm">More</a></td> 20</tr></table> 21<hr> 22<div class="spirit-nav"> 23<a accesskey="p" href="boost_test_universal_macro.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="extended_comparison.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a> 24</div> 25<div class="section"> 26<div class="titlepage"><div><div><h3 class="title"> 27<a name="boost_test.testing_tools.reports"></a><a class="link" href="reports.html" title="Reported information">Reported information</a> 28</h3></div></div></div> 29<h4> 30<a name="boost_test.testing_tools.reports.h0"></a> 31 <span class="phrase"><a name="boost_test.testing_tools.reports.failure_message_why"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.failure_message_why">Failure message, 32 why?</a> 33 </h4> 34<p> 35 When an assertion fails, a message is logged containing: 36 </p> 37<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 38<li class="listitem"> 39 the body of the statement that failed 40 </li> 41<li class="listitem"> 42 the name of the file and the line of the failed assertion 43 </li> 44<li class="listitem"> 45 the name of the test case containing this assertion 46 </li> 47</ul></div> 48<p> 49 The purpose of all these information is to isolate as quickly as possible 50 the test that failed from the others. The <span class="bold"><strong>feedback</strong></span> 51 that the execution of the test case provides is an important cue, for the 52 following reasons: 53 </p> 54<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 55<li class="listitem"> 56 within the scheme of a continuous build/test, the logs available from 57 the server contain this information, which points to a particular statement 58 in the code 59 </li> 60<li class="listitem"> 61 the <span class="bold"><strong>cost</strong></span> for reproducing an error is 62 induced by the following steps: 63 <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; "> 64<li class="listitem"> 65 identify the test module that failed in case there are many 66 </li> 67<li class="listitem"> 68 compile and run the test module to reproduce the error 69 </li> 70<li class="listitem"> 71 identify the line of the code that failed, 72 </li> 73<li class="listitem"> 74 fix the test directly if all the information is enough, or start 75 a debug session 76 </li> 77</ul></div> 78 </li> 79</ul></div> 80<p> 81 We can see from the scheme above that reproduction of an error is <span class="emphasis"><em>costly</em></span>, 82 since usually one tends to reproduce the error, which in turn induces at 83 least the compilation of the test module. Also, a hidden cost is the lookup 84 at the line of code that contains the failing statement, which triggers a 85 sequence of back and forth lookup between the log on one hand and the code 86 on the other hand. 87 </p> 88<p> 89 The information extracted from the logs suggests the following fact: 90 </p> 91<div class="tip"><table border="0" summary="Tip"> 92<tr> 93<td rowspan="2" align="center" valign="top" width="25"><img alt="[Tip]" src="../../../../../../doc/src/images/tip.png"></td> 94<th align="left">Tip</th> 95</tr> 96<tr><td align="left" valign="top"><p> 97 Richness of the information contained in the logs is a key for the rapid 98 understanding and the resolution of a failed statement 99 </p></td></tr> 100</table></div> 101<h4> 102<a name="boost_test.testing_tools.reports.h1"></a> 103 <span class="phrase"><a name="boost_test.testing_tools.reports.default_reporting"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.default_reporting">Default 104 reporting</a> 105 </h4> 106<p> 107 When an assertion fails, <a class="link" href="../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a> reports details and values 108 on the operands of <code class="computeroutput"><span class="identifier">statement</span></code> 109 that lead to the failure. 110 </p> 111<h6> 112<a name="boost_test.testing_tools.reports.h2"></a> 113 <span class="phrase"><a name="boost_test.testing_tools.reports.example_descr"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.example_descr">Example: 114 BOOST_TEST reporting</a> 115 </h6> 116<div class="informaltable"><table class="table"> 117<colgroup><col></colgroup> 118<thead><tr><th> 119 <p> 120 Code 121 </p> 122 </th></tr></thead> 123<tbody><tr><td> 124<pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="preprocessor">#define</span> <span class="identifier">BOOST_TEST_MODULE</span> <span class="identifier">boost_test_macro3</span> 125<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">included</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 126 127<span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span> <span class="identifier">test_op_reportings</span> <span class="special">)</span> 128<span class="special">{</span> 129 <span class="keyword">int</span> <span class="identifier">a</span> <span class="special">=</span> <span class="number">13</span><span class="special">,</span> <span class="identifier">b</span> <span class="special">=</span> <span class="number">12</span><span class="special">;</span> 130 <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special">==</span> <span class="identifier">b</span><span class="special">);</span> 131 <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special"><</span> <span class="identifier">b</span><span class="special">);</span> 132 <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special">-</span> <span class="number">1</span> <span class="special"><</span> <span class="identifier">b</span><span class="special">);</span> 133 <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">b</span> <span class="special">></span> <span class="identifier">a</span> <span class="special">-</span> <span class="number">1</span><span class="special">);</span> 134<span class="special">}</span> 135</pre> 136 </td></tr></tbody> 137</table></div> 138<div class="informaltable"><table class="table"> 139<colgroup><col></colgroup> 140<thead><tr><th> 141 <p> 142 Output 143 </p> 144 </th></tr></thead> 145<tbody><tr><td> 146<pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="special">></span> <span class="special">./</span><span class="identifier">boost_test_macro3</span> <span class="special">--</span><span class="identifier">log_level</span><span class="special">=</span><span class="identifier">all</span> 147<span class="identifier">Running</span> <span class="number">1</span> <span class="identifier">test</span> <span class="keyword">case</span><span class="special">...</span> 148<span class="identifier">Entering</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_macro3"</span> 149<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">12</span><span class="special">):</span> <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test_op_reportings"</span> 150<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">15</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_op_reportings"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">a</span> <span class="special">==</span> <span class="identifier">b</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="special">[</span><span class="number">13</span> <span class="special">!=</span> <span class="number">12</span><span class="special">]</span> 151<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">16</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_op_reportings"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">a</span> <span class="special"><</span> <span class="identifier">b</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="special">[</span><span class="number">13</span> <span class="special">>=</span> <span class="number">12</span><span class="special">]</span> 152<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">17</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_op_reportings"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">a</span> <span class="special">-</span> <span class="number">1</span> <span class="special"><</span> <span class="identifier">b</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="special">[</span><span class="number">13</span> <span class="special">-</span> <span class="number">1</span> <span class="special">>=</span> <span class="number">12</span><span class="special">]</span> 153<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">18</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_op_reportings"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">b</span> <span class="special">></span> <span class="identifier">a</span> <span class="special">-</span> <span class="number">1</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="special">[</span><span class="number">12</span> <span class="special"><=</span> <span class="number">12</span><span class="special">]</span> 154<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">12</span><span class="special">):</span> <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test_op_reportings"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">484u</span><span class="identifier">s</span> 155<span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_macro3"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">588u</span><span class="identifier">s</span> 156 157<span class="special">***</span> <span class="number">2</span> <span class="identifier">failures</span> <span class="identifier">are</span> <span class="identifier">detected</span> <span class="identifier">in</span> <span class="identifier">the</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_macro3"</span> 158</pre> 159 </td></tr></tbody> 160</table></div> 161<p> 162 In the above example, the values of the operands are reported for inspection, 163 which is more valuable as a copy of the full statement. However, we can observe 164 that they are not treated symmetrically: 165 </p> 166<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 167<li class="listitem"> 168 "<code class="computeroutput"><span class="identifier">a</span> <span class="special">-</span> 169 <span class="number">1</span> <span class="special"><</span> 170 <span class="identifier">b</span></code>" reports <code class="computeroutput"><span class="string">"13 - 1 >= 12"</span> <span class="identifier">failed</span></code> 171 </li> 172<li class="listitem"> 173 "<code class="computeroutput"><span class="identifier">b</span> <span class="special">></span> 174 <span class="identifier">a</span> <span class="special">-</span> 175 <span class="number">1</span></code>" reports <code class="computeroutput"><span class="string">"12 176 <= 12"</span> <span class="identifier">failed</span></code> 177 </li> 178</ul></div> 179<p> 180 More details on how the <span class="emphasis"><em>Unit Test Framework</em></span> parses the 181 statement are given in <a class="link" href="internal_details.html" title="BOOST_TEST: details on expressions">this</a> 182 section. 183 </p> 184<h4> 185<a name="boost_test.testing_tools.reports.h3"></a> 186 <span class="phrase"><a name="boost_test.testing_tools.reports.custom_messages"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.custom_messages">Custom 187 messages</a> 188 </h4> 189<p> 190 While perfectly exact and precise, the file name, test case name, line number 191 of a failed statement carry an information that is partial with regards to 192 the meaning of the failed statement. Sometimes these information are not 193 informative enough. The <code class="computeroutput"><span class="identifier">BOOST_TEST</span></code> 194 macro let you override the default message by the use of a second argument, 195 as shown on the following example. 196 </p> 197<h6> 198<a name="boost_test.testing_tools.reports.h4"></a> 199 <span class="phrase"><a name="boost_test.testing_tools.reports.example_descr0"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.example_descr0">Example: 200 BOOST_TEST optional failure message</a> 201 </h6> 202<div class="informaltable"><table class="table"> 203<colgroup><col></colgroup> 204<thead><tr><th> 205 <p> 206 Code 207 </p> 208 </th></tr></thead> 209<tbody><tr><td> 210<pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="preprocessor">#define</span> <span class="identifier">BOOST_TEST_MODULE</span> <span class="identifier">boost_test_message</span> 211<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">included</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 212 213<span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span> <span class="identifier">test_message</span> <span class="special">)</span> 214<span class="special">{</span> 215 <span class="keyword">const</span> <span class="keyword">int</span> <span class="identifier">a</span><span class="special">(</span><span class="number">1</span><span class="special">),</span> <span class="identifier">b</span><span class="special">(</span><span class="number">2</span><span class="special">);</span> 216 <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special">==</span> <span class="identifier">b</span><span class="special">,</span> <span class="string">"a should be equal to b: "</span> <span class="special"><<</span> <span class="identifier">a</span> <span class="special"><<</span> <span class="string">"!="</span> <span class="special"><<</span> <span class="identifier">b</span><span class="special">);</span> 217 <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special">!=</span> <span class="number">10</span><span class="special">,</span> <span class="string">"value of a="</span> <span class="special"><<</span> <span class="identifier">a</span><span class="special">);</span> 218<span class="special">}</span> 219</pre> 220 </td></tr></tbody> 221</table></div> 222<div class="informaltable"><table class="table"> 223<colgroup><col></colgroup> 224<thead><tr><th> 225 <p> 226 Output 227 </p> 228 </th></tr></thead> 229<tbody><tr><td> 230<pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="special">></span> <span class="special">./</span><span class="identifier">boost_test_message</span> <span class="special">--</span><span class="identifier">log_level</span><span class="special">=</span><span class="identifier">all</span> 231<span class="identifier">Running</span> <span class="number">1</span> <span class="identifier">test</span> <span class="keyword">case</span><span class="special">...</span> 232<span class="identifier">Entering</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_message"</span> 233<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">12</span><span class="special">):</span> <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test_message"</span> 234<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">15</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_message"</span><span class="special">:</span> <span class="identifier">a</span> <span class="identifier">should</span> <span class="identifier">be</span> <span class="identifier">equal</span> <span class="identifier">to</span> <span class="identifier">b</span><span class="special">:</span> <span class="number">1</span><span class="special">!=</span><span class="number">2</span> 235<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">16</span><span class="special">):</span> <span class="identifier">info</span><span class="special">:</span> <span class="identifier">check</span> <span class="char">'value of a=1'</span> <span class="identifier">has</span> <span class="identifier">passed</span> 236<span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">12</span><span class="special">):</span> <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test_message"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">219u</span><span class="identifier">s</span> 237<span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_message"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">318u</span><span class="identifier">s</span> 238 239<span class="special">***</span> <span class="number">1</span> <span class="identifier">failure</span> <span class="identifier">is</span> <span class="identifier">detected</span> <span class="identifier">in</span> <span class="identifier">the</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_message"</span> 240</pre> 241 </td></tr></tbody> 242</table></div> 243</div> 244<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> 245<td align="left"></td> 246<td align="right"><div class="copyright-footer">Copyright © 2001-2020 Boost.Test contributors<p> 247 Distributed under the Boost Software License, Version 1.0. (See accompanying 248 file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) 249 </p> 250</div></td> 251</tr></table> 252<hr> 253<div class="spirit-nav"> 254<a accesskey="p" href="boost_test_universal_macro.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="extended_comparison.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a> 255</div> 256</body> 257</html> 258
