1<html> 2<head> 3<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 4<title>A testing framework, what for?</title> 5<link rel="stylesheet" href="../../../boostbook.css" type="text/css"> 6<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> 7<link rel="home" href="../../../index.html" title="Boost.Test"> 8<link rel="up" href="../tutorials.html" title="Tutorials"> 9<link rel="prev" href="bt_and_tdd.html" title="Test driven development with Boost.Test"> 10<link rel="next" href="../web_wisdom.html" title="Web Wisdom"> 11</head> 12<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> 13<table cellpadding="2" width="100%"><tr> 14<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td> 15<td align="center"><a href="../../../../../../../index.html">Home</a></td> 16<td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td> 17<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> 18<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> 19<td align="center"><a href="../../../../../../../more/index.htm">More</a></td> 20</tr></table> 21<hr> 22<div class="spirit-nav"> 23<a accesskey="p" href="bt_and_tdd.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="../web_wisdom.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a> 24</div> 25<div class="section"> 26<div class="titlepage"><div><div><h4 class="title"> 27<a name="boost_test.practical_usage_recommendations.tutorials.hello"></a><a class="link" href="hello.html" title="A testing framework, what for?">A 28 testing framework, what for?</a> 29</h4></div></div></div> 30<p> 31 How should a test program report errors? Displaying an error message is 32 an obvious possibility: 33 </p> 34<pre class="programlisting"><span class="keyword">if</span><span class="special">(</span> <span class="identifier">something_bad_detected</span> <span class="special">)</span> 35 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"something bad has been detected"</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 36</pre> 37<p> 38 But that requires inspection of the program's output after each run to 39 determine if an error occurred. Since test programs are often run as part 40 of a regression test suite, human inspection of output to detect error 41 messages is time consuming and unreliable. Test frameworks like GNU/expect 42 can do the inspections automatically, but are overly complex for simple 43 testing. 44 </p> 45<p> 46 A better simple way to report errors is for the test program to return 47 <code class="computeroutput"><span class="identifier">EXIT_SUCCESS</span></code> (normally 48 0) if the test program completes satisfactorily, and <code class="computeroutput"><span class="identifier">EXIT_FAILURE</span></code> 49 if an error is detected. This allows a simple regression test script to 50 automatically and unambiguously detect success or failure. Further appropriate 51 actions such as creating an HTML table or emailing an alert can be taken 52 by the script, and can be modified as desired without having to change 53 the actual C++ test programs. 54 </p> 55<p> 56 A testing protocol based on a policy of test programs returning <code class="computeroutput"><span class="identifier">EXIT_SUCCESS</span></code> or <code class="computeroutput"><span class="identifier">EXIT_FAILURE</span></code> 57 does not require any supporting tools; the C++ language and standard library 58 are sufficient. The programmer must remember, however, to catch all exceptions 59 and convert them to program exits with non-zero return codes. The programmer 60 must also remember to not use the standard library <code class="computeroutput"><span class="identifier">assert</span><span class="special">()</span></code> macro for test code, because on some 61 systems it results in undesirable side effects like a message requiring 62 manual intervention. 63 </p> 64<p> 65 The Boost Test Library's Unit Test Framework is designed to automate those 66 tasks. The library supplied <code class="computeroutput"><span class="identifier">main</span><span class="special">()</span></code> relieves users from messy error detection 67 and reporting duties. Users could use supplied testing tools to perform 68 complex validation tasks. Let's take a look on the following simple test 69 program: 70 </p> 71<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">my_class</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 72 73<span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span> <span class="keyword">int</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="special">[]</span> <span class="special">)</span> 74<span class="special">{</span> 75 <span class="identifier">my_class</span> <span class="identifier">test_object</span><span class="special">(</span> <span class="string">"qwerty"</span> <span class="special">);</span> 76 <span class="keyword">return</span> <span class="identifier">test_object</span><span class="special">.</span><span class="identifier">is_valid</span><span class="special">()</span> <span class="special">?</span> <span class="identifier">EXIT_SUCCESS</span> <span class="special">:</span> <span class="identifier">EXIT_FAILURE</span><span class="special">;</span> 77<span class="special">}</span> 78</pre> 79<p> 80 There are several issues with above test. 81 </p> 82<div class="orderedlist"><ol class="orderedlist" type="1"> 83<li class="listitem"> 84 You need to convert <code class="computeroutput"><span class="identifier">is_valid</span></code> 85 result in proper result code. 86 </li> 87<li class="listitem"> 88 Would exception happen in test_object construction of method <code class="computeroutput"><span class="identifier">is_valid</span></code> invocation, the program 89 will crash. 90 </li> 91<li class="listitem"> 92 You won't see any output, would you run this test manually. 93 </li> 94</ol></div> 95<p> 96 The <span class="emphasis"><em>Unit Test Framework</em></span> solves all these issues. To 97 integrate with it above program needs to be changed to: 98 </p> 99<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">my_class</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 100<span class="preprocessor">#define</span> <a class="link" href="../../utf_reference/link_references/link_boost_test_module_macro.html" title="BOOST_TEST_MODULE"><code class="computeroutput"><span class="identifier">BOOST_TEST_MODULE</span></code></a> <span class="identifier">MyTest</span> 101<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 102 103<a class="link" href="../../utf_reference/test_org_reference/test_org_boost_auto_test_case.html" title="BOOST_AUTO_TEST_CASE"><code class="computeroutput"><span class="identifier">BOOST_AUTO_TEST_CASE</span></code></a><span class="special">(</span> <span class="identifier">my_test</span> <span class="special">)</span> 104<span class="special">{</span> 105 <span class="identifier">my_class</span> <span class="identifier">test_object</span><span class="special">(</span> <span class="string">"qwerty"</span> <span class="special">);</span> 106 <span class="identifier">BOOST_TEST</span><span class="special">(</span> <span class="identifier">test_object</span><span class="special">.</span><span class="identifier">is_valid</span><span class="special">()</span> <span class="special">);</span> 107<span class="special">}</span> 108</pre> 109<p> 110 Now, you not only receive uniform result code, even in case of exception, 111 but also nicely formatted output from <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a> tool, would you choose 112 to see it. Is there any other ways to perform checks? The following example 113 test program shows several different ways to detect and report an error 114 in the <code class="computeroutput"><span class="identifier">add</span><span class="special">()</span></code> 115 function. 116 </p> 117<pre class="programlisting"><span class="preprocessor">#define</span> <a class="link" href="../../utf_reference/link_references/link_boost_test_module_macro.html" title="BOOST_TEST_MODULE"><code class="computeroutput"><span class="identifier">BOOST_TEST_MODULE</span></code></a> <span class="identifier">MyTest</span> 118<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 119 120<span class="keyword">int</span> <span class="identifier">add</span><span class="special">(</span> <span class="keyword">int</span> <span class="identifier">i</span><span class="special">,</span> <span class="keyword">int</span> <span class="identifier">j</span> <span class="special">)</span> <span class="special">{</span> <span class="keyword">return</span> <span class="identifier">i</span> <span class="special">+</span> <span class="identifier">j</span><span class="special">;</span> <span class="special">}</span> 121 122<a class="link" href="../../utf_reference/test_org_reference/test_org_boost_auto_test_case.html" title="BOOST_AUTO_TEST_CASE"><code class="computeroutput"><span class="identifier">BOOST_AUTO_TEST_CASE</span></code></a><span class="special">(</span><span class="identifier">my_test</span><span class="special">)</span> 123<span class="special">{</span> 124 <span class="comment">// six ways to detect and report the same error:</span> 125 126 <span class="comment">// continues on error</span> 127 <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a><span class="special">(</span> <span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">==</span> <span class="number">4</span> <span class="special">);</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c0" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c1"><img src="../../../../../../../doc/src/images/callouts/1.png" alt="1" border="0"></a> 128 129 <span class="comment">// throws on error</span> 130 <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST_REQUIRE</span></code></a><span class="special">(</span> <span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">==</span> <span class="number">4</span> <span class="special">);</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c2" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c3"><img src="../../../../../../../doc/src/images/callouts/2.png" alt="2" border="0"></a> 131 132 <span class="comment">//continues on error</span> 133 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">!=</span> <span class="number">4</span><span class="special">)</span> 134 <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_error.html" title="BOOST_ERROR"><code class="computeroutput"><span class="identifier">BOOST_ERROR</span></code></a><span class="special">(</span> <span class="string">"Ouch..."</span> <span class="special">);</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c4" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c5"><img src="../../../../../../../doc/src/images/callouts/3.png" alt="3" border="0"></a> 135 136 <span class="comment">// throws on error</span> 137 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">!=</span> <span class="number">4</span><span class="special">)</span> 138 <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_fail.html" title="BOOST_FAIL"><code class="computeroutput"><span class="identifier">BOOST_FAIL</span></code></a><span class="special">(</span> <span class="string">"Ouch..."</span> <span class="special">);</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c6" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c7"><img src="../../../../../../../doc/src/images/callouts/4.png" alt="4" border="0"></a> 139 140 <span class="comment">// throws on error</span> 141 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">!=</span> <span class="number">4</span><span class="special">)</span> 142 <span class="keyword">throw</span> <span class="string">"Ouch..."</span><span class="special">;</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c8" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c9"><img src="../../../../../../../doc/src/images/callouts/5.png" alt="5" border="0"></a> 143 144 <span class="comment">// continues on error</span> 145 <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a><span class="special">(</span> <span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">==</span> <span class="number">4</span><span class="special">,</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c10" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c11"><img src="../../../../../../../doc/src/images/callouts/6.png" alt="6" border="0"></a> 146 <span class="string">"2 plus 2 is not 4 but "</span> <span class="special"><<</span> <span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">));</span> 147<span class="special">}</span> 148</pre> 149<div class="calloutlist"><table border="0" summary="Callout list"> 150<tr> 151<td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c1"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c0"><img src="../../../../../../../doc/src/images/callouts/1.png" alt="1" border="0"></a> </p></td> 152<td valign="top" align="left"><p> 153 This approach uses tool <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a>, which displays 154 an error message (by default on <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span></code>) 155 that includes the expression that failed, as well as the values on 156 the two side of the equation, the source file name, and the source 157 file line number. It also increments the error count. At program termination, 158 the error count will be displayed automatically by the <span class="emphasis"><em>Unit 159 Test Framework</em></span>. 160 </p></td> 161</tr> 162<tr> 163<td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c3"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c2"><img src="../../../../../../../doc/src/images/callouts/2.png" alt="2" border="0"></a> </p></td> 164<td valign="top" align="left"><p> 165 This approach uses tool <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST_REQUIRE</span></code></a>, is similar 166 to approach #1, except that after displaying the error, an exception 167 is thrown, to be caught by the <span class="emphasis"><em>Unit Test Framework</em></span>. 168 This approach is suitable when writing an explicit test program, and 169 the error would be so severe as to make further testing impractical. 170 </p></td> 171</tr> 172<tr> 173<td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c5"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c4"><img src="../../../../../../../doc/src/images/callouts/3.png" alt="3" border="0"></a> </p></td> 174<td valign="top" align="left"><p> 175 This approach is similar to approach #1, except that the error detection 176 and error reporting are coded separately. This is most useful when 177 the specific condition being tested requires several independent statements 178 and/or is not indicative of the reason for failure. 179 </p></td> 180</tr> 181<tr> 182<td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c7"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c6"><img src="../../../../../../../doc/src/images/callouts/4.png" alt="4" border="0"></a> </p></td> 183<td valign="top" align="left"><p> 184 This approach is similar to approach #2, except that the error detection 185 and error reporting are coded separately. This is most useful when 186 the specific condition being tested requires several independent statements 187 and/or is not indicative of the reason for failure. 188 </p></td> 189</tr> 190<tr> 191<td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c9"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c8"><img src="../../../../../../../doc/src/images/callouts/5.png" alt="5" border="0"></a> </p></td> 192<td valign="top" align="left"><p> 193 This approach throws an exception, which will be caught and reported 194 by the <span class="emphasis"><em>Unit Test Framework</em></span>. The error message 195 displayed when the exception is caught will be most meaningful if the 196 exception is derived from <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">exception</span></code>, 197 or is a <code class="computeroutput"><span class="keyword">char</span><span class="special">*</span></code> 198 or <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span></code>. 199 </p></td> 200</tr> 201<tr> 202<td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c11"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c10"><img src="../../../../../../../doc/src/images/callouts/6.png" alt="6" border="0"></a> </p></td> 203<td valign="top" align="left"><p> 204 This approach uses tool <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a> with additional 205 message argument, is similar to approach #1, except that similar to 206 the approach #3 displays an alternative error message specified as 207 a second argument. 208 </p></td> 209</tr> 210</table></div> 211</div> 212<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> 213<td align="left"></td> 214<td align="right"><div class="copyright-footer">Copyright © 2001-2020 Boost.Test contributors<p> 215 Distributed under the Boost Software License, Version 1.0. (See accompanying 216 file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) 217 </p> 218</div></td> 219</tr></table> 220<hr> 221<div class="spirit-nav"> 222<a accesskey="p" href="bt_and_tdd.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="../web_wisdom.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a> 223</div> 224</body> 225</html> 226
