1<html> 2<head> 3<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 4<title>Quickstart 1 - A word counter using Spirit.Lex</title> 5<link rel="stylesheet" href="../../../../../../../doc/src/boostbook.css" type="text/css"> 6<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> 7<link rel="home" href="../../../index.html" title="Spirit 2.5.8"> 8<link rel="up" href="../tutorials.html" title="Spirit.Lex Tutorials"> 9<link rel="prev" href="lexer_tutorials.html" title="Spirit.Lex Tutorials Overview"> 10<link rel="next" href="lexer_quickstart2.html" title="Quickstart 2 - A better word counter using Spirit.Lex"> 11</head> 12<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> 13<table cellpadding="2" width="100%"><tr> 14<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td> 15<td align="center"><a href="../../../../../../../index.html">Home</a></td> 16<td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td> 17<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> 18<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> 19<td align="center"><a href="../../../../../../../more/index.htm">More</a></td> 20</tr></table> 21<hr> 22<div class="spirit-nav"> 23<a accesskey="p" href="lexer_tutorials.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_quickstart2.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a> 24</div> 25<div class="section"> 26<div class="titlepage"><div><div><h4 class="title"> 27<a name="spirit.lex.tutorials.lexer_quickstart1"></a><a class="link" href="lexer_quickstart1.html" title="Quickstart 1 - A word counter using Spirit.Lex">Quickstart 28 1 - A word counter using <span class="emphasis"><em>Spirit.Lex</em></span></a> 29</h4></div></div></div> 30<p> 31 <span class="emphasis"><em>Spirit.Lex</em></span> is very modular, which follows the general 32 building principle of the <a href="http://boost-spirit.com" target="_top">Spirit</a> 33 libraries. You never pay for features you don't use. It is nicely integrated 34 with the other parts of <a href="http://boost-spirit.com" target="_top">Spirit</a> 35 but nevertheless can be used separately to build stand alone lexical analyzers. 36 The first quick start example describes a stand alone application: counting 37 characters, words, and lines in a file, very similar to what the well known 38 Unix command <code class="computeroutput"><span class="identifier">wc</span></code> is doing 39 (for the full example code see here: <a href="../../../../../example/lex/word_count_functor.cpp" target="_top">word_count_functor.cpp</a>). 40 </p> 41<h6> 42<a name="spirit.lex.tutorials.lexer_quickstart1.h0"></a> 43 <span class="phrase"><a name="spirit.lex.tutorials.lexer_quickstart1.prerequisites"></a></span><a class="link" href="lexer_quickstart1.html#spirit.lex.tutorials.lexer_quickstart1.prerequisites">Prerequisites</a> 44 </h6> 45<p> 46 The only required <code class="computeroutput"><span class="preprocessor">#include</span></code> 47 specific to <span class="emphasis"><em>Spirit.Lex</em></span> follows. It is a wrapper for 48 all necessary definitions to use <span class="emphasis"><em>Spirit.Lex</em></span> in a stand 49 alone fashion, and on top of the <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a> 50 library. Additionally we <code class="computeroutput"><span class="preprocessor">#include</span></code> 51 two of the Boost headers to define <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">bind</span><span class="special">()</span></code> and <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">ref</span><span class="special">()</span></code>. 52 </p> 53<p> 54</p> 55<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">spirit</span><span class="special">/</span><span class="identifier">include</span><span class="special">/</span><span class="identifier">lex_lexertl</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 56<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">bind</span><span class="special">/</span><span class="identifier">bind</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 57<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">ref</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 58</pre> 59<p> 60 </p> 61<p> 62 To make all the code below more readable we introduce the following namespaces. 63 </p> 64<p> 65</p> 66<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">lex</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">lex</span><span class="special">;</span> 67</pre> 68<p> 69 </p> 70<h6> 71<a name="spirit.lex.tutorials.lexer_quickstart1.h1"></a> 72 <span class="phrase"><a name="spirit.lex.tutorials.lexer_quickstart1.defining_tokens"></a></span><a class="link" href="lexer_quickstart1.html#spirit.lex.tutorials.lexer_quickstart1.defining_tokens">Defining 73 Tokens</a> 74 </h6> 75<p> 76 The most important step while creating a lexer using <span class="emphasis"><em>Spirit.Lex</em></span> 77 is to define the tokens to be recognized in the input sequence. This is 78 normally done by defining the regular expressions describing the matching 79 character sequences, and optionally their corresponding token ids. Additionally 80 the defined tokens need to be associated with an instance of a lexer object 81 as provided by the library. The following code snippet shows how this can 82 be done using <span class="emphasis"><em>Spirit.Lex</em></span>. 83 </p> 84<p> 85 The template <code class="computeroutput"><span class="identifier">word_count_tokens</span></code> 86 defines three different tokens: <code class="computeroutput"><span class="identifier">ID_WORD</span></code>, 87 <code class="computeroutput"><span class="identifier">ID_EOL</span></code>, and <code class="computeroutput"><span class="identifier">ID_CHAR</span></code>, representing a word (anything 88 except a whitespace or a newline), a newline character, and any other character 89 (<code class="computeroutput"><span class="identifier">ID_WORD</span></code>, <code class="computeroutput"><span class="identifier">ID_EOL</span></code>, and <code class="computeroutput"><span class="identifier">ID_CHAR</span></code> 90 are enum values representing the token ids, but could be anything else 91 convertible to an integer as well). The direct base class of any token 92 definition class needs to be the template <code class="computeroutput"><span class="identifier">lex</span><span class="special">::</span><span class="identifier">lexer</span><span class="special"><></span></code>, where the corresponding template 93 parameter (here: <code class="computeroutput"><span class="identifier">lex</span><span class="special">::</span><span class="identifier">lexertl</span><span class="special">::</span><span class="identifier">lexer</span><span class="special"><</span><span class="identifier">BaseIterator</span><span class="special">></span></code>) 94 defines which underlying lexer engine has to be used to provide the required 95 state machine functionality. In this example we use the Lexertl based lexer 96 engine as the underlying lexer type. 97 </p> 98<p> 99</p> 100<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Lexer</span><span class="special">></span> 101<span class="keyword">struct</span> <span class="identifier">word_count_tokens</span> <span class="special">:</span> <span class="identifier">lex</span><span class="special">::</span><span class="identifier">lexer</span><span class="special"><</span><span class="identifier">Lexer</span><span class="special">></span> 102<span class="special">{</span> 103 <span class="identifier">word_count_tokens</span><span class="special">()</span> 104 <span class="special">{</span> 105 <span class="comment">// define tokens (the regular expression to match and the corresponding</span> 106 <span class="comment">// token id) and add them to the lexer </span> 107 <span class="keyword">this</span><span class="special">-></span><span class="identifier">self</span><span class="special">.</span><span class="identifier">add</span> 108 <span class="special">(</span><span class="string">"[^ \t\n]+"</span><span class="special">,</span> <span class="identifier">ID_WORD</span><span class="special">)</span> <span class="comment">// words (anything except ' ', '\t' or '\n')</span> 109 <span class="special">(</span><span class="string">"\n"</span><span class="special">,</span> <span class="identifier">ID_EOL</span><span class="special">)</span> <span class="comment">// newline characters</span> 110 <span class="special">(</span><span class="string">"."</span><span class="special">,</span> <span class="identifier">ID_CHAR</span><span class="special">)</span> <span class="comment">// anything else is a plain character</span> 111 <span class="special">;</span> 112 <span class="special">}</span> 113<span class="special">};</span> 114</pre> 115<p> 116 </p> 117<h6> 118<a name="spirit.lex.tutorials.lexer_quickstart1.h2"></a> 119 <span class="phrase"><a name="spirit.lex.tutorials.lexer_quickstart1.doing_the_useful_work"></a></span><a class="link" href="lexer_quickstart1.html#spirit.lex.tutorials.lexer_quickstart1.doing_the_useful_work">Doing 120 the Useful Work</a> 121 </h6> 122<p> 123 We will use a setup, where we want the <span class="emphasis"><em>Spirit.Lex</em></span> 124 library to invoke a given function after any of the generated tokens is 125 recognized. For this reason we need to implement a functor taking at least 126 the generated token as an argument and returning a boolean value allowing 127 to stop the tokenization process. The default token type used in this example 128 carries a token value of the type <a href="../../../../../../../libs/range/doc/html/range/reference/utilities/iterator_range.html" target="_top"><code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">iterator_range</span></code></a><code class="computeroutput"><span class="special"><</span><span class="identifier">BaseIterator</span><span class="special">></span></code> 129 pointing to the matched range in the underlying input sequence. 130 </p> 131<p> 132 In this example the struct 'counter' is used as a functor counting the 133 characters, words and lines in the analyzed input sequence by identifying 134 the matched tokens as passed from the <span class="emphasis"><em>Spirit.Lex</em></span> library. 135 </p> 136<p> 137</p> 138<pre class="programlisting"><span class="keyword">struct</span> <span class="identifier">counter</span> 139<span class="special">{</span> 140 <span class="comment">// the function operator gets called for each of the matched tokens</span> 141 <span class="comment">// c, l, w are references to the counters used to keep track of the numbers</span> 142 <span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Token</span><span class="special">></span> 143 <span class="keyword">bool</span> <span class="keyword">operator</span><span class="special">()(</span><span class="identifier">Token</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">t</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span><span class="special">&</span> <span class="identifier">c</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span><span class="special">&</span> <span class="identifier">w</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span><span class="special">&</span> <span class="identifier">l</span><span class="special">)</span> <span class="keyword">const</span> 144 <span class="special">{</span> 145 <span class="keyword">switch</span> <span class="special">(</span><span class="identifier">t</span><span class="special">.</span><span class="identifier">id</span><span class="special">())</span> <span class="special">{</span> 146 <span class="keyword">case</span> <span class="identifier">ID_WORD</span><span class="special">:</span> <span class="comment">// matched a word</span> 147 <span class="comment">// since we're using a default token type in this example, every </span> 148 <span class="comment">// token instance contains a `iterator_range<BaseIterator>` as its token</span> 149 <span class="comment">// attribute pointing to the matched character sequence in the input </span> 150 <span class="special">++</span><span class="identifier">w</span><span class="special">;</span> <span class="identifier">c</span> <span class="special">+=</span> <span class="identifier">t</span><span class="special">.</span><span class="identifier">value</span><span class="special">().</span><span class="identifier">size</span><span class="special">();</span> 151 <span class="keyword">break</span><span class="special">;</span> 152 <span class="keyword">case</span> <span class="identifier">ID_EOL</span><span class="special">:</span> <span class="comment">// matched a newline character</span> 153 <span class="special">++</span><span class="identifier">l</span><span class="special">;</span> <span class="special">++</span><span class="identifier">c</span><span class="special">;</span> 154 <span class="keyword">break</span><span class="special">;</span> 155 <span class="keyword">case</span> <span class="identifier">ID_CHAR</span><span class="special">:</span> <span class="comment">// matched something else</span> 156 <span class="special">++</span><span class="identifier">c</span><span class="special">;</span> 157 <span class="keyword">break</span><span class="special">;</span> 158 <span class="special">}</span> 159 <span class="keyword">return</span> <span class="keyword">true</span><span class="special">;</span> <span class="comment">// always continue to tokenize</span> 160 <span class="special">}</span> 161<span class="special">};</span> 162</pre> 163<p> 164 </p> 165<p> 166 All what is left is to write some boilerplate code helping to tie together 167 the pieces described so far. To simplify this example we call the <code class="computeroutput"><span class="identifier">lex</span><span class="special">::</span><span class="identifier">tokenize</span><span class="special">()</span></code> 168 function implemented in <span class="emphasis"><em>Spirit.Lex</em></span> (for a more detailed 169 description of this function see here: <span class="bold"><strong>FIXME</strong></span>), 170 even if we could have written a loop to iterate over the lexer iterators 171 [<code class="computeroutput"><span class="identifier">first</span></code>, <code class="computeroutput"><span class="identifier">last</span></code>) 172 as well. 173 </p> 174<h6> 175<a name="spirit.lex.tutorials.lexer_quickstart1.h3"></a> 176 <span class="phrase"><a name="spirit.lex.tutorials.lexer_quickstart1.pulling_everything_together"></a></span><a class="link" href="lexer_quickstart1.html#spirit.lex.tutorials.lexer_quickstart1.pulling_everything_together">Pulling 177 Everything Together</a> 178 </h6> 179<p> 180 The main function simply loads the given file into memory (as a <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span></code>), instantiates an instance of 181 the token definition template using the correct iterator type (<code class="computeroutput"><span class="identifier">word_count_tokens</span><span class="special"><</span><span class="keyword">char</span> <span class="keyword">const</span><span class="special">*></span></code>), and finally calls <code class="computeroutput"><span class="identifier">lex</span><span class="special">::</span><span class="identifier">tokenize</span></code>, passing an instance of the 182 counter function object. The return value of <code class="computeroutput"><span class="identifier">lex</span><span class="special">::</span><span class="identifier">tokenize</span><span class="special">()</span></code> will be <code class="computeroutput"><span class="keyword">true</span></code> 183 if the whole input sequence has been successfully tokenized, and <code class="computeroutput"><span class="keyword">false</span></code> otherwise. 184 </p> 185<p> 186</p> 187<pre class="programlisting"><span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span> 188<span class="special">{</span> 189 <span class="comment">// these variables are used to count characters, words and lines</span> 190 <span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span> <span class="identifier">c</span> <span class="special">=</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">w</span> <span class="special">=</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">l</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> 191 192 <span class="comment">// read input from the given file</span> 193 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">str</span> <span class="special">(</span><span class="identifier">read_from_file</span><span class="special">(</span><span class="number">1</span> <span class="special">==</span> <span class="identifier">argc</span> <span class="special">?</span> <span class="string">"word_count.input"</span> <span class="special">:</span> <span class="identifier">argv</span><span class="special">[</span><span class="number">1</span><span class="special">]));</span> 194 195 <span class="comment">// create the token definition instance needed to invoke the lexical analyzer</span> 196 <span class="identifier">word_count_tokens</span><span class="special"><</span><span class="identifier">lex</span><span class="special">::</span><span class="identifier">lexertl</span><span class="special">::</span><span class="identifier">lexer</span><span class="special"><></span> <span class="special">></span> <span class="identifier">word_count_functor</span><span class="special">;</span> 197 198 <span class="comment">// tokenize the given string, the bound functor gets invoked for each of </span> 199 <span class="comment">// the matched tokens</span> 200 <span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">placeholders</span><span class="special">::</span><span class="identifier">_1</span><span class="special">;</span> 201 <span class="keyword">char</span> <span class="keyword">const</span><span class="special">*</span> <span class="identifier">first</span> <span class="special">=</span> <span class="identifier">str</span><span class="special">.</span><span class="identifier">c_str</span><span class="special">();</span> 202 <span class="keyword">char</span> <span class="keyword">const</span><span class="special">*</span> <span class="identifier">last</span> <span class="special">=</span> <span class="special">&</span><span class="identifier">first</span><span class="special">[</span><span class="identifier">str</span><span class="special">.</span><span class="identifier">size</span><span class="special">()];</span> 203 <span class="keyword">bool</span> <span class="identifier">r</span> <span class="special">=</span> <span class="identifier">lex</span><span class="special">::</span><span class="identifier">tokenize</span><span class="special">(</span><span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">,</span> <span class="identifier">word_count_functor</span><span class="special">,</span> 204 <span class="identifier">boost</span><span class="special">::</span><span class="identifier">bind</span><span class="special">(</span><span class="identifier">counter</span><span class="special">(),</span> <span class="identifier">_1</span><span class="special">,</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">ref</span><span class="special">(</span><span class="identifier">c</span><span class="special">),</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">ref</span><span class="special">(</span><span class="identifier">w</span><span class="special">),</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">ref</span><span class="special">(</span><span class="identifier">l</span><span class="special">)));</span> 205 206 <span class="comment">// print results</span> 207 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">r</span><span class="special">)</span> <span class="special">{</span> 208 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"lines: "</span> <span class="special"><<</span> <span class="identifier">l</span> <span class="special"><<</span> <span class="string">", words: "</span> <span class="special"><<</span> <span class="identifier">w</span> 209 <span class="special"><<</span> <span class="string">", characters: "</span> <span class="special"><<</span> <span class="identifier">c</span> <span class="special"><<</span> <span class="string">"\n"</span><span class="special">;</span> 210 <span class="special">}</span> 211 <span class="keyword">else</span> <span class="special">{</span> 212 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">rest</span><span class="special">(</span><span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">);</span> 213 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"Lexical analysis failed\n"</span> <span class="special"><<</span> <span class="string">"stopped at: \""</span> 214 <span class="special"><<</span> <span class="identifier">rest</span> <span class="special"><<</span> <span class="string">"\"\n"</span><span class="special">;</span> 215 <span class="special">}</span> 216 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 217<span class="special">}</span> 218</pre> 219<p> 220 </p> 221<h6> 222<a name="spirit.lex.tutorials.lexer_quickstart1.h4"></a> 223 <span class="phrase"><a name="spirit.lex.tutorials.lexer_quickstart1.comparing__emphasis_spirit_lex__emphasis__with__ulink_url__http___flex_sourceforge_net___flex__ulink_"></a></span><a class="link" href="lexer_quickstart1.html#spirit.lex.tutorials.lexer_quickstart1.comparing__emphasis_spirit_lex__emphasis__with__ulink_url__http___flex_sourceforge_net___flex__ulink_">Comparing 224 <span class="emphasis"><em>Spirit.Lex</em></span> with Flex</a> 225 </h6> 226<p> 227 This example was deliberately chosen to be as much as possible similar 228 to the equivalent <a href="http://flex.sourceforge.net/" target="_top">Flex</a> 229 program (see below), which isn't too different from what has to be written 230 when using <span class="emphasis"><em>Spirit.Lex</em></span>. 231 </p> 232<div class="note"><table border="0" summary="Note"> 233<tr> 234<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../images/note.png"></td> 235<th align="left">Note</th> 236</tr> 237<tr><td align="left" valign="top"><p> 238 Interestingly enough, performance comparisons of lexical analyzers written 239 using <span class="emphasis"><em>Spirit.Lex</em></span> with equivalent programs generated 240 by <a href="http://flex.sourceforge.net/" target="_top">Flex</a> show that both 241 have comparable execution speeds! Generally, thanks to the highly optimized 242 <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a> library 243 and due its carefully designed integration with <a href="http://boost-spirit.com" target="_top">Spirit</a> 244 the abstraction penalty to be paid for using <span class="emphasis"><em>Spirit.Lex</em></span> 245 is negligible. 246 </p></td></tr> 247</table></div> 248<p> 249 The remaining examples in this tutorial will use more sophisticated features 250 of <span class="emphasis"><em>Spirit.Lex</em></span>, mainly to allow further simplification 251 of the code to be written, while maintaining the similarity with corresponding 252 features of <a href="http://flex.sourceforge.net/" target="_top">Flex</a>. <span class="emphasis"><em>Spirit.Lex</em></span> 253 has been designed to be as similar to <a href="http://flex.sourceforge.net/" target="_top">Flex</a> 254 as possible. That is why this documentation will provide the corresponding 255 <a href="http://flex.sourceforge.net/" target="_top">Flex</a> code for the shown 256 <span class="emphasis"><em>Spirit.Lex</em></span> examples almost everywhere. So consequently, 257 here is the <a href="http://flex.sourceforge.net/" target="_top">Flex</a> code 258 corresponding to the example as shown above. 259 </p> 260<p> 261</p> 262<pre class="programlisting"><span class="special">%{</span> 263 <span class="preprocessor">#define</span> <span class="identifier">ID_WORD</span> <span class="number">1000</span> 264 <span class="preprocessor">#define</span> <span class="identifier">ID_EOL</span> <span class="number">1001</span> 265 <span class="preprocessor">#define</span> <span class="identifier">ID_CHAR</span> <span class="number">1002</span> 266 <span class="keyword">int</span> <span class="identifier">c</span> <span class="special">=</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">w</span> <span class="special">=</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">l</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> 267<span class="special">%}</span> 268<span class="special">%%</span> 269<span class="special">[^</span> <span class="special">\</span><span class="identifier">t</span><span class="special">\</span><span class="identifier">n</span><span class="special">]+</span> <span class="special">{</span> <span class="keyword">return</span> <span class="identifier">ID_WORD</span><span class="special">;</span> <span class="special">}</span> 270<span class="special">\</span><span class="identifier">n</span> <span class="special">{</span> <span class="keyword">return</span> <span class="identifier">ID_EOL</span><span class="special">;</span> <span class="special">}</span> 271<span class="special">.</span> <span class="special">{</span> <span class="keyword">return</span> <span class="identifier">ID_CHAR</span><span class="special">;</span> <span class="special">}</span> 272<span class="special">%%</span> 273<span class="keyword">bool</span> <span class="identifier">count</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">tok</span><span class="special">)</span> 274<span class="special">{</span> 275 <span class="keyword">switch</span> <span class="special">(</span><span class="identifier">tok</span><span class="special">)</span> <span class="special">{</span> 276 <span class="keyword">case</span> <span class="identifier">ID_WORD</span><span class="special">:</span> <span class="special">++</span><span class="identifier">w</span><span class="special">;</span> <span class="identifier">c</span> <span class="special">+=</span> <span class="identifier">yyleng</span><span class="special">;</span> <span class="keyword">break</span><span class="special">;</span> 277 <span class="keyword">case</span> <span class="identifier">ID_EOL</span><span class="special">:</span> <span class="special">++</span><span class="identifier">l</span><span class="special">;</span> <span class="special">++</span><span class="identifier">c</span><span class="special">;</span> <span class="keyword">break</span><span class="special">;</span> 278 <span class="keyword">case</span> <span class="identifier">ID_CHAR</span><span class="special">:</span> <span class="special">++</span><span class="identifier">c</span><span class="special">;</span> <span class="keyword">break</span><span class="special">;</span> 279 <span class="keyword">default</span><span class="special">:</span> 280 <span class="keyword">return</span> <span class="keyword">false</span><span class="special">;</span> 281 <span class="special">}</span> 282 <span class="keyword">return</span> <span class="keyword">true</span><span class="special">;</span> 283<span class="special">}</span> 284<span class="keyword">void</span> <span class="identifier">main</span><span class="special">()</span> 285<span class="special">{</span> 286 <span class="keyword">int</span> <span class="identifier">tok</span> <span class="special">=</span> <span class="identifier">EOF</span><span class="special">;</span> 287 <span class="keyword">do</span> <span class="special">{</span> 288 <span class="identifier">tok</span> <span class="special">=</span> <span class="identifier">yylex</span><span class="special">();</span> 289 <span class="keyword">if</span> <span class="special">(!</span><span class="identifier">count</span><span class="special">(</span><span class="identifier">tok</span><span class="special">))</span> 290 <span class="keyword">break</span><span class="special">;</span> 291 <span class="special">}</span> <span class="keyword">while</span> <span class="special">(</span><span class="identifier">EOF</span> <span class="special">!=</span> <span class="identifier">tok</span><span class="special">);</span> 292 <span class="identifier">printf</span><span class="special">(</span><span class="string">"%d %d %d\n"</span><span class="special">,</span> <span class="identifier">l</span><span class="special">,</span> <span class="identifier">w</span><span class="special">,</span> <span class="identifier">c</span><span class="special">);</span> 293<span class="special">}</span> 294</pre> 295<p> 296 </p> 297</div> 298<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> 299<td align="left"></td> 300<td align="right"><div class="copyright-footer">Copyright © 2001-2011 Joel de Guzman, Hartmut Kaiser<p> 301 Distributed under the Boost Software License, Version 1.0. (See accompanying 302 file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) 303 </p> 304</div></td> 305</tr></table> 306<hr> 307<div class="spirit-nav"> 308<a accesskey="p" href="lexer_tutorials.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_quickstart2.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a> 309</div> 310</body> 311</html> 312
