1<html> 2<head> 3<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 4<title>Parser Directive Inhibiting Skipping (lexeme[])</title> 5<link rel="stylesheet" href="../../../../../../../../doc/src/boostbook.css" type="text/css"> 6<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> 7<link rel="home" href="../../../../index.html" title="Spirit 2.5.8"> 8<link rel="up" href="../directive.html" title="Parser Directives"> 9<link rel="prev" href="../directive.html" title="Parser Directives"> 10<link rel="next" href="no_skip.html" title="Parser Directive Inhibiting Skipping Without Pre-skip (no_skip[])"> 11</head> 12<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> 13<table cellpadding="2" width="100%"><tr> 14<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../../boost.png"></td> 15<td align="center"><a href="../../../../../../../../index.html">Home</a></td> 16<td align="center"><a href="../../../../../../../../libs/libraries.htm">Libraries</a></td> 17<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> 18<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> 19<td align="center"><a href="../../../../../../../../more/index.htm">More</a></td> 20</tr></table> 21<hr> 22<div class="spirit-nav"> 23<a accesskey="p" href="../directive.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../directive.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="no_skip.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a> 24</div> 25<div class="section"> 26<div class="titlepage"><div><div><h5 class="title"> 27<a name="spirit.qi.reference.directive.lexeme"></a><a class="link" href="lexeme.html" title="Parser Directive Inhibiting Skipping (lexeme[])">Parser Directive 28 Inhibiting Skipping (<code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[]</span></code>)</a> 29</h5></div></div></div> 30<h6> 31<a name="spirit.qi.reference.directive.lexeme.h0"></a> 32 <span class="phrase"><a name="spirit.qi.reference.directive.lexeme.description"></a></span><a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.description">Description</a> 33 </h6> 34<p> 35 The <code class="computeroutput"><span class="identifier">lexeme</span></code> directive 36 makes its subject a primitive. In a logical point of view, lexemes (and 37 primitives) are minimal atomic units (e.g. words, numbers, identifiers, 38 etc). These are the things that you'd normally put in the lexer (hinting 39 at the term "lexeme"), but in a lexer-less world, you put these 40 in a lexeme. Seeing its subject as a primitive, the <code class="computeroutput"><span class="identifier">lexeme</span></code> 41 directive does an initial pre-skip (as all primitives do) and turns off 42 white space skipping. 43 </p> 44<p> 45 At the phrase level, the parser ignores white spaces, possibly including 46 comments. Use <code class="computeroutput"><span class="identifier">lexeme</span></code> 47 in situations where you want to work at the character level instead of 48 the phrase level. Parsers can be made to work at the character level 49 by enclosing the pertinent parts inside the <code class="computeroutput"><span class="identifier">lexeme</span></code> 50 directive. For example, here's a rule that parses integers: 51 </p> 52<pre class="programlisting"><span class="identifier">integer</span> <span class="special">=</span> <span class="identifier">lexeme</span><span class="special">[</span> <span class="special">-(</span><span class="identifier">lit</span><span class="special">(</span><span class="char">'+'</span><span class="special">)</span> <span class="special">|</span> <span class="char">'-'</span><span class="special">)</span> <span class="special">>></span> <span class="special">+</span><span class="identifier">digit</span> <span class="special">];</span> 53</pre> 54<p> 55 The <code class="computeroutput"><span class="identifier">lexeme</span></code> directive 56 instructs its subject parser to work on the character level. Without 57 it, the <code class="computeroutput"><span class="identifier">integer</span></code> rule 58 would have allowed erroneous embedded white spaces in inputs such as 59 <code class="computeroutput"><span class="string">"1 2 345"</span></code> which 60 will be parsed as <code class="computeroutput"><span class="string">"12345"</span></code>. 61 </p> 62<div class="note"><table border="0" summary="Note"> 63<tr> 64<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../images/note.png"></td> 65<th align="left">Note</th> 66</tr> 67<tr><td align="left" valign="top"><p> 68 Keep in mind that <code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[]</span></code> pre-skips spaces. If this is not 69 desired, use the <a class="link" href="no_skip.html" title="Parser Directive Inhibiting Skipping Without Pre-skip (no_skip[])"><code class="computeroutput"><span class="identifier">no_skip</span></code></a> directive instead. 70 </p></td></tr> 71</table></div> 72<h6> 73<a name="spirit.qi.reference.directive.lexeme.h1"></a> 74 <span class="phrase"><a name="spirit.qi.reference.directive.lexeme.header"></a></span><a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.header">Header</a> 75 </h6> 76<pre class="programlisting"><span class="comment">// forwards to <boost/spirit/home/qi/directive/lexeme.hpp></span> 77<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">spirit</span><span class="special">/</span><span class="identifier">include</span><span class="special">/</span><span class="identifier">qi_lexeme</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 78</pre> 79<p> 80 Also, see <a class="link" href="../../../structure/include.html" title="Include">Include Structure</a>. 81 </p> 82<h6> 83<a name="spirit.qi.reference.directive.lexeme.h2"></a> 84 <span class="phrase"><a name="spirit.qi.reference.directive.lexeme.namespace"></a></span><a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.namespace">Namespace</a> 85 </h6> 86<div class="informaltable"><table class="table"> 87<colgroup><col></colgroup> 88<thead><tr><th> 89 <p> 90 Name 91 </p> 92 </th></tr></thead> 93<tbody><tr><td> 94 <p> 95 <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">lexeme</span> <span class="comment">// 96 alias: boost::spirit::qi::lexeme</span></code> 97 </p> 98 </td></tr></tbody> 99</table></div> 100<h6> 101<a name="spirit.qi.reference.directive.lexeme.h3"></a> 102 <span class="phrase"><a name="spirit.qi.reference.directive.lexeme.model_of"></a></span><a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.model_of">Model 103 of</a> 104 </h6> 105<div class="blockquote"><blockquote class="blockquote"><p> 106 <a class="link" href="../parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a> 107 </p></blockquote></div> 108<div class="variablelist"> 109<p class="title"><b>Notation</b></p> 110<dl class="variablelist"> 111<dt><span class="term"><code class="computeroutput"><span class="identifier">a</span></code></span></dt> 112<dd><p> 113 A <a class="link" href="../parser_concepts/parser.html" title="Parser"><code class="computeroutput"><span class="identifier">Parser</span></code></a>. 114 </p></dd> 115</dl> 116</div> 117<h6> 118<a name="spirit.qi.reference.directive.lexeme.h4"></a> 119 <span class="phrase"><a name="spirit.qi.reference.directive.lexeme.expression_semantics"></a></span><a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.expression_semantics">Expression 120 Semantics</a> 121 </h6> 122<p> 123 Semantics of an expression is defined only where it differs from, or 124 is not defined in <a class="link" href="../parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a>. 125 </p> 126<div class="informaltable"><table class="table"> 127<colgroup> 128<col> 129<col> 130</colgroup> 131<thead><tr> 132<th> 133 <p> 134 Expression 135 </p> 136 </th> 137<th> 138 <p> 139 Semantics 140 </p> 141 </th> 142</tr></thead> 143<tbody><tr> 144<td> 145 <p> 146 <code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[</span><span class="identifier">a</span><span class="special">]</span></code> 147 </p> 148 </td> 149<td> 150 <p> 151 Pre-skip and turn off white space skipping for the subject 152 parser, <code class="computeroutput"><span class="identifier">a</span></code> (and 153 all its children). 154 </p> 155 </td> 156</tr></tbody> 157</table></div> 158<h6> 159<a name="spirit.qi.reference.directive.lexeme.h5"></a> 160 <span class="phrase"><a name="spirit.qi.reference.directive.lexeme.attributes"></a></span><a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.attributes">Attributes</a> 161 </h6> 162<p> 163 See <a class="link" href="../../quick_reference/compound_attribute_rules.html#spirit.qi.quick_reference.compound_attribute_rules.notation">Compound 164 Attribute Notation</a>. 165 </p> 166<div class="informaltable"><table class="table"> 167<colgroup> 168<col> 169<col> 170</colgroup> 171<thead><tr> 172<th> 173 <p> 174 Expression 175 </p> 176 </th> 177<th> 178 <p> 179 Attribute 180 </p> 181 </th> 182</tr></thead> 183<tbody><tr> 184<td> 185 <p> 186 <code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[</span><span class="identifier">a</span><span class="special">]</span></code> 187 </p> 188 </td> 189<td> 190 <p> 191</p> 192<pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="identifier">a</span><span class="special">:</span> <span class="identifier">A</span> <span class="special">--></span> <span class="identifier">lexeme</span><span class="special">[</span><span class="identifier">a</span><span class="special">]:</span> <span class="identifier">A</span> 193<span class="identifier">a</span><span class="special">:</span> <span class="identifier">Unused</span> <span class="special">--></span> <span class="identifier">lexeme</span><span class="special">[</span><span class="identifier">a</span><span class="special">]:</span> <span class="identifier">Unused</span></pre> 194<p> 195 </p> 196 </td> 197</tr></tbody> 198</table></div> 199<h6> 200<a name="spirit.qi.reference.directive.lexeme.h6"></a> 201 <span class="phrase"><a name="spirit.qi.reference.directive.lexeme.complexity"></a></span><a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.complexity">Complexity</a> 202 </h6> 203<div class="blockquote"><blockquote class="blockquote"><p> 204 The complexity is defined by the complexity of the subject parser, 205 <code class="computeroutput"><span class="identifier">a</span></code> 206 </p></blockquote></div> 207<h6> 208<a name="spirit.qi.reference.directive.lexeme.h7"></a> 209 <span class="phrase"><a name="spirit.qi.reference.directive.lexeme.example"></a></span><a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.example">Example</a> 210 </h6> 211<div class="note"><table border="0" summary="Note"> 212<tr> 213<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../images/note.png"></td> 214<th align="left">Note</th> 215</tr> 216<tr><td align="left" valign="top"><p> 217 The test harness for the example(s) below is presented in the <a class="link" href="../basics.html#spirit.qi.reference.basics.examples">Basics Examples</a> 218 section. 219 </p></td></tr> 220</table></div> 221<p> 222 Some using declarations: 223 </p> 224<p> 225</p> 226<pre class="programlisting"><span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">qi</span><span class="special">::</span><span class="identifier">lexeme</span><span class="special">;</span> 227<span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">qi</span><span class="special">::</span><span class="identifier">lit</span><span class="special">;</span> 228<span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">ascii</span><span class="special">::</span><span class="identifier">digit</span><span class="special">;</span> 229</pre> 230<p> 231 </p> 232<p> 233 Simple usage of <code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[]</span></code>: 234 </p> 235<p> 236 The use of lexeme here will prevent skipping in between the digits and 237 the sign making inputs such as <code class="computeroutput"><span class="string">"1 2 238 345"</span></code> erroneous. 239 </p> 240<p> 241</p> 242<pre class="programlisting"><span class="identifier">test_phrase_parser</span><span class="special">(</span><span class="string">"12345"</span><span class="special">,</span> <span class="identifier">lexeme</span><span class="special">[</span> <span class="special">-(</span><span class="identifier">lit</span><span class="special">(</span><span class="char">'+'</span><span class="special">)</span> <span class="special">|</span> <span class="char">'-'</span><span class="special">)</span> <span class="special">>></span> <span class="special">+</span><span class="identifier">digit</span> <span class="special">]);</span> 243</pre> 244<p> 245 </p> 246</div> 247<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> 248<td align="left"></td> 249<td align="right"><div class="copyright-footer">Copyright © 2001-2011 Joel de Guzman, Hartmut Kaiser<p> 250 Distributed under the Boost Software License, Version 1.0. (See accompanying 251 file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) 252 </p> 253</div></td> 254</tr></table> 255<hr> 256<div class="spirit-nav"> 257<a accesskey="p" href="../directive.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../directive.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="no_skip.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a> 258</div> 259</body> 260</html> 261
