1<html> 2<head> 3<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 4<title>Annotations - Decorating the ASTs</title> 5<link rel="stylesheet" href="../../../../../../../doc/src/boostbook.css" type="text/css"> 6<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> 7<link rel="home" href="../../index.html" title="Spirit X3 3.0.4"> 8<link rel="up" href="../tutorials.html" title="Tutorials"> 9<link rel="prev" href="minimal.html" title="X3 Program Structure"> 10<link rel="next" href="rexpr.html" title="RExpressions - Recursive ASTs!"> 11</head> 12<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> 13<table cellpadding="2" width="100%"><tr> 14<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td> 15<td align="center"><a href="../../../../../../../index.html">Home</a></td> 16<td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td> 17<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> 18<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> 19<td align="center"><a href="../../../../../../../more/index.htm">More</a></td> 20</tr></table> 21<hr> 22<div class="spirit-nav"> 23<a accesskey="p" href="minimal.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="rexpr.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a> 24</div> 25<div class="section"> 26<div class="titlepage"><div><div><h3 class="title"> 27<a name="spirit_x3.tutorials.annotation"></a><a class="link" href="annotation.html" title="Annotations - Decorating the ASTs">Annotations - Decorating 28 the ASTs</a> 29</h3></div></div></div> 30<p> 31 As a prerequisite in understanding this tutorial, please review the previous 32 <a class="link" href="employee.html" title="Employee - Parsing into structs">employee example</a>. This 33 example builds on top of that example. 34 </p> 35<p> 36 Stop and think about it... We're actually generating ASTs (abstract syntax 37 trees) in our previoius examples. We parsed a single structure and generated 38 an in-memory representation of it in the form of a struct: the struct employee. 39 If we changed the implementation to parse one or more employees, the result 40 would be a std::vector<employee>. We can go on and add more hierarchy: 41 teams, departments, corporations, etc. We can have an AST representation 42 of it all. 43 </p> 44<p> 45 This example shows how to annotate the AST with the iterator positions for 46 access to the source code when post processing using a client supplied <code class="computeroutput"><span class="identifier">on_success</span></code> handler. The example will show 47 how to get the position in input source stream that corresponds to a given 48 element in the AST. 49 </p> 50<p> 51 In addition, This example also shows how to "inject" client data, 52 using the "with" directive, that the <code class="computeroutput"><span class="identifier">on_success</span></code> 53 handler can access as it is called within the parse traversal through the 54 parser's context. 55 </p> 56<p> 57 The full cpp file for this example can be found here: <a href="../../../../../example/x3/annotation.cpp" target="_top">annotation.cpp</a> 58 </p> 59<h5> 60<a name="spirit_x3.tutorials.annotation.h0"></a> 61 <span class="phrase"><a name="spirit_x3.tutorials.annotation.the_ast"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.the_ast">The 62 AST</a> 63 </h5> 64<p> 65 First, we'll update our previous employee struct, this time separating the 66 person into its own struct. So now, we have two structs, the <code class="computeroutput"><span class="identifier">person</span></code> and the <code class="computeroutput"><span class="identifier">employee</span></code>. 67 Take note too that we now inherit <code class="computeroutput"><span class="identifier">person</span></code> 68 and <code class="computeroutput"><span class="identifier">employee</span></code> from <code class="computeroutput"><span class="identifier">x3</span><span class="special">::</span><span class="identifier">position_tagged</span></code> 69 which provides positional information that we can use to tell the AST's position 70 in the input stream anytime. 71 </p> 72<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">client</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">ast</span> 73<span class="special">{</span> 74 <span class="keyword">struct</span> <span class="identifier">person</span> <span class="special">:</span> <span class="identifier">x3</span><span class="special">::</span><span class="identifier">position_tagged</span> 75 <span class="special">{</span> 76 <span class="identifier">person</span><span class="special">(</span> 77 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">first_name</span> <span class="special">=</span> <span class="string">""</span> 78 <span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">last_name</span> <span class="special">=</span> <span class="string">""</span> 79 <span class="special">)</span> 80 <span class="special">:</span> <span class="identifier">first_name</span><span class="special">(</span><span class="identifier">first_name</span><span class="special">)</span> 81 <span class="special">,</span> <span class="identifier">last_name</span><span class="special">(</span><span class="identifier">last_name</span><span class="special">)</span> 82 <span class="special">{}</span> 83 84 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">first_name</span><span class="special">,</span> <span class="identifier">last_name</span><span class="special">;</span> 85 <span class="special">};</span> 86 87 <span class="keyword">struct</span> <span class="identifier">employee</span> <span class="special">:</span> <span class="identifier">x3</span><span class="special">::</span><span class="identifier">position_tagged</span> 88 <span class="special">{</span> 89 <span class="keyword">int</span> <span class="identifier">age</span><span class="special">;</span> 90 <span class="identifier">person</span> <span class="identifier">who</span><span class="special">;</span> 91 <span class="keyword">double</span> <span class="identifier">salary</span><span class="special">;</span> 92 <span class="special">};</span> 93<span class="special">}}</span> 94</pre> 95<p> 96 Like before, we need to tell <a href="../../../../../../../libs/fusion/doc/html/index.html" target="_top">Boost.Fusion</a> 97 about our structs to make them first-class fusion citizens that the grammar 98 can utilize: 99 </p> 100<pre class="programlisting"><span class="identifier">BOOST_FUSION_ADAPT_STRUCT</span><span class="special">(</span><span class="identifier">client</span><span class="special">::</span><span class="identifier">ast</span><span class="special">::</span><span class="identifier">person</span><span class="special">,</span> 101 <span class="identifier">first_name</span><span class="special">,</span> <span class="identifier">last_name</span> 102<span class="special">)</span> 103 104<span class="identifier">BOOST_FUSION_ADAPT_STRUCT</span><span class="special">(</span><span class="identifier">client</span><span class="special">::</span><span class="identifier">ast</span><span class="special">::</span><span class="identifier">employee</span><span class="special">,</span> 105 <span class="identifier">age</span><span class="special">,</span> <span class="identifier">who</span><span class="special">,</span> <span class="identifier">salary</span> 106<span class="special">)</span> 107</pre> 108<h5> 109<a name="spirit_x3.tutorials.annotation.h1"></a> 110 <span class="phrase"><a name="spirit_x3.tutorials.annotation.x3__position_cache"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.x3__position_cache">x3::position_cache</a> 111 </h5> 112<p> 113 Before we proceed, let me introduce a helper class called the <code class="computeroutput"><span class="identifier">position_cache</span></code>. It is a simple class that 114 collects iterator ranges that point to where each element in the AST are 115 located in the input stream. Given an AST, you can query the position_cache 116 about AST's position. For example: 117 </p> 118<pre class="programlisting"><span class="keyword">auto</span> <span class="identifier">pos</span> <span class="special">=</span> <span class="identifier">positions</span><span class="special">.</span><span class="identifier">position_of</span><span class="special">(</span><span class="identifier">my_ast</span><span class="special">);</span> 119</pre> 120<p> 121 Where <code class="computeroutput"><span class="identifier">my_ast</span></code> is the AST, 122 <code class="computeroutput"><span class="identifier">positions</span></code> and is the <code class="computeroutput"><span class="identifier">position_cache</span></code>, <code class="computeroutput"><span class="identifier">position_of</span></code> 123 returns an iterator range that points to the start and end (<code class="computeroutput"><span class="identifier">pos</span><span class="special">.</span><span class="identifier">begin</span><span class="special">()</span></code> and <code class="computeroutput"><span class="identifier">pos</span><span class="special">.</span><span class="identifier">end</span><span class="special">()</span></code>) 124 positions where the AST was parsed from. <code class="computeroutput"><span class="identifier">positions</span><span class="special">.</span><span class="identifier">begin</span><span class="special">()</span></code> and <code class="computeroutput"><span class="identifier">positions</span><span class="special">.</span><span class="identifier">end</span><span class="special">()</span></code> 125 points to the start and end of the entire input stream. 126 </p> 127<h5> 128<a name="spirit_x3.tutorials.annotation.h2"></a> 129 <span class="phrase"><a name="spirit_x3.tutorials.annotation.on_success"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.on_success">on_success</a> 130 </h5> 131<p> 132 The <code class="computeroutput"><span class="identifier">on_success</span></code> gives you 133 everything you want from semantic actions without the visual clutter. Declarative 134 code can and should be free from imperative code. <code class="computeroutput"><span class="identifier">on_success</span></code> 135 as a concept and mechanism is an important departure from how things are 136 done in Spirit's previous version: Qi. 137 </p> 138<p> 139 As demonstrated in the previous <a class="link" href="employee.html" title="Employee - Parsing into structs">employee 140 example</a>, the preferred way to extract data from an input source is 141 by having the parser collect the data for us into C++ structs as it traverses 142 the input stream. Ideally, Spirit X3 grammars are fully attributed and declared 143 in such a way that you do not have to add any imperative code and there should 144 be no need for semantic actions at all. The parser simply works as declared 145 and you get your data back as a result. 146 </p> 147<p> 148 However, there are certain cases where there's no way to avoid introducing 149 imperative code. But semantic actions mess up our clean declarative grammars. 150 If we care to keep our code clean, <code class="computeroutput"><span class="identifier">on_success</span></code> 151 handlers are alternative callback hooks to client code that are executed 152 by the parser after a successful parse without polluting the grammar. Like 153 semantic actions, <code class="computeroutput"><span class="identifier">on_success</span></code> 154 handlers have access to the AST, the iterators, and context. But, unlike 155 semantic actions, <code class="computeroutput"><span class="identifier">on_success</span></code> 156 handlers are cleanly separated from the actual grammar. 157 </p> 158<h5> 159<a name="spirit_x3.tutorials.annotation.h3"></a> 160 <span class="phrase"><a name="spirit_x3.tutorials.annotation.annotation_handler"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.annotation_handler">Annotation 161 Handler</a> 162 </h5> 163<p> 164 As discussed, we annotate the AST with its position in the input stream with 165 our <code class="computeroutput"><span class="identifier">on_success</span></code> handler: 166 </p> 167<pre class="programlisting"><span class="comment">// tag used to get the position cache from the context</span> 168<span class="keyword">struct</span> <span class="identifier">position_cache_tag</span><span class="special">;</span> 169 170<span class="keyword">struct</span> <span class="identifier">annotate_position</span> 171<span class="special">{</span> 172 <span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">T</span><span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Iterator</span><span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Context</span><span class="special">></span> 173 <span class="keyword">inline</span> <span class="keyword">void</span> <span class="identifier">on_success</span><span class="special">(</span><span class="identifier">Iterator</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">first</span><span class="special">,</span> <span class="identifier">Iterator</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">last</span> 174 <span class="special">,</span> <span class="identifier">T</span><span class="special">&</span> <span class="identifier">ast</span><span class="special">,</span> <span class="identifier">Context</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">context</span><span class="special">)</span> 175 <span class="special">{</span> 176 <span class="keyword">auto</span><span class="special">&</span> <span class="identifier">position_cache</span> <span class="special">=</span> <span class="identifier">x3</span><span class="special">::</span><span class="identifier">get</span><span class="special"><</span><span class="identifier">position_cache_tag</span><span class="special">>(</span><span class="identifier">context</span><span class="special">).</span><span class="identifier">get</span><span class="special">();</span> 177 <span class="identifier">position_cache</span><span class="special">.</span><span class="identifier">annotate</span><span class="special">(</span><span class="identifier">ast</span><span class="special">,</span> <span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">);</span> 178 <span class="special">}</span> 179<span class="special">};</span> 180</pre> 181<p> 182 <code class="computeroutput"><span class="identifier">position_cache_tag</span></code> is a special 183 tag we will use to get a reference to the actual <code class="computeroutput"><span class="identifier">position_cache</span></code>, 184 client data that we will inject at very start, when we call parse. More on 185 that later. 186 </p> 187<p> 188 Our <code class="computeroutput"><span class="identifier">on_success</span></code> handler gets 189 a reference to the actual <code class="computeroutput"><span class="identifier">position_cache</span></code> 190 and calls its <code class="computeroutput"><span class="identifier">annotate</span></code> member 191 function, passing in the AST and the iterators. <code class="computeroutput"><span class="identifier">position_cache</span><span class="special">.</span><span class="identifier">annotate</span><span class="special">(</span><span class="identifier">ast</span><span class="special">,</span> 192 <span class="identifier">first</span><span class="special">,</span> 193 <span class="identifier">last</span><span class="special">)</span></code> 194 annotates the AST with information required by <code class="computeroutput"><span class="identifier">x3</span><span class="special">::</span><span class="identifier">position_tagged</span></code>. 195 </p> 196<h5> 197<a name="spirit_x3.tutorials.annotation.h4"></a> 198 <span class="phrase"><a name="spirit_x3.tutorials.annotation.the_parser"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.the_parser">The 199 Parser</a> 200 </h5> 201<p> 202 Now we'll write a parser for our employee. To simplify, inputs will be of 203 the form: 204 </p> 205<pre class="programlisting"><span class="special">{</span> <span class="identifier">age</span><span class="special">,</span> <span class="string">"forename"</span><span class="special">,</span> <span class="string">"surname"</span><span class="special">,</span> <span class="identifier">salary</span> <span class="special">}</span> 206</pre> 207<p> 208 <a name="__tutorial_annotated_employee_parser__"></a>Here we go: 209 </p> 210<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">parser</span> 211<span class="special">{</span> 212 <span class="keyword">using</span> <span class="identifier">x3</span><span class="special">::</span><span class="identifier">int_</span><span class="special">;</span> 213 <span class="keyword">using</span> <span class="identifier">x3</span><span class="special">::</span><span class="identifier">double_</span><span class="special">;</span> 214 <span class="keyword">using</span> <span class="identifier">x3</span><span class="special">::</span><span class="identifier">lexeme</span><span class="special">;</span> 215 <span class="keyword">using</span> <span class="identifier">ascii</span><span class="special">::</span><span class="identifier">char_</span><span class="special">;</span> 216 217 <span class="keyword">struct</span> <span class="identifier">quoted_string_class</span><span class="special">;</span> 218 <span class="keyword">struct</span> <span class="identifier">person_class</span><span class="special">;</span> 219 <span class="keyword">struct</span> <span class="identifier">employee_class</span><span class="special">;</span> 220 221 <span class="identifier">x3</span><span class="special">::</span><span class="identifier">rule</span><span class="special"><</span><span class="identifier">quoted_string_class</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">></span> <span class="keyword">const</span> <span class="identifier">quoted_string</span> <span class="special">=</span> <span class="string">"quoted_string"</span><span class="special">;</span> 222 <span class="identifier">x3</span><span class="special">::</span><span class="identifier">rule</span><span class="special"><</span><span class="identifier">person_class</span><span class="special">,</span> <span class="identifier">ast</span><span class="special">::</span><span class="identifier">person</span><span class="special">></span> <span class="keyword">const</span> <span class="identifier">person</span> <span class="special">=</span> <span class="string">"person"</span><span class="special">;</span> 223 <span class="identifier">x3</span><span class="special">::</span><span class="identifier">rule</span><span class="special"><</span><span class="identifier">employee_class</span><span class="special">,</span> <span class="identifier">ast</span><span class="special">::</span><span class="identifier">employee</span><span class="special">></span> <span class="keyword">const</span> <span class="identifier">employee</span> <span class="special">=</span> <span class="string">"employee"</span><span class="special">;</span> 224 225 <span class="keyword">auto</span> <span class="keyword">const</span> <span class="identifier">quoted_string_def</span> <span class="special">=</span> <span class="identifier">lexeme</span><span class="special">[</span><span class="char">'"'</span> <span class="special">>></span> <span class="special">+(</span><span class="identifier">char_</span> <span class="special">-</span> <span class="char">'"'</span><span class="special">)</span> <span class="special">>></span> <span class="char">'"'</span><span class="special">];</span> 226 <span class="keyword">auto</span> <span class="keyword">const</span> <span class="identifier">person_def</span> <span class="special">=</span> <span class="identifier">quoted_string</span> <span class="special">>></span> <span class="char">','</span> <span class="special">>></span> <span class="identifier">quoted_string</span><span class="special">;</span> 227 228 <span class="keyword">auto</span> <span class="keyword">const</span> <span class="identifier">employee_def</span> <span class="special">=</span> 229 <span class="char">'{'</span> 230 <span class="special">>></span> <span class="identifier">int_</span> <span class="special">>></span> <span class="char">','</span> 231 <span class="special">>></span> <span class="identifier">person</span> <span class="special">>></span> <span class="char">','</span> 232 <span class="special">>></span> <span class="identifier">double_</span> 233 <span class="special">>></span> <span class="char">'}'</span> 234 <span class="special">;</span> 235 236 <span class="keyword">auto</span> <span class="keyword">const</span> <span class="identifier">employees</span> <span class="special">=</span> <span class="identifier">employee</span> <span class="special">>></span> <span class="special">*(</span><span class="char">','</span> <span class="special">>></span> <span class="identifier">employee</span><span class="special">);</span> 237 238 <span class="identifier">BOOST_SPIRIT_DEFINE</span><span class="special">(</span><span class="identifier">quoted_string</span><span class="special">,</span> <span class="identifier">person</span><span class="special">,</span> <span class="identifier">employee</span><span class="special">);</span> 239<span class="special">}</span> 240</pre> 241<h5> 242<a name="spirit_x3.tutorials.annotation.h5"></a> 243 <span class="phrase"><a name="spirit_x3.tutorials.annotation.rule_declarations"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.rule_declarations">Rule 244 Declarations</a> 245 </h5> 246<pre class="programlisting"><span class="keyword">struct</span> <span class="identifier">quoted_string_class</span><span class="special">;</span> 247<span class="keyword">struct</span> <span class="identifier">person_class</span><span class="special">;</span> 248<span class="keyword">struct</span> <span class="identifier">employee_class</span><span class="special">;</span> 249 250<span class="identifier">x3</span><span class="special">::</span><span class="identifier">rule</span><span class="special"><</span><span class="identifier">quoted_string_class</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">></span> <span class="keyword">const</span> <span class="identifier">quoted_string</span> <span class="special">=</span> <span class="string">"quoted_string"</span><span class="special">;</span> 251<span class="identifier">x3</span><span class="special">::</span><span class="identifier">rule</span><span class="special"><</span><span class="identifier">person_class</span><span class="special">,</span> <span class="identifier">ast</span><span class="special">::</span><span class="identifier">person</span><span class="special">></span> <span class="keyword">const</span> <span class="identifier">person</span> <span class="special">=</span> <span class="string">"person"</span><span class="special">;</span> 252<span class="identifier">x3</span><span class="special">::</span><span class="identifier">rule</span><span class="special"><</span><span class="identifier">employee_class</span><span class="special">,</span> <span class="identifier">ast</span><span class="special">::</span><span class="identifier">employee</span><span class="special">></span> <span class="keyword">const</span> <span class="identifier">employee</span> <span class="special">=</span> <span class="string">"employee"</span><span class="special">;</span> 253</pre> 254<p> 255 Go back and review the original <a class="link" href="employee.html#__tutorial_employee_parser__">employee 256 parser</a>. What has changed? 257 </p> 258<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 259<li class="listitem"> 260 We split the single employee rule into three smaller rules: <code class="computeroutput"><span class="identifier">quoted_string</span></code>, <code class="computeroutput"><span class="identifier">person</span></code> 261 and <code class="computeroutput"><span class="identifier">employee</span></code>. 262 </li> 263<li class="listitem"> 264 We're using forward declared rule classes: <code class="computeroutput"><span class="identifier">quoted_string_class</span></code>, 265 <code class="computeroutput"><span class="identifier">person_class</span></code>, and <code class="computeroutput"><span class="identifier">employee_class</span></code>. 266 </li> 267</ul></div> 268<h5> 269<a name="spirit_x3.tutorials.annotation.h6"></a> 270 <span class="phrase"><a name="spirit_x3.tutorials.annotation.rule_classes"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.rule_classes">Rule 271 Classes</a> 272 </h5> 273<p> 274 Like before, in this example, the rule classes, <code class="computeroutput"><span class="identifier">quoted_string_class</span></code>, 275 <code class="computeroutput"><span class="identifier">person_class</span></code>, and <code class="computeroutput"><span class="identifier">employee_class</span></code> provide statically known 276 IDs for the rules required by X3 to perform its tasks. In addition to that, 277 the rule class can also be extended to have some user-defined customization 278 hooks that are called: 279 </p> 280<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 281<li class="listitem"> 282 On success: After a rule successfully parses an input. 283 </li> 284<li class="listitem"> 285 On Error: After a rule fails to parse. 286 </li> 287</ul></div> 288<p> 289 By subclassing the rule class from a client supplied handler such as our 290 <code class="computeroutput"><span class="identifier">annotate_position</span></code> handler 291 above: 292 </p> 293<pre class="programlisting"><span class="keyword">struct</span> <span class="identifier">person_class</span> <span class="special">:</span> <span class="identifier">annotate_position</span> <span class="special">{};</span> 294<span class="keyword">struct</span> <span class="identifier">employee_class</span> <span class="special">:</span> <span class="identifier">annotate_position</span> <span class="special">{};</span> 295</pre> 296<p> 297 The code above tells X3 to check the rule class if it has an <code class="computeroutput"><span class="identifier">on_success</span></code> or <code class="computeroutput"><span class="identifier">on_error</span></code> 298 member functions and appropriately calls them on such events. 299 </p> 300<a name="__tutorial_with_directive__"></a><h5> 301<a name="spirit_x3.tutorials.annotation.h7"></a> 302 <span class="phrase"><a name="spirit_x3.tutorials.annotation.the_with_directive"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.the_with_directive">The 303 with Directive</a> 304 </h5> 305<p> 306 For any parser <code class="computeroutput"><span class="identifier">p</span></code>, one can 307 inject supplementary data that semantic actions and handlers can access later 308 on when they are called. The general syntax is: 309 </p> 310<pre class="programlisting"><span class="identifier">with</span><span class="special"><</span><span class="identifier">tag</span><span class="special">>(</span><span class="identifier">data</span><span class="special">)[</span><span class="identifier">p</span><span class="special">]</span> 311</pre> 312<p> 313 For our particular example, we use to inject the <code class="computeroutput"><span class="identifier">position_cache</span></code> 314 into the parse for our <code class="computeroutput"><span class="identifier">annotate_position</span></code> 315 on_success handler to have access to: 316 </p> 317<pre class="programlisting"><span class="keyword">auto</span> <span class="keyword">const</span> <span class="identifier">parser</span> <span class="special">=</span> 318 <span class="comment">// we pass our position_cache to the parser so we can access</span> 319 <span class="comment">// it later in our on_sucess handlers</span> 320 <span class="identifier">with</span><span class="special"><</span><span class="identifier">position_cache_tag</span><span class="special">>(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">ref</span><span class="special">(</span><span class="identifier">positions</span><span class="special">))</span> 321 <span class="special">[</span> 322 <span class="identifier">employees</span> 323 <span class="special">];</span> 324</pre> 325<p> 326 Typically this is done just before calling <code class="computeroutput"><span class="identifier">x3</span><span class="special">::</span><span class="identifier">parse</span></code> 327 or <code class="computeroutput"><span class="identifier">x3</span><span class="special">::</span><span class="identifier">phrase_parse</span></code>. <code class="computeroutput"><span class="identifier">with</span></code> 328 is a very lightwight operation. It is possible to inject as much data as 329 you want, even multiple <code class="computeroutput"><span class="identifier">with</span></code> 330 directives: 331 </p> 332<pre class="programlisting"><span class="identifier">with</span><span class="special"><</span><span class="identifier">tag1</span><span class="special">>(</span><span class="identifier">data1</span><span class="special">)</span> 333<span class="special">[</span> 334 <span class="identifier">with</span><span class="special"><</span><span class="identifier">tag2</span><span class="special">>(</span><span class="identifier">data2</span><span class="special">)[</span><span class="identifier">p</span><span class="special">]</span> 335<span class="special">]</span> 336</pre> 337<p> 338 Multiple <code class="computeroutput"><span class="identifier">with</span></code> directives 339 can (perhaps not obviously) be injected from outside the called function. 340 Here's an outline: 341 </p> 342<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Parser</span><span class="special">></span> 343<span class="keyword">void</span> <span class="identifier">bar</span><span class="special">(</span><span class="identifier">Parser</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">p</span><span class="special">)</span> 344<span class="special">{</span> 345 <span class="comment">// Inject data2</span> 346 <span class="keyword">auto</span> <span class="keyword">const</span> <span class="identifier">parser</span> <span class="special">=</span> <span class="identifier">with</span><span class="special"><</span><span class="identifier">tag2</span><span class="special">>(</span><span class="identifier">data2</span><span class="special">)[</span><span class="identifier">p</span><span class="special">];</span> 347 <span class="identifier">x3</span><span class="special">::</span><span class="identifier">parse</span><span class="special">(</span><span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">,</span> <span class="identifier">parser</span><span class="special">);</span> 348<span class="special">}</span> 349 350<span class="keyword">void</span> <span class="identifier">foo</span><span class="special">()</span> 351<span class="special">{</span> 352 <span class="comment">// Inject data1</span> 353 <span class="keyword">auto</span> <span class="keyword">const</span> <span class="identifier">parser</span> <span class="special">=</span> <span class="identifier">with</span><span class="special"><</span><span class="identifier">tag1</span><span class="special">>(</span><span class="identifier">data1</span><span class="special">)[</span><span class="identifier">my_parser</span><span class="special">];</span> 354 <span class="identifier">bar</span><span class="special">(</span><span class="identifier">p</span><span class="special">);</span> 355<span class="special">}</span> 356</pre> 357<h5> 358<a name="spirit_x3.tutorials.annotation.h8"></a> 359 <span class="phrase"><a name="spirit_x3.tutorials.annotation.let_s_parse"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.let_s_parse">Let's 360 Parse</a> 361 </h5> 362<p> 363 Now we have the complete parse mechanism with support for annotations: 364 </p> 365<pre class="programlisting"><span class="keyword">using</span> <span class="identifier">iterator_type</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">::</span><span class="identifier">const_iterator</span><span class="special">;</span> 366<span class="keyword">using</span> <span class="identifier">position_cache</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">x3</span><span class="special">::</span><span class="identifier">position_cache</span><span class="special"><</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="identifier">iterator_type</span><span class="special">>>;</span> 367 368<span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="identifier">client</span><span class="special">::</span><span class="identifier">ast</span><span class="special">::</span><span class="identifier">employee</span><span class="special">></span> 369<span class="identifier">parse</span><span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">input</span><span class="special">,</span> <span class="identifier">position_cache</span><span class="special">&</span> <span class="identifier">positions</span><span class="special">)</span> 370<span class="special">{</span> 371 <span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">x3</span><span class="special">::</span><span class="identifier">ascii</span><span class="special">::</span><span class="identifier">space</span><span class="special">;</span> 372 373 <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="identifier">client</span><span class="special">::</span><span class="identifier">ast</span><span class="special">::</span><span class="identifier">employee</span><span class="special">></span> <span class="identifier">ast</span><span class="special">;</span> 374 <span class="identifier">iterator_type</span> <span class="identifier">iter</span> <span class="special">=</span> <span class="identifier">input</span><span class="special">.</span><span class="identifier">begin</span><span class="special">();</span> 375 <span class="identifier">iterator_type</span> <span class="keyword">const</span> <span class="identifier">end</span> <span class="special">=</span> <span class="identifier">input</span><span class="special">.</span><span class="identifier">end</span><span class="special">();</span> 376 377 <span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">x3</span><span class="special">::</span><span class="identifier">with</span><span class="special">;</span> 378 379 <span class="comment">// Our parser</span> 380 <span class="keyword">using</span> <span class="identifier">client</span><span class="special">::</span><span class="identifier">parser</span><span class="special">::</span><span class="identifier">employees</span><span class="special">;</span> 381 <span class="keyword">using</span> <span class="identifier">client</span><span class="special">::</span><span class="identifier">parser</span><span class="special">::</span><span class="identifier">position_cache_tag</span><span class="special">;</span> 382 383 <span class="keyword">auto</span> <span class="keyword">const</span> <span class="identifier">parser</span> <span class="special">=</span> 384 <span class="comment">// we pass our position_cache to the parser so we can access</span> 385 <span class="comment">// it later in our on_sucess handlers</span> 386 <span class="identifier">with</span><span class="special"><</span><span class="identifier">position_cache_tag</span><span class="special">>(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">ref</span><span class="special">(</span><span class="identifier">positions</span><span class="special">))</span> 387 <span class="special">[</span> 388 <span class="identifier">employees</span> 389 <span class="special">];</span> 390 391 <span class="keyword">bool</span> <span class="identifier">r</span> <span class="special">=</span> <span class="identifier">phrase_parse</span><span class="special">(</span><span class="identifier">iter</span><span class="special">,</span> <span class="identifier">end</span><span class="special">,</span> <span class="identifier">parser</span><span class="special">,</span> <span class="identifier">space</span><span class="special">,</span> <span class="identifier">ast</span><span class="special">);</span> 392 393 <span class="comment">// ... Some error checking here</span> 394 395 <span class="keyword">return</span> <span class="identifier">ast</span><span class="special">;</span> 396<span class="special">}</span> 397</pre> 398<p> 399 Let's walk through the code. 400 </p> 401<p> 402 First, we have some typedefs for 1) The iterator type we are using for the 403 parser, <code class="computeroutput"><span class="identifier">iterator_type</span></code> and 404 2) For the <code class="computeroutput"><span class="identifier">position_cache</span></code> 405 type. The latter is a template that accepts the type of container it will 406 hold. In this case, a <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="identifier">iterator_type</span><span class="special">></span></code>. 407 </p> 408<p> 409 The main parse function accepts an input, a std::string and a reference to 410 a position_cache, and returns an AST: <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="identifier">client</span><span class="special">::</span><span class="identifier">ast</span><span class="special">::</span><span class="identifier">employee</span><span class="special">></span></code>. 411 </p> 412<p> 413 Inside the parse function, we first create an AST where parsed data will 414 be stored: 415 </p> 416<pre class="programlisting"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="identifier">client</span><span class="special">::</span><span class="identifier">ast</span><span class="special">::</span><span class="identifier">employee</span><span class="special">></span> <span class="identifier">ast</span><span class="special">;</span> 417</pre> 418<p> 419 Then finally, we create a parser, injecting a reference to the <code class="computeroutput"><span class="identifier">position_cache</span></code>, and call phrase_parse: 420 </p> 421<pre class="programlisting"><span class="keyword">using</span> <span class="identifier">client</span><span class="special">::</span><span class="identifier">parser</span><span class="special">::</span><span class="identifier">employees</span><span class="special">;</span> 422<span class="keyword">using</span> <span class="identifier">client</span><span class="special">::</span><span class="identifier">parser</span><span class="special">::</span><span class="identifier">position_cache_tag</span><span class="special">;</span> 423 424<span class="keyword">auto</span> <span class="keyword">const</span> <span class="identifier">parser</span> <span class="special">=</span> 425 <span class="comment">// we pass our position_cache to the parser so we can access</span> 426 <span class="comment">// it later in our on_sucess handlers</span> 427 <span class="identifier">with</span><span class="special"><</span><span class="identifier">position_cache_tag</span><span class="special">>(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">ref</span><span class="special">(</span><span class="identifier">positions</span><span class="special">))</span> 428 <span class="special">[</span> 429 <span class="identifier">employees</span> 430 <span class="special">];</span> 431 432<span class="keyword">bool</span> <span class="identifier">r</span> <span class="special">=</span> <span class="identifier">phrase_parse</span><span class="special">(</span><span class="identifier">iter</span><span class="special">,</span> <span class="identifier">end</span><span class="special">,</span> <span class="identifier">parser</span><span class="special">,</span> <span class="identifier">space</span><span class="special">,</span> <span class="identifier">ast</span><span class="special">);</span> 433</pre> 434<p> 435 On successful parse, the AST, <code class="computeroutput"><span class="identifier">ast</span></code>, 436 will contain the actual parsed data. 437 </p> 438<h5> 439<a name="spirit_x3.tutorials.annotation.h9"></a> 440 <span class="phrase"><a name="spirit_x3.tutorials.annotation.getting_the_source_positions"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.getting_the_source_positions">Getting 441 The Source Positions</a> 442 </h5> 443<p> 444 Now that we have our main parse function, let's have an example sourcefile 445 to parse and show how we can obtain the position of an AST element, returned 446 after a successful parse. 447 </p> 448<p> 449 Given this input: 450 </p> 451<pre class="programlisting"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">input</span> <span class="special">=</span> <span class="identifier">R</span><span class="string">"( 452{ 453 23, 454 "</span><span class="identifier">Amanda</span><span class="string">", 455 "</span><span class="identifier">Stefanski</span><span class="string">", 456 1000.99 457}, 458{ 459 35, 460 "</span><span class="identifier">Angie</span><span class="string">", 461 "</span><span class="identifier">Chilcote</span><span class="string">", 462 2000.99 463}, 464{ 465 43, 466 "</span><span class="identifier">Dannie</span><span class="string">", 467 "</span><span class="identifier">Dillinger</span><span class="string">", 468 3000.99 469}, 470{ 471 22, 472 "</span><span class="identifier">Dorene</span><span class="string">", 473 "</span><span class="identifier">Dole</span><span class="string">", 474 2500.99 475}, 476{ 477 38, 478 "</span><span class="identifier">Rossana</span><span class="string">", 479 "</span><span class="identifier">Rafferty</span><span class="string">", 480 5000.99 481} 482)"</span><span class="special">;</span> 483</pre> 484<p> 485 We call our parse function after instantiating a <code class="computeroutput"><span class="identifier">position_cache</span></code> 486 object that will hold the source stream positions: 487 </p> 488<pre class="programlisting"><span class="identifier">position_cache</span> <span class="identifier">positions</span><span class="special">{</span><span class="identifier">input</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">input</span><span class="special">.</span><span class="identifier">end</span><span class="special">()};</span> 489<span class="keyword">auto</span> <span class="identifier">ast</span> <span class="special">=</span> <span class="identifier">parse</span><span class="special">(</span><span class="identifier">input</span><span class="special">,</span> <span class="identifier">positions</span><span class="special">);</span> 490</pre> 491<p> 492 We now have an AST, <code class="computeroutput"><span class="identifier">ast</span></code>, 493 that contains the parsed results. Let us get the source positions of the 494 2nd employee: 495 </p> 496<pre class="programlisting"><span class="keyword">auto</span> <span class="identifier">pos</span> <span class="special">=</span> <span class="identifier">positions</span><span class="special">.</span><span class="identifier">position_of</span><span class="special">(</span><span class="identifier">ast</span><span class="special">[</span><span class="number">1</span><span class="special">]);</span> <span class="comment">// zero based of course!</span> 497</pre> 498<p> 499 <code class="computeroutput"><span class="identifier">pos</span></code> is an iterator range 500 that contains iterators to the start and end of <code class="computeroutput"><span class="identifier">ast</span><span class="special">[</span><span class="number">1</span><span class="special">]</span></code> 501 in the input stream. 502 </p> 503<h5> 504<a name="spirit_x3.tutorials.annotation.h10"></a> 505 <span class="phrase"><a name="spirit_x3.tutorials.annotation.config"></a></span><a class="link" href="annotation.html#spirit_x3.tutorials.annotation.config">Config</a> 506 </h5> 507<p> 508 If you read the previous <a class="link" href="minimal.html" title="X3 Program Structure">Program 509 Structure</a> tutorial where we separated various logical modules of the 510 parser into separate cpp and header files, and you are wondering how to provide 511 the context configuration information (see <a class="link" href="minimal.html#tutorial_configuration">Config 512 Section</a>), we need to supplement the context like this: 513 </p> 514<pre class="programlisting"><span class="keyword">using</span> <span class="identifier">phrase_context_type</span> <span class="special">=</span> <span class="identifier">x3</span><span class="special">::</span><span class="identifier">phrase_parse_context</span><span class="special"><</span><span class="identifier">x3</span><span class="special">::</span><span class="identifier">ascii</span><span class="special">::</span><span class="identifier">space_type</span><span class="special">>::</span><span class="identifier">type</span><span class="special">;</span> 515 516<span class="keyword">typedef</span> <span class="identifier">x3</span><span class="special">::</span><span class="identifier">context</span><span class="special"><</span> 517 <span class="identifier">error_handler_tag</span> 518 <span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">reference_wrapper</span><span class="special"><</span><span class="identifier">position_cache</span><span class="special">></span> 519 <span class="special">,</span> <span class="identifier">phrase_context_type</span><span class="special">></span> 520<span class="identifier">context_type</span><span class="special">;</span> 521</pre> 522</div> 523<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> 524<td align="left"></td> 525<td align="right"><div class="copyright-footer">Copyright © 2001-2018 Joel de Guzman, 526 Hartmut Kaiser<p> 527 Distributed under the Boost Software License, Version 1.0. (See accompanying 528 file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) 529 </p> 530</div></td> 531</tr></table> 532<hr> 533<div class="spirit-nav"> 534<a accesskey="p" href="minimal.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="rexpr.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a> 535</div> 536</body> 537</html> 538
