1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> 2<html> 3<head> 4<title>List Parsers</title> 5<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> 6<link href="theme/style.css" rel="stylesheet" type="text/css"> 7</head> 8 9<body> 10<table width="100%" border="0" background="theme/bkd2.gif" cellspacing="2"> 11 <tr> 12 <td width="10"> <font size="6" face="Verdana, Arial, Helvetica, sans-serif"><b> </b></font></td> 13 <td width="85%"> <font size="6" face="Verdana, Arial, Helvetica, sans-serif"><b>List Parsers</b></font></td> 14 <td width="112"><a href="http://spirit.sf.net"><img src="theme/spirit.gif" width="112" height="48" align="right" border="0"></a></td> 15 </tr> 16</table> 17<br> 18<table border="0"> 19 <tr> 20 <td width="10"></td> 21 <td width="30"><a href="../index.html"><img src="theme/u_arr.gif" border="0"></a></td> 22 <td width="30"><a href="confix.html"><img src="theme/l_arr.gif" border="0"></a></td> 23 <td width="30"><a href="functor_parser.html"><img src="theme/r_arr.gif" border="0"></a></td> 24 </tr> 25</table> 26<p>List Parsers are generated by the special predefined parser generator object 27 <tt>list_p</tt>, which generates parsers recognizing list structures 28 of the type </p> 29<pre><span class=identifier> item </span><span class=special>>> </span><span class=special>*(</span><span class=identifier>delimiter </span><span class=special>>> </span><span class=identifier>item</span><span class=special>) </span><span class=special>>> </span><span class=special>!</span><span class=identifier>end</span></pre> 30<p>where <tt>item</tt> is an expression, delimiter is a delimiter and end is an 31 optional closing expression. As you can see, the <tt>list_p</tt> generated parser 32 does not recognize empty lists, i.e. the parser must find at least one item 33 in the input stream to return a successful match. If you wish to also match 34 an empty list, you can make your list_p optional with operator! An example where 35 this utility parser is helpful is parsing comma separated C/C++ strings, which 36 can be easily formulated as:</p> 37<pre><span class=special> </span><span class=identifier>rule</span><span class=special><> </span><span class=identifier>list_of_c_strings_rule 38 </span><span class=special>= </span><span class=identifier>list_p</span><span class=special>(</span><span class=identifier>confix_p</span><span class=special>(</span><span class=literal>'\"'</span><span class=special>, </span><span class=special>*</span><span class=identifier>c_escape_char_p</span><span class=special>, </span><span class=literal>'\"'</span><span class=special>), </span><span class=literal>','</span><span class=special>) 39 </span><span class=special>;</span></pre> 40<p>The <tt>confix_p</tt> and <tt>c_escape_char_p</tt> parser generators 41 are described <a href="confix.html">here</a> and <a href="escape_char_parser.html">here</a>.</p> 42<p>The <tt>list_p</tt> parser generator object can be used to generate the following 43 different types of List Parsers:</p> 44<table width="90%" border="0" align="center"> 45 <tr> 46 <td colspan="2" class="table_title"><b>List Parsers</b></td> 47 </tr> 48 <tr> 49 <td width="29%" height="27" class="table_cells"><b>list_p</b></td> 50 <td width="71%" class="table_cells"><p><tt>list_p</tt> used by itself parses 51 comma separated lists without special item formatting, i.e. everything 52 in between two commas is matched as an <tt>item</tt>, no <tt>end</tt> 53 of list token is matched</p></td> 54 </tr> 55 <tr> 56 <td height="27" class="table_cells"><strong>list_p(delimiter)</strong></td> 57 <td class="table_cells"><p>generates a list parser, which recognizes lists 58 with the given <tt>delimiter</tt> and matches everything in between them 59 as an <tt>item</tt>, no <tt>end</tt> of list token is matched</p></td> 60 </tr> 61 <tr> 62 <td height="27" class="table_cells"><strong>list_p(item, delimiter)</strong></td> 63 <td class="table_cells"><p>generates a list parser, which recognizes lists 64 with the given <tt>delimiter</tt> and matches items based on the given 65 item parser, no <tt>end</tt> of list token is matched</p></td> 66 </tr> 67 <tr> 68 <td height="27" class="table_cells"><strong>list_p(item, delimiter, end)</strong></td> 69 <td class="table_cells"><p>generates a list parser, which recognizes lists 70 with the given <tt>delimiter</tt> and matches items based on the given 71 <tt>item</tt> parser and additionally recognizes an optional <tt>end</tt> 72 expression</p></td> 73 </tr> 74</table> 75<p>All of the parameters to list_p can be single characters, strings 76 or, if more complex parsing logic is required, auxiliary parsers, each of which 77 is automatically converted to the corresponding parser type needed for successful 78 parsing.</p> 79<p>If the <tt>item</tt> parser is an <tt>action_parser_category</tt> type (parser 80 with an attached semantic action) we have to do something special. This happens, 81 if the user wrote something like:</p> 82<pre><span class=special> </span><span class=identifier>list_p</span><span class=special>(</span><span class=identifier>item</span><span class=special>[</span><span class=identifier>func</span><span class=special>], </span><span class=identifier>delim</span><span class=special>)</span></pre> 83<p> where <tt>item</tt> is the parser matching one item of the list sequence and 84 <tt>func</tt> is a functor to be called after matching one item. If we would 85 do nothing, the resulting code would parse the sequence as follows:</p> 86<pre><span class=special> </span><span class=special>(</span><span class=identifier>item</span><span class=special>[</span><span class=identifier>func</span><span class=special>] </span><span class=special>- </span><span class=identifier>delim</span><span class=special>) </span><span class=special>>> </span><span class=special>*(</span><span class=identifier>delim </span><span class=special>>> </span><span class=special>(</span><span class=identifier>item</span><span class=special>[</span><span class=identifier>func</span><span class=special>] </span><span class=special>- </span><span class=identifier>delim</span><span class=special>))</span></pre> 87<p> what in most cases is not what the user expects. (If this <u>is</u> what you've 88 expected, then please use one of the <tt>list_p</tt> generator 89 functions <tt>direct()</tt>, which will inhibit refactoring of the <tt>item</tt> 90 parser). To make the list parser behave as expected:</p> 91<pre><span class=special> </span><span class=special>(</span><span class=identifier>item </span><span class=special>- </span><span class=identifier>delim</span><span class=special>)[</span><span class=identifier>func</span><span class=special>] </span><span class=special>>> </span><span class=special>*(</span><span class=identifier>delim </span><span class=special>>> </span><span class=special>(</span><span class=identifier>item </span><span class=special>- </span><span class=identifier>delim</span><span class=special>)[</span><span class=identifier>func</span><span class=special>])</span></pre> 92<p> the actor attached to the item parser has to be re-attached to the <tt>(item 93 - delim)</tt> parser construct, which will make the resulting list parser 'do 94 the right thing'. This refactoring is done by the help of the <a href="refactoring.html">Refactoring 95 Parsers</a>. Additionally special care must be taken, if the item parser is 96 a <tt>unary_parser_category</tt> type parser as for instance:</p> 97<pre><span class=special> </span><span class=identifier>list_p</span><span class=special>(*</span><span class=identifier>anychar_p</span><span class=special>, </span><span class=literal>','</span><span class=special>)</span></pre> 98<p> which without any refactoring would result in </p> 99<pre><span class=special> </span><span class=special>(*</span><span class=identifier>anychar_p </span><span class=special>- </span><span class=identifier>ch_p</span><span class=special>(</span><span class=literal>','</span><span class=special>)) 100 </span><span class=special>>> </span><span class=special>*( </span><span class=identifier>ch_p</span><span class=special>(</span><span class=literal>','</span><span class=special>) </span><span class=special>>> </span><span class=special>(*</span><span class=identifier>anychar_p </span><span class=special>- </span><span class=identifier>ch_p</span><span class=special>(</span><span class=literal>','</span><span class=special>)) </span><span class=special>)</span></pre> 101<p> and will not give the expected result (the first <tt>*anychar_p</tt> will 102 eat up all the input up to the end of the input stream). So we have to refactor 103 this into:</p> 104<pre><span class=special> </span><span class=special>*(</span><span class=identifier>anychar_p </span><span class=special>- </span><span class=identifier>ch_p</span><span class=special>(</span><span class=literal>','</span><span class=special>)) 105 </span><span class=special>>> </span><span class=special>*( </span><span class=identifier>ch_p</span><span class=special>(</span><span class=literal>','</span><span class=special>) </span><span class=special>>> </span><span class=special>*(</span><span class=identifier>anychar_p </span><span class=special>- </span><span class=identifier>ch_p</span><span class=special>(</span><span class=literal>','</span><span class=special>)) </span><span class=special>)</span></pre> 106<p> what will give the correct result.</p> 107<p> The case, where the item parser is a combination of the two mentioned problems 108 (i.e. the item parser is a unary parser with an attached action), is handled 109 accordingly too:</p> 110<pre><span class=special> </span><span class=identifier>list_p</span><span class=special>((*</span><span class=identifier>anychar_p</span><span class=special>)[</span><span class=identifier>func</span><span class=special>], </span><span class=literal>','</span><span class=special>)</span></pre> 111<p> will be parsed as expected:</p> 112<pre><span class=special> </span><span class=special>(*(</span><span class=identifier>anychar_p </span><span class=special>- </span><span class=identifier>ch_p</span><span class=special>(</span><span class=literal>','</span><span class=special>)))[</span><span class=identifier>func</span><span class=special>] 113 </span><span class=special>>> </span><span class=special>*( </span><span class=identifier>ch_p</span><span class=special>(</span><span class=literal>','</span><span class=special>) </span><span class=special>>> </span><span class=special>(*(</span><span class=identifier>anychar_p </span><span class=special>- </span><span class=identifier>ch_p</span><span class=special>(</span><span class=literal>','</span><span class=special>)))[</span><span class=identifier>func</span><span class=special>] </span><span class=special>)</span></pre> 114<p>The required refactoring is implemented with the help of the <a href="refactoring.html">Refactoring 115 Parsers</a>.</p> 116<table width="90%" border="0" align="center"> 117 <tr> 118 <td colspan="2" class="table_title"><b>Summary of List Parser refactorings</b></td> 119 </tr> 120 <tr class="table_title"> 121 <td width="34%"><b>You write it as:</b></td> 122 <td width="66%"><code><font face="Verdana, Arial, Helvetica, sans-serif">It 123 is refactored to:</font></code></td> 124 </tr> 125 <tr> 126 <td width="34%" class="table_cells"><code><span class=identifier>list_p</span><span class=special>(</span><span class=identifier>item</span><span class=special>, 127 </span><span class=identifier>delimiter</span><span class=special>)</span></code></td> 128 <td width="66%" class="table_cells"> <code><span class=special> (</span><span class=identifier>item 129 </span><span class=special>- </span><span class=identifier>delimiter</span><span class=special>) 130 <br> 131 >> *(</span><span class=identifier>delimiter </span><span class=special> 132 >> (</span><span class=identifier>item </span><span class=special>- 133 </span><span class=identifier>delimiter</span><span class=special>))</span></code></td> 134 </tr> 135 <tr> 136 <td width="34%" class="table_cells"><code><span class=identifier>list_p</span><span class=special>(</span><span class=identifier>item</span><span class=special>[</span><span class=identifier>func</span><span class=special>], 137 </span><span class=identifier>delimiter</span><span class=special>)</span></code></td> 138 <td width="66%" class="table_cells"> <code><span class=special> (</span><span class=identifier>item 139 </span><span class=special> - </span><span class=identifier>delimiter</span><span class=special>)[</span><span class=identifier>func</span><span class=special>] 140 <br> 141 >> *(</span><span class=identifier>delimiter </span><span class=special>>> 142 (</span><span class=identifier>item </span><span class=special>- </span><span class=identifier>delimiter</span><span class=special>)[</span><span class=identifier>func</span><span class=special>])</span></code></td> 143 </tr> 144 <tr> 145 <td width="34%" class="table_cells"><code><span class=identifier>list_p</span><span class=special>(*</span><span class=identifier>item</span><span class=special>, 146 </span><span class=identifier>delimiter</span><span class=special>)</span></code></td> 147 <td width="66%" class="table_cells"> <code><span class=special>*(</span><span class=identifier>item 148 </span><span class=special>- </span><span class=identifier>delimiter</span><span class=special>) 149 <br> 150 >> *(</span><span class=identifier>delimiter </span><span class=special>>> 151 *(</span><span class=identifier>item </span><span class=special>- </span><span class=identifier>delimiter</span><span class=special>))</span></code></td> 152 </tr> 153 <tr> 154 <td width="34%" class="table_cells"><code><span class=identifier>list_p</span><span class=special>((*</span><span class=identifier>item</span><span class=special>)[</span><span class=identifier>func</span><span class=special>], 155 </span><span class=identifier>delimiter</span><span class=special>)</span></code></td> 156 <td width="66%" class="table_cells"> <code><span class=special>(*(</span><span class=identifier>item 157 </span><span class=special>- </span><span class=identifier>delimiter</span><span class=special>))[</span><span class=identifier>func</span><span class=special>] 158 <br> 159 >> *(</span><span class=identifier>delimiter </span><span class=special>>> 160 (*(</span><span class=identifier>item </span><span class=special>- </span><span class=identifier>delimiter</span><span class=special>))[</span><span class=identifier>func</span><span class=special>])</span></code></td> 161 </tr> 162</table> 163<p> <img height="16" width="15" src="theme/lens.gif"> <a href="../example/fundamental/list_parser.cpp">list_parser.cpp </a> sample shows the usage of the list_p utility parser:</p> 164<ol> 165 <li>parsing a simple ',' delimited list w/o item formatting</li> 166 <li> parsing a CSV list (comma separated values - strings, integers or reals)</li> 167 <li>parsing a token list (token separated values - strings, integers or reals) <br> 168 with an action parser directly attached to the item part of the list_p generated parser</li> 169</ol> 170<p>This is part of the Spirit distribution.</p> 171<table border="0"> 172 <tr> 173 <td width="10"></td> 174 <td width="30"><a href="../index.html"><img src="theme/u_arr.gif" border="0"></a></td> 175 <td width="30"><a href="confix.html"><img src="theme/l_arr.gif" border="0"></a></td> 176 <td width="30"><a href="functor_parser.html"><img src="theme/r_arr.gif" border="0"></a></td> 177 </tr> 178</table> 179<br> 180<hr size="1"> 181<p class="copyright">Copyright © 2001-2003 Hartmut Kaiser<br> 182 <br> 183 <font size="2">Use, modification and distribution is subject to the Boost Software 184 License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at 185 http://www.boost.org/LICENSE_1_0.txt) </font> </p> 186</body> 187</html> 188
