1<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> 2 3<html> 4<head> 5 <meta http-equiv="Content-Language" content="en-us"> 6 <meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> 7 <meta name="GENERATOR" content="Microsoft FrontPage 6.0"> 8 <meta name="ProgId" content="FrontPage.Editor.Document"> 9 10 <title>Boost Char Delimiters Separator</title> 11</head> 12 13<body bgcolor="#FFFFFF" text="#000000" link="#0000EE" vlink="#551A8B" alink= 14"#FF0000"> 15 <p><img src="../../../boost.png" alt="C++ Boost" width="277" height= 16 "86"><br></p><font color="red">Note: This class is deprecated. Please use 17 <a href="char_separator.htm"><tt>char_separator</tt></a> instead.</font> 18 19 <h1 align="center">Char Delimiters Separator</h1> 20 <pre> 21template <class Char, class Traits = std::char_traits<Char> > 22class char_delimiters_separator{ 23</pre> 24 25 <p>The char_delimiters_separator class is an implementation of the <a href= 26 "tokenizerfunction.htm">TokenizerFunction</a> concept that can be used to 27 break text up into tokens. It is the default TokenizerFunction for 28 tokenizer and token_iterator_generator. An example is below.</p> 29 30 <h2>Example</h2> 31 <pre> 32// simple_example_4.cpp 33#include<iostream> 34#include<boost/tokenizer.hpp> 35#include<string> 36 37int main(){ 38 using namespace std; 39 using namespace boost; 40 string s = "This is, a test"; 41 tokenizer<char_delimiters_separator<char> > tok(s); 42 for(tokenizer<char_delimiters_separator<char> >::iterator beg=tok.begin(); beg!=tok.end();++beg){ 43 cout << *beg << "\n"; 44 } 45} 46</pre> 47 48 <h2>Construction and Usage</h2> 49 50 <p>There is one constructor of interest. It is as follows</p> 51 <pre> 52explicit char_delimiters_separator(bool return_delims = false, 53const Char* returnable = "",const Char* nonreturnable = "" ) 54</pre> 55 56 <table border="1" summary=""> 57 <tr> 58 <td> 59 <p align="center"><strong>Parameter</strong></p> 60 </td> 61 62 <td> 63 <p align="center"><strong>Description</strong></p> 64 </td> 65 </tr> 66 67 <tr> 68 <td>return_delims</td> 69 70 <td>Whether or not to return the delimiters that have been found. Note 71 that not all delimiters can be returned. See the other two parameters 72 for explanation.</td> 73 </tr> 74 75 <tr> 76 <td>returnable</td> 77 78 <td>This specifies the returnable delimiters. These are the delimiters 79 that can be returned as tokens when return_delims is true. Since these 80 are typically punctuation, if a 0 is provided as the argument, then the 81 returnable delmiters will be all characters Cfor which std::ispunct(C) 82 yields a true value. If an argument of "" is provided, then this is 83 taken to mean that there are noreturnable delimiters.</td> 84 </tr> 85 86 <tr> 87 <td>nonreturnable</td> 88 89 <td>This specifies the nonreturnable delimiters. These are delimiters 90 that cannot be returned as tokens. Since these are typically 91 whitespace, if 0 is specified as an argument, then the nonreturnable 92 delimiters will be all characters C for which std::isspace(C) yields a 93 true value. If an argument of "" is provided, then this is taken to 94 mean that there are no non-returnable delimiters.</td> 95 </tr> 96 </table> 97 98 <p>The reason there is a distinction between nonreturnable and returnable 99 delimiters is that some delimiters are just used to split up tokens and are 100 nothing more. Take for example the following string "b c +". Assume you are 101 writing a simple calculator to parse expression in post fix notation. While 102 both the space and the + separate tokens, you only only interested in the + 103 and not in the space. Indeed having the space returned as a token would 104 only complicate your code. In this case you would specify + as a 105 returnable, and space as a nonreturnable delimiter.</p> 106 107 <p>To use this class, pass an object of it anywhere a TokenizerFunction 108 object is required.</p> 109 110 <h3>Template Parameters</h3> 111 112 <table border="1" summary=""> 113 <tr> 114 <th>Parameter</th> 115 116 <th>Description</th> 117 </tr> 118 119 <tr> 120 <td><tt>Char</tt></td> 121 122 <td>The type of the elements within a token, typically 123 <tt>char</tt>.</td> 124 </tr> 125 126 <tr> 127 <td>Traits</td> 128 129 <td>The traits class for Char, typically 130 std::char_traits<Char></td> 131 </tr> 132 </table> 133 134 <h2>Model of</h2> 135 136 <p><a href="tokenizerfunction.htm">TokenizerFunction</a></p> 137 138 <p> </p> 139 <hr> 140 141 <p><a href="http://validator.w3.org/check?uri=referer"><img border="0" src= 142 "../../doc/images/valid-html401.png" alt="Valid HTML 4.01 Transitional" 143 height="31" width="88"></a></p> 144 145 <p>Revised 146 <!--webbot bot="Timestamp" s-type="EDITED" s-format="%d %B, %Y" startspan -->25 147 December, 2006<!--webbot bot="Timestamp" endspan i-checksum="38518" --></p> 148 149 <p><i>Copyright © 2001 John R. Bandela</i></p> 150 151 <p><i>Distributed under the Boost Software License, Version 1.0. (See 152 accompanying file <a href="../../LICENSE_1_0.txt">LICENSE_1_0.txt</a> or 153 copy at <a href= 154 "http://www.boost.org/LICENSE_1_0.txt">http://www.boost.org/LICENSE_1_0.txt</a>)</i></p> 155</body> 156</html> 157
