1<html> 2<head> 3<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 4<title>Spirit.Lex Tutorials Overview</title> 5<link rel="stylesheet" href="../../../../../../../doc/src/boostbook.css" type="text/css"> 6<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> 7<link rel="home" href="../../../index.html" title="Spirit 2.5.8"> 8<link rel="up" href="../tutorials.html" title="Spirit.Lex Tutorials"> 9<link rel="prev" href="../tutorials.html" title="Spirit.Lex Tutorials"> 10<link rel="next" href="lexer_quickstart1.html" title="Quickstart 1 - A word counter using Spirit.Lex"> 11</head> 12<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> 13<table cellpadding="2" width="100%"><tr> 14<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td> 15<td align="center"><a href="../../../../../../../index.html">Home</a></td> 16<td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td> 17<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> 18<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> 19<td align="center"><a href="../../../../../../../more/index.htm">More</a></td> 20</tr></table> 21<hr> 22<div class="spirit-nav"> 23<a accesskey="p" href="../tutorials.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_quickstart1.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a> 24</div> 25<div class="section"> 26<div class="titlepage"><div><div><h4 class="title"> 27<a name="spirit.lex.tutorials.lexer_tutorials"></a><a class="link" href="lexer_tutorials.html" title="Spirit.Lex Tutorials Overview"><span class="emphasis"><em>Spirit.Lex</em></span> 28 Tutorials Overview</a> 29</h4></div></div></div> 30<p> 31 The <span class="emphasis"><em>Spirit.Lex</em></span> library implements several components 32 on top of possibly different lexer generator libraries. It exposes a pair 33 of iterators, which, when dereferenced, return a stream of tokens generated 34 from the underlying character stream. The generated tokens are based on 35 the token definitions supplied by the user. 36 </p> 37<p> 38 Currently, <span class="emphasis"><em>Spirit.Lex</em></span> is built on top of Ben Hanson's 39 excellent <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a> 40 library (which is a proposed Boost library). <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a> 41 provides the necessary functionality to build state machines based on a 42 set of supplied regular expressions. But <span class="emphasis"><em>Spirit.Lex</em></span> 43 is not restricted to be used with <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>. 44 We expect it to be usable in conjunction with any other lexical scanner 45 generator library, all what needs to be implemented is a set of wrapper 46 objects exposing a well defined interface as described in this documentation. 47 </p> 48<div class="note"><table border="0" summary="Note"> 49<tr> 50<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../images/note.png"></td> 51<th align="left">Note</th> 52</tr> 53<tr><td align="left" valign="top"><p> 54 For the sake of clarity all examples in this documentation assume <span class="emphasis"><em>Spirit.Lex</em></span> 55 to be used on top of <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>. 56 </p></td></tr> 57</table></div> 58<p> 59 Building a lexer using <span class="emphasis"><em>Spirit.Lex</em></span> is highly configurable, 60 where most of this configuration is done at compile time. Almost all of 61 the configurable parameters have generally useful default values, allowing 62 project startup to be a easy and straightforward task. Here is a (non-complete) 63 list of features you can tweak to adjust the generated lexer instance to 64 the actual needs: 65 </p> 66<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 67<li class="listitem"> 68 Select and customize the token type to be generated by the lexer instance. 69 </li> 70<li class="listitem"> 71 Select and customize the token value types the generated token instances 72 will be able to hold. 73 </li> 74<li class="listitem"> 75 Select the iterator type of the underlying input stream, which will 76 be used as the source for the character stream to tokenize. 77 </li> 78<li class="listitem"> 79 Customize the iterator type returned by the lexer to enable debug support, 80 special handling of certain input sequences, etc. 81 </li> 82<li class="listitem"> 83 Select the <span class="emphasis"><em>dynamic</em></span> or the <span class="emphasis"><em>static</em></span> 84 runtime model for the lexical analyzer. 85 </li> 86</ul></div> 87<p> 88 Special care has been taken during the development of the library that 89 optimal code will be generated regardless of the configuration options 90 selected. 91 </p> 92<p> 93 The series of tutorial examples of this section will guide you through 94 some common use cases helping to understand the big picture. The first 95 two quick start examples (<a class="link" href="lexer_quickstart1.html" title="Quickstart 1 - A word counter using Spirit.Lex">Lex 96 Quickstart 1 - A word counter using <span class="emphasis"><em>Spirit.Lex</em></span></a> 97 and <a class="link" href="lexer_quickstart2.html" title="Quickstart 2 - A better word counter using Spirit.Lex">Lex Quickstart 98 2 - A better word counter using <span class="emphasis"><em>Spirit.Lex</em></span></a>) 99 introduce the <span class="emphasis"><em>Spirit.Lex</em></span> library while building two 100 stand alone applications, not being connected to or depending on any other 101 part of <a href="http://boost-spirit.com" target="_top">Spirit</a>. The section 102 <a class="link" href="lexer_quickstart3.html" title="Quickstart 3 - Counting Words Using a Parser">Lex Quickstart 3 103 - Counting Words Using a Parser</a> demonstrates how to use a lexer 104 in conjunction with a parser (where obviously the parser is built using 105 <span class="emphasis"><em>Spirit.Qi</em></span>). 106 </p> 107</div> 108<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> 109<td align="left"></td> 110<td align="right"><div class="copyright-footer">Copyright © 2001-2011 Joel de Guzman, Hartmut Kaiser<p> 111 Distributed under the Boost Software License, Version 1.0. (See accompanying 112 file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) 113 </p> 114</div></td> 115</tr></table> 116<hr> 117<div class="spirit-nav"> 118<a accesskey="p" href="../tutorials.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_quickstart1.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a> 119</div> 120</body> 121</html> 122
