• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1[/==============================================================================
2    Copyright (C) 2001-2018 Joel de Guzman
3
4    Distributed under the Boost Software License, Version 1.0. (See accompanying
5    file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
6
7    I would like to thank Rainbowverse, llc (https://primeorbial.com/)
8    for sponsoring this work and donating it to the community.
9===============================================================================/]
10
11[section:minimal X3 Program Structure]
12
13As a prerequisite in understanding this tutorial, please review the previous
14[tutorial_employee employee example]. This example builds on top of that
15example.
16
17So far, to keep things simple, all of the tutorial programs are self
18contained in one cpp file. In reality, you will want to separate various
19logical modules of the parser into separate cpp and header files, decoupling
20the interface from the implememtation.
21
22There are many ways to structure an X3 parser, but the "minimal" example in
23this tutorial shows the preferred way. This example basically reuses the same
24parser as the [tutorial_employee employee example] for the sake of
25familiarity, but structured to allow separate compilation of the actual
26parser in its own definition file and cpp file. The cpp files, including main
27see only the header files --the interfaces. This is a good example on how X3
28parsers are structured in a C++ application.
29
30[heading Structure]
31
32The program is structured in a directory with the following header and cpp
33files:
34
35[table
36    [[`File`            ]                                               [Description            ]]
37    [[[@../../../example/x3/minimal/ast.hpp ast.hpp]]                   [The AST                ]]
38    [[[@../../../example/x3/minimal/ast_adapted.hpp ast_adapted.hpp]]   [Fusion adapters        ]]
39    [[[@../../../example/x3/minimal/config.hpp config.hpp]]             [Configuration          ]]
40    [[[@../../../example/x3/minimal/employee.hpp employee.hpp]]         [Main parser API        ]]
41    [[[@../../../example/x3/minimal/employee_def.hpp employee_def.hpp]] [Parser definitions     ]]
42    [[[@../../../example/x3/minimal/employee.cpp employee.cpp]]         [Parser instantiation   ]]
43    [[[@../../../example/x3/minimal/main.cpp main.cpp]]                 [Main program           ]]
44]
45
46The contents of the files should already be familiar. It's essentially the
47same [tutorial_employee employee example]. So I will skip the details on how
48the parser works and focus only on the features needed for refactoring the
49program into a modular structure suitable for real-world deployment.
50
51[heading AST]
52
53We place the AST declaration here:
54
55    namespace client { namespace ast
56    {
57        struct employee
58        {
59            int age;
60            std::string forename;
61            std::string surname;
62            double salary;
63        };
64
65        using boost::fusion::operator<<;
66    }}
67
68[heading Fusion adapters]
69
70Here, we adapt the AST for Fusion, making it a first-class fusion citizen:
71
72   BOOST_FUSION_ADAPT_STRUCT(client::ast::employee,
73      age, forename, surname, salary
74   )
75
76[heading Main parser API]
77
78This is the main header file that all other cpp files need to include.
79
80[#__tutorial_spirit_declare__]
81[heading BOOST_SPIRIT_DECLARE]
82
83Remember [link __tutorial_spirit_define__ `BOOST_SPIRIT_DEFINE`]? If not,
84then you probably want to go back and review that section to get a better
85understanding of what's happening.
86
87Here in the header file, instead of `BOOST_SPIRIT_DEFINE`, we use
88`BOOST_SPIRIT_DECLARE` for the *top* rule. Behind the scenes, what's actually
89happening is that we are declaring a `parse_rule` function in the client
90namespace.
91
92If you went back and reviewed [link __tutorial_spirit_define__
93BOOST_SPIRIT_DEFINE], you'll see why it is exactly what we need to use for
94header files. `BOOST_SPIRIT_DECLARE` generates function declarations that are
95meant to be placed in hpp (header) files while `BOOST_SPIRIT_DEFINE`
96generates function definitions that are meant to be placed in cpp files.
97
98[note `BOOST_SPIRIT_DECLARE` is variadic and may be used for one or more rules.
99Example: `BOOST_SPIRIT_DECLARE(r1, r2, r3);`]
100
101In this example, the top rule is `employee`. We declare `employee` in this
102header file:
103
104    namespace client
105    {
106        namespace parser
107        {
108            namespace x3 = boost::spirit::x3;
109            using employee_type = x3::rule<class employee, ast::employee>;
110            BOOST_SPIRIT_DECLARE(employee_type);
111        }
112
113        parser::employee_type employee();
114    }
115
116We also provide a function that returns an `employee` object. This is the
117parser that we will use anywhere it is needed. X3 parser objects are very
118lightweight. They are basically simple tags with no data other than the name
119of the rule (e.g. "employee"). Notice that we are passing this by value.
120
121[heading Parser Definitions]
122
123Here is where we place the actual rules that make up our grammar:
124
125    namespace parser
126    {
127        namespace x3 = boost::spirit::x3;
128        namespace ascii = boost::spirit::x3::ascii;
129
130        using x3::int_;
131        using x3::lit;
132        using x3::double_;
133        using x3::lexeme;
134        using ascii::char_;
135
136        x3::rule<class employee, ast::employee> const employee = "employee";
137
138        auto const quoted_string = lexeme['"' >> +(char_ - '"') >> '"'];
139
140        auto const employee_def =
141            lit("employee")
142            >> '{'
143            >>  int_ >> ','
144            >>  quoted_string >> ','
145            >>  quoted_string >> ','
146            >>  double_
147            >>  '}'
148            ;
149
150        BOOST_SPIRIT_DEFINE(employee);
151    }
152
153    parser::employee_type employee()
154    {
155        return parser::employee;
156    }
157
158In the parser definition, we use [link __tutorial_spirit_define__
159`BOOST_SPIRIT_DEFINE`] just like we did in the [tutorial_employee employee
160example].
161
162While this is another header file, it is not meant to be included by the
163client. Its purpose is to be included by an instantiations cpp file (see
164below). We place this in an `.hpp` file for flexibility, so we have the
165freedom to instantiate the parser with different iterator types.
166
167[#tutorial_configuration]
168[heading Configuration]
169
170Here, we declare some types for instatntaiting our X3 parser with. Rememeber
171that Spirit parsers can work with any __fwditer__. We'll also need to provide
172the initial context type. This is the context that X3 will use to initiate a
173parse. For calling `phrase_parse`, you will need the `phrase_parse_context`
174like we do below, passing in the skipper type.
175
176    using iterator_type = std::string::const_iterator;
177    using context_type = x3::phrase_parse_context<x3::ascii::space_type>::type;
178
179For plain `parse`, we simply use `x3::unused_type`.
180
181[heading Parser Instantiation]
182
183Now we instantiate our parser here, for our specific configuration:
184
185    namespace client { namespace parser
186    {
187        BOOST_SPIRIT_INSTANTIATE(employee_type, iterator_type, context_type);
188    }}
189
190For that, we use `BOOST_SPIRIT_INSTANTIATE`, passing in the parser type,
191the iterator type, and the context type.
192
193[heading BOOST_SPIRIT_INSTANTIATE]
194
195Go back and review [link __tutorial_spirit_define__ `BOOST_SPIRIT_DEFINE`]
196and [link __tutorial_spirit_declare__ `BOOST_SPIRIT_DECLARE`] to get a better
197grasp of what's happening with `BOOST_SPIRIT_INSTANTIATE` and why it is
198needed.
199
200So what the heck is `BOOST_SPIRIT_INSTANTIATE`? What we want is to isolate
201the instantiation of our parsers (rules and all that), into separate
202translation units (or cpp files, if you will). In this example, we want to
203place our x3 employee stuff in [@../../../example/x3/minimal/employee.cpp
204employee.cpp]. That way, we have separate compilation. Every time we update
205our employee parser source code, we only have to build the `employee.cpp`
206file. All the rest will not be affected. By compiling only once in one
207translation unit, we save on build times and avoid code bloat. There is no
208code duplication, which can happen otherwise if you simply include the
209employee parser ([@../../../example/x3/minimal/employee.hpp employee.hpp])
210everywhere.
211
212But how do you do that. Remember that our parser definitions are also placed
213in its own header file for flexibility, so we have the freedom to instantiate
214the parser with different iterator types.
215
216What we need to do is explicitly instantiate the `parse_rule` function we
217declared and defined via `BOOST_SPIRIT_DECLARE` and `BOOST_SPIRIT_DEFINE`
218respectively, using `BOOST_SPIRIT_INSTANTIATE`. For our particular example,
219`BOOST_SPIRIT_INSTANTIATE` expands to this code:
220
221    template bool parse_rule<iterator_type, context_type>(
222        employee_type rule_
223      , iterator_type& first, iterator_type const& last
224      , context_type const& context, employee_type::attribute_type& attr);
225
226[heading Main Program]
227
228Finally, we have our main program. The code is the same as single cpp file
229[tutorial_employee employee example], but here, we simply include three
230header files:
231
232    #include "ast.hpp"
233    #include "ast_adapted.hpp"
234    #include "employee.hpp"
235
236# `ast.hpp` for the AST declaration
237# `ast_adapted.hpp` if you need to traverse the AST using fusion
238# `employee.hpp` the main parser API
239
240[endsect]
241