README-design.md
1Design document for the unified scheme data
2===========================================
3
4How are things connected?
5-------------------------
6
7The unified scheme takes all its data from the `build.info` files seen
8throughout the source tree. These files hold the minimum information
9needed to build end product files from diverse sources. See the
10section on `build.info` files below.
11
12From the information in `build.info` files, `Configure` builds up an
13information database as a hash table called `%unified_info`, which is
14stored in configdata.pm, found at the top of the build tree (which may
15or may not be the same as the source tree).
16
17[`Configurations/common.tmpl`](common.tmpl) uses the data from `%unified_info` to
18generate the rules for building end product files as well as
19intermediary files with the help of a few functions found in the
20build-file templates. See the section on build-file templates further
21down for more information.
22
23build.info files
24----------------
25
26As mentioned earlier, `build.info` files are meant to hold the minimum
27information needed to build output files, and therefore only (with a
28few possible exceptions [1]) have information about end products (such
29as scripts, library files and programs) and source files (such as C
30files, C header files, assembler files, etc). Intermediate files such
31as object files are rarely directly referred to in `build.info` files (and
32when they are, it's always with the file name extension `.o`), they are
33inferred by `Configure`. By the same rule of minimalism, end product
34file name extensions (such as `.so`, `.a`, `.exe`, etc) are never mentioned
35in `build.info`. Their file name extensions will be inferred by the
36build-file templates, adapted for the platform they are meant for (see
37sections on `%unified_info` and build-file templates further down).
38
39The variables `PROGRAMS`, `LIBS`, `MODULES` and `SCRIPTS` are used to declare
40end products. There are variants for them with `_NO_INST` as suffix
41(`PROGRAM_NO_INST` etc) to specify end products that shouldn't get installed.
42
43The variables `SOURCE`, `DEPEND`, `INCLUDE` and `DEFINE` are indexed by a
44produced file, and their values are the source used to produce that
45particular produced file, extra dependencies, include directories
46needed, or C macros to be defined.
47
48All their values in all the `build.info` throughout the source tree are
49collected together and form a set of programs, libraries, modules and
50scripts to be produced, source files, dependencies, etc etc etc.
51
52Let's have a pretend example, a very limited contraption of OpenSSL,
53composed of the program `apps/openssl`, the libraries `libssl` and
54`libcrypto`, an module `engines/ossltest` and their sources and
55dependencies.
56
57 # build.info
58 LIBS=libcrypto libssl
59 INCLUDE[libcrypto]=include
60 INCLUDE[libssl]=include
61 DEPEND[libssl]=libcrypto
62
63This is the top directory `build.info` file, and it tells us that two
64libraries are to be built, the include directory `include/` shall be
65used throughout when building anything that will end up in each
66library, and that the library `libssl` depend on the library
67`libcrypto` to function properly.
68
69 # apps/build.info
70 PROGRAMS=openssl
71 SOURCE[openssl]=openssl.c
72 INCLUDE[openssl]=.. ../include
73 DEPEND[openssl]=../libssl
74
75This is the `build.info` file in `apps/`, one may notice that all file
76paths mentioned are relative to the directory the `build.info` file is
77located in. This one tells us that there's a program to be built
78called `apps/openss` (the file name extension will depend on the
79platform and is therefore not mentioned in the `build.info` file). It's
80built from one source file, `apps/openssl.c`, and building it requires
81the use of `.` and `include/` include directories (both are declared
82from the point of view of the `apps/` directory), and that the program
83depends on the library `libssl` to function properly.
84
85 # crypto/build.info
86 LIBS=../libcrypto
87 SOURCE[../libcrypto]=aes.c evp.c cversion.c
88 DEPEND[cversion.o]=buildinf.h
89
90 GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
91 DEPEND[buildinf.h]=../Makefile
92 DEPEND[../util/mkbuildinf.pl]=../util/Foo.pm
93
94This is the `build.info` file in `crypto/`, and it tells us a little more
95about what's needed to produce `libcrypto`. LIBS is used again to
96declare that `libcrypto` is to be produced. This declaration is
97really unnecessary as it's already mentioned in the top `build.info`
98file, but can make the info file easier to understand. This is to
99show that duplicate information isn't an issue.
100
101This `build.info` file informs us that `libcrypto` is built from a few
102source files, `crypto/aes.c`, `crypto/evp.c` and `crypto/cversion.c`.
103It also shows us that building the object file inferred from
104`crypto/cversion.c` depends on `crypto/buildinf.h`. Finally, it
105also shows the possibility to declare how some files are generated
106using some script, in this case a perl script, and how such scripts
107can be declared to depend on other files, in this case a perl module.
108
109Two things are worth an extra note:
110
111`DEPEND[cversion.o]` mentions an object file. DEPEND indexes is the
112only location where it's valid to mention them
113
114 # ssl/build.info
115 LIBS=../libssl
116 SOURCE[../libssl]=tls.c
117
118This is the build.info file in `ssl/`, and it tells us that the
119library `libssl` is built from the source file `ssl/tls.c`.
120
121 # engines/build.info
122 MODULES=dasync
123 SOURCE[dasync]=e_dasync.c
124 DEPEND[dasync]=../libcrypto
125 INCLUDE[dasync]=../include
126
127 MODULES_NO_INST=ossltest
128 SOURCE[ossltest]=e_ossltest.c
129 DEPEND[ossltest]=../libcrypto.a
130 INCLUDE[ossltest]=../include
131
132This is the `build.info` file in `engines/`, telling us that two modules
133called `engines/dasync` and `engines/ossltest` shall be built, that
134`dasync`'s source is `engines/e_dasync.c` and `ossltest`'s source is
135`engines/e_ossltest.c` and that the include directory `include/` may
136be used when building anything that will be part of these modules.
137Also, both modules depend on the library `libcrypto` to function
138properly. `ossltest` is explicitly linked with the static variant of
139the library `libcrypto`. Finally, only `dasync` is being installed, as
140`ossltest` is only for internal testing.
141
142When `Configure` digests these `build.info` files, the accumulated
143information comes down to this:
144
145 LIBS=libcrypto libssl
146 SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c
147 DEPEND[crypto/cversion.o]=crypto/buildinf.h
148 INCLUDE[libcrypto]=include
149 SOURCE[libssl]=ssl/tls.c
150 INCLUDE[libssl]=include
151 DEPEND[libssl]=libcrypto
152
153 PROGRAMS=apps/openssl
154 SOURCE[apps/openssl]=apps/openssl.c
155 INCLUDE[apps/openssl]=. include
156 DEPEND[apps/openssl]=libssl
157
158 MODULES=engines/dasync
159 SOURCE[engines/dasync]=engines/e_dasync.c
160 DEPEND[engines/dasync]=libcrypto
161 INCLUDE[engines/dasync]=include
162
163 MODULES_NO_INST=engines/ossltest
164 SOURCE[engines/ossltest]=engines/e_ossltest.c
165 DEPEND[engines/ossltest]=libcrypto.a
166 INCLUDE[engines/ossltest]=include
167
168 GENERATE[crypto/buildinf.h]=util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
169 DEPEND[crypto/buildinf.h]=Makefile
170 DEPEND[util/mkbuildinf.pl]=util/Foo.pm
171
172A few notes worth mentioning:
173
174`LIBS` may be used to declare routine libraries only.
175
176`PROGRAMS` may be used to declare programs only.
177
178`MODULES` may be used to declare modules only.
179
180The indexes for `SOURCE` must only be end product files, such as
181libraries, programs or modules. The values of `SOURCE` variables must
182only be source files (possibly generated).
183
184`INCLUDE` and `DEPEND` shows a relationship between different files
185(usually produced files) or between files and directories, such as a
186program depending on a library, or between an object file and some
187extra source file.
188
189When `Configure` processes the `build.info` files, it will take it as
190truth without question, and will therefore perform very few checks.
191If the build tree is separate from the source tree, it will assume
192that all built files and up in the build directory and that all source
193files are to be found in the source tree, if they can be found there.
194`Configure` will assume that source files that can't be found in the
195source tree (such as `crypto/bildinf.h` in the example above) are
196generated and will be found in the build tree.
197
198The `%unified_info` database
199----------------------------
200
201The information in all the `build.info` get digested by `Configure` and
202collected into the `%unified_info` database, divided into the following
203indexes:
204
205 depends => a hash table containing 'file' => [ 'dependency' ... ]
206 pairs. These are directly inferred from the DEPEND
207 variables in build.info files.
208
209 modules => a list of modules. These are directly inferred from
210 the MODULES variable in build.info files.
211
212 generate => a hash table containing 'file' => [ 'generator' ... ]
213 pairs. These are directly inferred from the GENERATE
214 variables in build.info files.
215
216 includes => a hash table containing 'file' => [ 'include' ... ]
217 pairs. These are directly inferred from the INCLUDE
218 variables in build.info files.
219
220 install => a hash table containing 'type' => [ 'file' ... ] pairs.
221 The types are 'programs', 'libraries', 'modules' and
222 'scripts', and the array of files list the files of
223 that type that should be installed.
224
225 libraries => a list of libraries. These are directly inferred from
226 the LIBS variable in build.info files.
227
228 programs => a list of programs. These are directly inferred from
229 the PROGRAMS variable in build.info files.
230
231 scripts => a list of scripts. There are directly inferred from
232 the SCRIPTS variable in build.info files.
233
234 sources => a hash table containing 'file' => [ 'sourcefile' ... ]
235 pairs. These are indirectly inferred from the SOURCE
236 variables in build.info files. Object files are
237 mentioned in this hash table, with source files from
238 SOURCE variables, and AS source files for programs and
239 libraries.
240
241 shared_sources =>
242 a hash table just like 'sources', but only as source
243 files (object files) for building shared libraries.
244
245As an example, here is how the `build.info` files example from the
246section above would be digested into a `%unified_info` table:
247
248 our %unified_info = (
249 "depends" =>
250 {
251 "apps/openssl" =>
252 [
253 "libssl",
254 ],
255 "crypto/buildinf.h" =>
256 [
257 "Makefile",
258 ],
259 "crypto/cversion.o" =>
260 [
261 "crypto/buildinf.h",
262 ],
263 "engines/dasync" =>
264 [
265 "libcrypto",
266 ],
267 "engines/ossltest" =>
268 [
269 "libcrypto.a",
270 ],
271 "libssl" =>
272 [
273 "libcrypto",
274 ],
275 "util/mkbuildinf.pl" =>
276 [
277 "util/Foo.pm",
278 ],
279 },
280 "modules" =>
281 [
282 "engines/dasync",
283 "engines/ossltest",
284 ],
285 "generate" =>
286 {
287 "crypto/buildinf.h" =>
288 [
289 "util/mkbuildinf.pl",
290 "\"\$(CC)",
291 "\$(CFLAGS)\"",
292 "\"$(PLATFORM)\"",
293 ],
294 },
295 "includes" =>
296 {
297 "apps/openssl" =>
298 [
299 ".",
300 "include",
301 ],
302 "engines/ossltest" =>
303 [
304 "include"
305 ],
306 "libcrypto" =>
307 [
308 "include",
309 ],
310 "libssl" =>
311 [
312 "include",
313 ],
314 "util/mkbuildinf.pl" =>
315 [
316 "util",
317 ],
318 }
319 "install" =>
320 {
321 "modules" =>
322 [
323 "engines/dasync",
324 ],
325 "libraries" =>
326 [
327 "libcrypto",
328 "libssl",
329 ],
330 "programs" =>
331 [
332 "apps/openssl",
333 ],
334 },
335 "libraries" =>
336 [
337 "libcrypto",
338 "libssl",
339 ],
340 "programs" =>
341 [
342 "apps/openssl",
343 ],
344 "sources" =>
345 {
346 "apps/openssl" =>
347 [
348 "apps/openssl.o",
349 ],
350 "apps/openssl.o" =>
351 [
352 "apps/openssl.c",
353 ],
354 "crypto/aes.o" =>
355 [
356 "crypto/aes.c",
357 ],
358 "crypto/cversion.o" =>
359 [
360 "crypto/cversion.c",
361 ],
362 "crypto/evp.o" =>
363 [
364 "crypto/evp.c",
365 ],
366 "engines/e_dasync.o" =>
367 [
368 "engines/e_dasync.c",
369 ],
370 "engines/dasync" =>
371 [
372 "engines/e_dasync.o",
373 ],
374 "engines/e_ossltest.o" =>
375 [
376 "engines/e_ossltest.c",
377 ],
378 "engines/ossltest" =>
379 [
380 "engines/e_ossltest.o",
381 ],
382 "libcrypto" =>
383 [
384 "crypto/aes.c",
385 "crypto/cversion.c",
386 "crypto/evp.c",
387 ],
388 "libssl" =>
389 [
390 "ssl/tls.c",
391 ],
392 "ssl/tls.o" =>
393 [
394 "ssl/tls.c",
395 ],
396 },
397 );
398
399As can be seen, everything in `%unified_info` is fairly simple suggest
400of information. Still, it tells us that to build all programs, we
401must build `apps/openssl`, and to build the latter, we will need to
402build all its sources (`apps/openssl.o` in this case) and all the
403other things it depends on (such as `libssl`). All those dependencies
404need to be built as well, using the same logic, so to build `libssl`,
405we need to build `ssl/tls.o` as well as `libcrypto`, and to build the
406latter...
407
408Build-file templates
409--------------------
410
411Build-file templates are essentially build-files (such as `Makefile` on
412Unix) with perl code fragments mixed in. Those perl code fragment
413will generate all the configuration dependent data, including all the
414rules needed to build end product files and intermediary files alike.
415At a minimum, there must be a perl code fragment that defines a set of
416functions that are used to generates specific build-file rules, to
417build static libraries from object files, to build shared libraries
418from static libraries, to programs from object files and libraries,
419etc.
420
421 generatesrc - function that produces build file lines to generate
422 a source file from some input.
423
424 It's called like this:
425
426 generatesrc(src => "PATH/TO/tobegenerated",
427 generator => [ "generatingfile", ... ]
428 generator_incs => [ "INCL/PATH", ... ]
429 generator_deps => [ "dep1", ... ]
430 incs => [ "INCL/PATH", ... ],
431 deps => [ "dep1", ... ],
432 intent => one of "libs", "dso", "bin" );
433
434 'src' has the name of the file to be generated.
435 'generator' is the command or part of command to
436 generate the file, of which the first item is
437 expected to be the file to generate from.
438 generatesrc() is expected to analyse and figure out
439 exactly how to apply that file and how to capture
440 the result. 'generator_incs' and 'generator_deps'
441 are include directories and files that the generator
442 file itself depends on. 'incs' and 'deps' are
443 include directories and files that are used if $(CC)
444 is used as an intermediary step when generating the
445 end product (the file indicated by 'src'). 'intent'
446 indicates what the generated file is going to be
447 used for.
448
449 src2obj - function that produces build file lines to build an
450 object file from source files and associated data.
451
452 It's called like this:
453
454 src2obj(obj => "PATH/TO/objectfile",
455 srcs => [ "PATH/TO/sourcefile", ... ],
456 deps => [ "dep1", ... ],
457 incs => [ "INCL/PATH", ... ]
458 intent => one of "lib", "dso", "bin" );
459
460 'obj' has the intended object file with `.o`
461 extension, src2obj() is expected to change it to
462 something more suitable for the platform.
463 'srcs' has the list of source files to build the
464 object file, with the first item being the source
465 file that directly corresponds to the object file.
466 'deps' is a list of explicit dependencies. 'incs'
467 is a list of include file directories. Finally,
468 'intent' indicates what this object file is going
469 to be used for.
470
471 obj2lib - function that produces build file lines to build a
472 static library file ("libfoo.a" in Unix terms) from
473 object files.
474
475 called like this:
476
477 obj2lib(lib => "PATH/TO/libfile",
478 objs => [ "PATH/TO/objectfile", ... ]);
479
480 'lib' has the intended library file name *without*
481 extension, obj2lib is expected to add that. 'objs'
482 has the list of object files to build this library.
483
484 libobj2shlib - backward compatibility function that's used the
485 same way as obj2shlib (described next), and was
486 expected to build the shared library from the
487 corresponding static library when that was suitable.
488 NOTE: building a shared library from a static
489 library is now DEPRECATED, as they no longer share
490 object files. Attempting to do this will fail.
491
492 obj2shlib - function that produces build file lines to build a
493 shareable object library file ("libfoo.so" in Unix
494 terms) from the corresponding object files.
495
496 called like this:
497
498 obj2shlib(shlib => "PATH/TO/shlibfile",
499 lib => "PATH/TO/libfile",
500 objs => [ "PATH/TO/objectfile", ... ],
501 deps => [ "PATH/TO/otherlibfile", ... ]);
502
503 'lib' has the base (static) library file name
504 *without* extension. This is useful in case
505 supporting files are needed (such as import
506 libraries on Windows).
507 'shlib' has the corresponding shared library name
508 *without* extension. 'deps' has the list of other
509 libraries (also *without* extension) this library
510 needs to be linked with. 'objs' has the list of
511 object files to build this library.
512
513 obj2dso - function that produces build file lines to build a
514 dynamic shared object file from object files.
515
516 called like this:
517
518 obj2dso(lib => "PATH/TO/libfile",
519 objs => [ "PATH/TO/objectfile", ... ],
520 deps => [ "PATH/TO/otherlibfile",
521 ... ]);
522
523 This is almost the same as obj2shlib, but the
524 intent is to build a shareable library that can be
525 loaded in runtime (a "plugin"...).
526
527 obj2bin - function that produces build file lines to build an
528 executable file from object files.
529
530 called like this:
531
532 obj2bin(bin => "PATH/TO/binfile",
533 objs => [ "PATH/TO/objectfile", ... ],
534 deps => [ "PATH/TO/libfile", ... ]);
535
536 'bin' has the intended executable file name
537 *without* extension, obj2bin is expected to add
538 that. 'objs' has the list of object files to build
539 this library. 'deps' has the list of library files
540 (also *without* extension) that the programs needs
541 to be linked with.
542
543 in2script - function that produces build file lines to build a
544 script file from some input.
545
546 called like this:
547
548 in2script(script => "PATH/TO/scriptfile",
549 sources => [ "PATH/TO/infile", ... ]);
550
551 'script' has the intended script file name.
552 'sources' has the list of source files to build the
553 resulting script from.
554
555Along with the build-file templates is the driving template
556[`Configurations/common.tmpl`](common.tmpl), which looks through all the
557information in `%unified_info` and generates all the rulesets to build libraries,
558programs and all intermediate files, using the rule generating
559functions defined in the build-file template.
560
561As an example with the smaller `build.info` set we've seen as an
562example, producing the rules to build `libcrypto` would result in the
563following calls:
564
565 # Note: obj2shlib will only be called if shared libraries are
566 # to be produced.
567 # Note 2: obj2shlib must convert the '.o' extension to whatever
568 # is suitable on the local platform.
569 obj2shlib(shlib => "libcrypto",
570 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ],
571 deps => [ ]);
572
573 obj2lib(lib => "libcrypto"
574 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ]);
575
576 src2obj(obj => "crypto/aes.o"
577 srcs => [ "crypto/aes.c" ],
578 deps => [ ],
579 incs => [ "include" ],
580 intent => "lib");
581
582 src2obj(obj => "crypto/evp.o"
583 srcs => [ "crypto/evp.c" ],
584 deps => [ ],
585 incs => [ "include" ],
586 intent => "lib");
587
588 src2obj(obj => "crypto/cversion.o"
589 srcs => [ "crypto/cversion.c" ],
590 deps => [ "crypto/buildinf.h" ],
591 incs => [ "include" ],
592 intent => "lib");
593
594 generatesrc(src => "crypto/buildinf.h",
595 generator => [ "util/mkbuildinf.pl", "\"$(CC)",
596 "$(CFLAGS)\"", "\"$(PLATFORM)\"" ],
597 generator_incs => [ "util" ],
598 generator_deps => [ "util/Foo.pm" ],
599 incs => [ ],
600 deps => [ ],
601 intent => "lib");
602
603The returned strings from all those calls are then concatenated
604together and written to the resulting build-file.
605
README.md
1Intro
2=====
3
4This directory contains a few sets of files that are used for
5configuration in diverse ways:
6
7 *.conf Target platform configurations, please read
8 'Configurations of OpenSSL target platforms' for more
9 information.
10 *.tmpl Build file templates, please read 'Build-file
11 programming with the "unified" build system' as well
12 as 'Build info files' for more information.
13 *.pm Helper scripts / modules for the main `Configure`
14 script. See 'Configure helper scripts for more
15 information.
16
17Configurations of OpenSSL target platforms
18==========================================
19
20Configuration targets are a collection of facts that we know about
21different platforms and their capabilities. We organise them in a
22hash table, where each entry represent a specific target.
23
24Note that configuration target names must be unique across all config
25files. The Configure script does check that a config file doesn't
26have config targets that shadow config targets from other files.
27
28In each table entry, the following keys are significant:
29
30 inherit_from => Other targets to inherit values from.
31 Explained further below. [1]
32 template => Set to 1 if this isn't really a platform
33 target. Instead, this target is a template
34 upon which other targets can be built.
35 Explained further below. [1]
36
37 sys_id => System identity for systems where that
38 is difficult to determine automatically.
39
40 enable => Enable specific configuration features.
41 This MUST be an array of words.
42 disable => Disable specific configuration features.
43 This MUST be an array of words.
44 Note: if the same feature is both enabled
45 and disabled, disable wins.
46
47 as => The assembler command. This is not always
48 used (for example on Unix, where the C
49 compiler is used instead).
50 asflags => Default assembler command flags [4].
51 cpp => The C preprocessor command, normally not
52 given, as the build file defaults are
53 usually good enough.
54 cppflags => Default C preprocessor flags [4].
55 defines => As an alternative, macro definitions may be
56 given here instead of in 'cppflags' [4].
57 If given here, they MUST be as an array of
58 the string such as "MACRO=value", or just
59 "MACRO" for definitions without value.
60 includes => As an alternative, inclusion directories
61 may be given here instead of in 'cppflags'
62 [4]. If given here, the MUST be an array
63 of strings, one directory specification
64 each.
65 cc => The C compiler command, usually one of "cc",
66 "gcc" or "clang". This command is normally
67 also used to link object files and
68 libraries into the final program.
69 cxx => The C++ compiler command, usually one of
70 "c++", "g++" or "clang++". This command is
71 also used when linking a program where at
72 least one of the object file is made from
73 C++ source.
74 cflags => Defaults C compiler flags [4].
75 cxxflags => Default C++ compiler flags [4]. If unset,
76 it gets the same value as cflags.
77
78 (linking is a complex thing, see [3] below)
79 ld => Linker command, usually not defined
80 (meaning the compiler command is used
81 instead).
82 (NOTE: this is here for future use, it's
83 not implemented yet)
84 lflags => Default flags used when linking apps,
85 shared libraries or DSOs [4].
86 ex_libs => Extra libraries that are needed when
87 linking shared libraries, DSOs or programs.
88 The value is also assigned to Libs.private
89 in $(libdir)/pkgconfig/libcrypto.pc.
90
91 shared_cppflags => Extra C preprocessor flags used when
92 processing C files for shared libraries.
93 shared_cflag => Extra C compiler flags used when compiling
94 for shared libraries, typically something
95 like "-fPIC".
96 shared_ldflag => Extra linking flags used when linking
97 shared libraries.
98 module_cppflags
99 module_cflags
100 module_ldflags => Has the same function as the corresponding
101 'shared_' attributes, but for building DSOs.
102 When unset, they get the same values as the
103 corresponding 'shared_' attributes.
104
105 ar => The library archive command, the default is
106 "ar".
107 (NOTE: this is here for future use, it's
108 not implemented yet)
109 arflags => Flags to be used with the library archive
110 command. On Unix, this includes the
111 command letter, 'r' by default.
112
113 ranlib => The library archive indexing command, the
114 default is 'ranlib' it it exists.
115
116 unistd => An alternative header to the typical
117 '<unistd.h>'. This is very rarely needed.
118
119 shared_extension => File name extension used for shared
120 libraries.
121 obj_extension => File name extension used for object files.
122 On unix, this defaults to ".o" (NOTE: this
123 is here for future use, it's not
124 implemented yet)
125 exe_extension => File name extension used for executable
126 files. On unix, this defaults to "" (NOTE:
127 this is here for future use, it's not
128 implemented yet)
129 shlib_variant => A "variant" identifier inserted between the base
130 shared library name and the extension. On "unixy"
131 platforms (BSD, Linux, Solaris, MacOS/X, ...) this
132 supports installation of custom OpenSSL libraries
133 that don't conflict with other builds of OpenSSL
134 installed on the system. The variant identifier
135 becomes part of the SONAME of the library and also
136 any symbol versions (symbol versions are not used or
137 needed with MacOS/X). For example, on a system
138 where a default build would normally create the SSL
139 shared library as 'libssl.so -> libssl.so.1.1' with
140 the value of the symlink as the SONAME, a target
141 definition that sets 'shlib_variant => "-abc"' will
142 create 'libssl.so -> libssl-abc.so.1.1', again with
143 an SONAME equal to the value of the symlink. The
144 symbol versions associated with the variant library
145 would then be 'OPENSSL_ABC_<version>' rather than
146 the default 'OPENSSL_<version>'. The string inserted
147 into symbol versions is obtained by mapping all
148 letters in the "variant" identifier to upper case
149 and all non-alphanumeric characters to '_'.
150
151 thread_scheme => The type of threads is used on the
152 configured platform. Currently known
153 values are "(unknown)", "pthreads",
154 "uithreads" (a.k.a solaris threads) and
155 "winthreads". Except for "(unknown)", the
156 actual value is currently ignored but may
157 be used in the future. See further notes
158 below [2].
159 dso_scheme => The type of dynamic shared objects to build
160 for. This mostly comes into play with
161 modules, but can be used for other purposes
162 as well. Valid values are "DLFCN"
163 (dlopen() et al), "DLFCN_NO_H" (for systems
164 that use dlopen() et al but do not have
165 fcntl.h), "DL" (shl_load() et al), "WIN32"
166 and "VMS".
167 asm_arch => The architecture to be used for compiling assembly
168 source. This acts as a selector in build.info files.
169 uplink_arch => The architecture to be used for compiling uplink
170 source. This acts as a selector in build.info files.
171 This is separate from asm_arch because it's compiled
172 even when 'no-asm' is given, even though it contains
173 assembler source.
174 perlasm_scheme => The perlasm method used to create the
175 assembler files used when compiling with
176 assembler implementations.
177 shared_target => The shared library building method used.
178 This serves multiple purposes:
179 - as index for targets found in shared_info.pl.
180 - as linker script generation selector.
181 To serve both purposes, the index for shared_info.pl
182 should end with '-shared', and this suffix will be
183 removed for use as a linker script generation
184 selector. Note that the latter is only used if
185 'shared_defflag' is defined.
186 build_scheme => The scheme used to build up a Makefile.
187 In its simplest form, the value is a string
188 with the name of the build scheme.
189 The value may also take the form of a list
190 of strings, if the build_scheme is to have
191 some options. In this case, the first
192 string in the list is the name of the build
193 scheme.
194 Currently recognised build scheme is "unified".
195 For the "unified" build scheme, this item
196 *must* be an array with the first being the
197 word "unified" and the second being a word
198 to identify the platform family.
199
200 multilib => On systems that support having multiple
201 implementations of a library (typically a
202 32-bit and a 64-bit variant), this is used
203 to have the different variants in different
204 directories.
205
206 bn_ops => Building options (was just bignum options in
207 the earlier history of this option, hence the
208 name). This is a string of words that describe
209 algorithms' implementation parameters that
210 are optimal for the designated target platform,
211 such as the type of integers used to build up
212 the bignum, different ways to implement certain
213 ciphers and so on. To fully comprehend the
214 meaning, the best is to read the affected
215 source.
216 The valid words are:
217
218 THIRTY_TWO_BIT bignum limbs are 32 bits,
219 this is default if no
220 option is specified, it
221 works on any supported
222 system [unless "wider"
223 limb size is implied in
224 assembly code];
225 BN_LLONG bignum limbs are 32 bits,
226 but 64-bit 'unsigned long
227 long' is used internally
228 in calculations;
229 SIXTY_FOUR_BIT_LONG bignum limbs are 64 bits
230 and sizeof(long) is 8;
231 SIXTY_FOUR_BIT bignums limbs are 64 bits,
232 but execution environment
233 is ILP32;
234 RC4_CHAR RC4 key schedule is made
235 up of 'unsigned char's;
236 RC4_INT RC4 key schedule is made
237 up of 'unsigned int's;
238
239[1] as part of the target configuration, one can have a key called
240 `inherit_from` that indicates what other configurations to inherit
241 data from. These are resolved recursively.
242
243 Inheritance works as a set of default values that can be overridden
244 by corresponding key values in the inheriting configuration.
245
246 Note 1: any configuration table can be used as a template.
247 Note 2: pure templates have the attribute `template => 1` and
248 cannot be used as build targets.
249
250 If several configurations are given in the `inherit_from` array,
251 the values of same attribute are concatenated with space
252 separation. With this, it's possible to have several smaller
253 templates for different configuration aspects that can be combined
254 into a complete configuration.
255
256 Instead of a scalar value or an array, a value can be a code block
257 of the form `sub { /* your code here */ }`. This code block will
258 be called with the list of inherited values for that key as
259 arguments. In fact, the concatenation of strings is really done
260 by using `sub { join(" ",@_) }` on the list of inherited values.
261
262 An example:
263
264 "foo" => {
265 template => 1,
266 haha => "ha ha",
267 hoho => "ho",
268 ignored => "This should not appear in the end result",
269 },
270 "bar" => {
271 template => 1,
272 haha => "ah",
273 hoho => "haho",
274 hehe => "hehe"
275 },
276 "laughter" => {
277 inherit_from => [ "foo", "bar" ],
278 hehe => sub { join(" ",(@_,"!!!")) },
279 ignored => "",
280 }
281
282 The entry for "laughter" will become as follows after processing:
283
284 "laughter" => {
285 haha => "ha ha ah",
286 hoho => "ho haho",
287 hehe => "hehe !!!",
288 ignored => ""
289 }
290
291[2] OpenSSL is built with threading capabilities unless the user
292 specifies `no-threads`. The value of the key `thread_scheme` may
293 be `(unknown)`, in which case the user MUST give some compilation
294 flags to `Configure`.
295
296[3] OpenSSL has three types of things to link from object files or
297 static libraries:
298
299 - shared libraries; that would be libcrypto and libssl.
300 - shared objects (sometimes called dynamic libraries); that would
301 be the modules.
302 - applications; those are apps/openssl and all the test apps.
303
304 Very roughly speaking, linking is done like this (words in braces
305 represent the configuration settings documented at the beginning
306 of this file):
307
308 shared libraries:
309 {ld} $(CFLAGS) {lflags} {shared_ldflag} -o libfoo.so \
310 foo/something.o foo/somethingelse.o {ex_libs}
311
312 shared objects:
313 {ld} $(CFLAGS) {lflags} {module_ldflags} -o libeng.so \
314 blah1.o blah2.o -lcrypto {ex_libs}
315
316 applications:
317 {ld} $(CFLAGS) {lflags} -o app \
318 app1.o utils.o -lssl -lcrypto {ex_libs}
319
320[4] There are variants of these attribute, prefixed with `lib_`,
321 `dso_` or `bin_`. Those variants replace the unprefixed attribute
322 when building library, DSO or program modules specifically.
323
324Historically, the target configurations came in form of a string with
325values separated by colons. This use is deprecated. The string form
326looked like this:
327
328 "target" => "{cc}:{cflags}:{unistd}:{thread_cflag}:{sys_id}:{lflags}:
329 {bn_ops}:{cpuid_obj}:{bn_obj}:{ec_obj}:{des_obj}:{aes_obj}:
330 {bf_obj}:{md5_obj}:{sha1_obj}:{cast_obj}:{rc4_obj}:
331 {rmd160_obj}:{rc5_obj}:{wp_obj}:{cmll_obj}:{modes_obj}:
332 {padlock_obj}:{perlasm_scheme}:{dso_scheme}:{shared_target}:
333 {shared_cflag}:{shared_ldflag}:{shared_extension}:{ranlib}:
334 {arflags}:{multilib}"
335
336Build info files
337================
338
339The `build.info` files that are spread over the source tree contain the
340minimum information needed to build and distribute OpenSSL. It uses a
341simple and yet fairly powerful language to determine what needs to be
342built, from what sources, and other relationships between files.
343
344For every `build.info` file, all file references are relative to the
345directory of the `build.info` file for source files, and the
346corresponding build directory for built files if the build tree
347differs from the source tree.
348
349When processed, every line is processed with the perl module
350Text::Template, using the delimiters `{-` and `-}`. The hashes
351`%config` and `%target` are passed to the perl fragments, along with
352$sourcedir and $builddir, which are the locations of the source
353directory for the current `build.info` file and the corresponding build
354directory, all relative to the top of the build tree.
355
356`Configure` only knows inherently about the top `build.info` file. For
357any other directory that has one, further directories to look into
358must be indicated like this:
359
360 SUBDIRS=something someelse
361
362On to things to be built; they are declared by setting specific
363variables:
364
365 PROGRAMS=foo bar
366 LIBS=libsomething
367 MODULES=libeng
368 SCRIPTS=myhack
369
370Note that the files mentioned for PROGRAMS, LIBS and MODULES *must* be
371without extensions. The build file templates will figure them out.
372
373For each thing to be built, it is then possible to say what sources
374they are built from:
375
376 PROGRAMS=foo bar
377 SOURCE[foo]=foo.c common.c
378 SOURCE[bar]=bar.c extra.c common.c
379
380It's also possible to tell some other dependencies:
381
382 DEPEND[foo]=libsomething
383 DEPEND[libbar]=libsomethingelse
384
385(it could be argued that 'libsomething' and 'libsomethingelse' are
386source as well. However, the files given through SOURCE are expected
387to be located in the source tree while files given through DEPEND are
388expected to be located in the build tree)
389
390It's also possible to depend on static libraries explicitly:
391
392 DEPEND[foo]=libsomething.a
393 DEPEND[libbar]=libsomethingelse.a
394
395This should be rarely used, and care should be taken to make sure it's
396only used when supported. For example, native Windows build doesn't
397support building static libraries and DLLs at the same time, so using
398static libraries on Windows can only be done when configured
399`no-shared`.
400
401In some cases, it's desirable to include some source files in the
402shared form of a library only:
403
404 SHARED_SOURCE[libfoo]=dllmain.c
405
406For any file to be built, it's also possible to tell what extra
407include paths the build of their source files should use:
408
409 INCLUDE[foo]=include
410
411It's also possible to specify C macros that should be defined:
412
413 DEFINE[foo]=FOO BAR=1
414
415In some cases, one might want to generate some source files from
416others, that's done as follows:
417
418 GENERATE[foo.s]=asm/something.pl $(CFLAGS)
419 GENERATE[bar.s]=asm/bar.S
420
421The value of each GENERATE line is a command line or part of it.
422Configure places no rules on the command line, except that the first
423item must be the generator file. It is, however, entirely up to the
424build file template to define exactly how those command lines should
425be handled, how the output is captured and so on.
426
427Sometimes, the generator file itself depends on other files, for
428example if it is a perl script that depends on other perl modules.
429This can be expressed using DEPEND like this:
430
431 DEPEND[asm/something.pl]=../perlasm/Foo.pm
432
433There may also be cases where the exact file isn't easily specified,
434but an inclusion directory still needs to be specified. INCLUDE can
435be used in that case:
436
437 INCLUDE[asm/something.pl]=../perlasm
438
439NOTE: GENERATE lines are limited to one command only per GENERATE.
440
441Finally, you can have some simple conditional use of the `build.info`
442information, looking like this:
443
444 IF[1]
445 something
446 ELSIF[2]
447 something other
448 ELSE
449 something else
450 ENDIF
451
452The expression in square brackets is interpreted as a string in perl,
453and will be seen as true if perl thinks it is, otherwise false. For
454example, the above would have "something" used, since 1 is true.
455
456Together with the use of Text::Template, this can be used as
457conditions based on something in the passed variables, for example:
458
459 IF[{- $disabled{shared} -}]
460 LIBS=libcrypto
461 SOURCE[libcrypto]=...
462 ELSE
463 LIBS=libfoo
464 SOURCE[libfoo]=...
465 ENDIF
466
467Build-file programming with the "unified" build system
468======================================================
469
470"Build files" are called `Makefile` on Unix-like operating systems,
471`descrip.mms` for MMS on VMS, `makefile` for `nmake` on Windows, etc.
472
473To use the "unified" build system, the target configuration needs to
474set the three items `build_scheme`, `build_file` and `build_command`.
475In the rest of this section, we will assume that `build_scheme` is set
476to "unified" (see the configurations documentation above for the
477details).
478
479For any name given by `build_file`, the "unified" system expects a
480template file in `Configurations/` named like the build file, with
481`.tmpl` appended, or in case of possible ambiguity, a combination of
482the second `build_scheme` list item and the `build_file` name. For
483example, if `build_file` is set to `Makefile`, the template could be
484`Configurations/Makefile.tmpl` or `Configurations/unix-Makefile.tmpl`.
485In case both `Configurations/unix-Makefile.tmpl` and
486`Configurations/Makefile.tmpl` are present, the former takes precedence.
487
488The build-file template is processed with the perl module
489Text::Template, using `{-` and `-}` as delimiters that enclose the
490perl code fragments that generate configuration-dependent content.
491Those perl fragments have access to all the hash variables from
492configdata.pem.
493
494The build-file template is expected to define at least the following
495perl functions in a perl code fragment enclosed with `{-` and `-}`.
496They are all expected to return a string with the lines they produce.
497
498 generatesrc - function that produces build file lines to generate
499 a source file from some input.
500
501 It's called like this:
502
503 generatesrc(src => "PATH/TO/tobegenerated",
504 generator => [ "generatingfile", ... ]
505 generator_incs => [ "INCL/PATH", ... ]
506 generator_deps => [ "dep1", ... ]
507 generator => [ "generatingfile", ... ]
508 incs => [ "INCL/PATH", ... ],
509 deps => [ "dep1", ... ],
510 intent => one of "libs", "dso", "bin" );
511
512 'src' has the name of the file to be generated.
513 'generator' is the command or part of command to
514 generate the file, of which the first item is
515 expected to be the file to generate from.
516 generatesrc() is expected to analyse and figure out
517 exactly how to apply that file and how to capture
518 the result. 'generator_incs' and 'generator_deps'
519 are include directories and files that the generator
520 file itself depends on. 'incs' and 'deps' are
521 include directories and files that are used if $(CC)
522 is used as an intermediary step when generating the
523 end product (the file indicated by 'src'). 'intent'
524 indicates what the generated file is going to be
525 used for.
526
527 src2obj - function that produces build file lines to build an
528 object file from source files and associated data.
529
530 It's called like this:
531
532 src2obj(obj => "PATH/TO/objectfile",
533 srcs => [ "PATH/TO/sourcefile", ... ],
534 deps => [ "dep1", ... ],
535 incs => [ "INCL/PATH", ... ]
536 intent => one of "lib", "dso", "bin" );
537
538 'obj' has the intended object file with '.o'
539 extension, src2obj() is expected to change it to
540 something more suitable for the platform.
541 'srcs' has the list of source files to build the
542 object file, with the first item being the source
543 file that directly corresponds to the object file.
544 'deps' is a list of explicit dependencies. 'incs'
545 is a list of include file directories. Finally,
546 'intent' indicates what this object file is going
547 to be used for.
548
549 obj2lib - function that produces build file lines to build a
550 static library file ("libfoo.a" in Unix terms) from
551 object files.
552
553 called like this:
554
555 obj2lib(lib => "PATH/TO/libfile",
556 objs => [ "PATH/TO/objectfile", ... ]);
557
558 'lib' has the intended library file name *without*
559 extension, obj2lib is expected to add that. 'objs'
560 has the list of object files to build this library.
561
562 libobj2shlib - backward compatibility function that's used the
563 same way as obj2shlib (described next), and was
564 expected to build the shared library from the
565 corresponding static library when that was suitable.
566 NOTE: building a shared library from a static
567 library is now DEPRECATED, as they no longer share
568 object files. Attempting to do this will fail.
569
570 obj2shlib - function that produces build file lines to build a
571 shareable object library file ("libfoo.so" in Unix
572 terms) from the corresponding object files.
573
574 called like this:
575
576 obj2shlib(shlib => "PATH/TO/shlibfile",
577 lib => "PATH/TO/libfile",
578 objs => [ "PATH/TO/objectfile", ... ],
579 deps => [ "PATH/TO/otherlibfile", ... ]);
580
581 'lib' has the base (static) library ffile name
582 *without* extension. This is useful in case
583 supporting files are needed (such as import
584 libraries on Windows).
585 'shlib' has the corresponding shared library name
586 *without* extension. 'deps' has the list of other
587 libraries (also *without* extension) this library
588 needs to be linked with. 'objs' has the list of
589 object files to build this library.
590
591 obj2dso - function that produces build file lines to build a
592 dynamic shared object file from object files.
593
594 called like this:
595
596 obj2dso(lib => "PATH/TO/libfile",
597 objs => [ "PATH/TO/objectfile", ... ],
598 deps => [ "PATH/TO/otherlibfile",
599 ... ]);
600
601 This is almost the same as obj2shlib, but the
602 intent is to build a shareable library that can be
603 loaded in runtime (a "plugin"...).
604
605 obj2bin - function that produces build file lines to build an
606 executable file from object files.
607
608 called like this:
609
610 obj2bin(bin => "PATH/TO/binfile",
611 objs => [ "PATH/TO/objectfile", ... ],
612 deps => [ "PATH/TO/libfile", ... ]);
613
614 'bin' has the intended executable file name
615 *without* extension, obj2bin is expected to add
616 that. 'objs' has the list of object files to build
617 this library. 'deps' has the list of library files
618 (also *without* extension) that the programs needs
619 to be linked with.
620
621 in2script - function that produces build file lines to build a
622 script file from some input.
623
624 called like this:
625
626 in2script(script => "PATH/TO/scriptfile",
627 sources => [ "PATH/TO/infile", ... ]);
628
629 'script' has the intended script file name.
630 'sources' has the list of source files to build the
631 resulting script from.
632
633In all cases, file file paths are relative to the build tree top, and
634the build file actions run with the build tree top as current working
635directory.
636
637Make sure to end the section with these functions with a string that
638you thing is appropriate for the resulting build file. If nothing
639else, end it like this:
640
641 ""; # Make sure no lingering values end up in the Makefile
642 -}
643
644Configure helper scripts
645========================
646
647Configure uses helper scripts in this directory:
648
649Checker scripts
650---------------
651
652These scripts are per platform family, to check the integrity of the
653tools used for configuration and building. The checker script used is
654either `{build_platform}-{build_file}-checker.pm` or
655`{build_platform}-checker.pm`, where `{build_platform}` is the second
656`build_scheme` list element from the configuration target data, and
657`{build_file}` is `build_file` from the same target data.
658
659If the check succeeds, the script is expected to end with a non-zero
660expression. If the check fails, the script can end with a zero, or
661with a `die`.
662