1# Regenerating project files 2 3Prerequisites 4- `python` 5- `pip install mako` (the template processor) 6- `pip install pyyaml` (to read the yaml files) 7- `go` (required by boringssl dependency) 8 9``` 10# Regenerate the projects files (and other generated files) using templates 11tools/buildgen/generate_projects.sh 12``` 13 14# Quick justification 15 16We've approached the problem of the build system from a lot of different 17angles. The main issue was that there isn't a single build system that 18was going to single handedly cover all of our usage cases. 19 20So instead we decided to work the following way: 21 22* A `build.yaml` file at the root is the source of truth for listing all the 23targets and files needed to build grpc and its tests, as well as a basic system 24for dependency description. 25 26* Most of the build systems supported by gRPC (e.g. Makefile, cmake, XCode) have a template defined in this directory. The templates use the information from the `build.yaml` file to generate the project files specific to a given build system. 27 28This way we can maintain as many project system as we see fit, without having 29to manually maintain them when we add or remove new code to the repository. 30Only the structure of the project file is relevant to the template. The actual 31list of source code and targets isn't. 32 33# Structure of `build.yaml` 34 35The `build.yaml` file has the following structure: 36 37``` 38settings: # global settings, such as version number 39 ... 40filegroups: # groups of files that are automatically expanded 41 ... 42libs: # list of libraries to build 43 ... 44targets: # list of targets to build 45 ... 46``` 47 48The `filegroups` are helpful to re-use a subset of files in multiple targets. 49One `filegroups` entry has the following structure: 50 51``` 52- name: "arbitrary string", # the name of the filegroup 53 public_headers: # list of public headers defined in that filegroup 54 - ... 55 headers: # list of headers defined in that filegroup 56 - ... 57 src: # list of source files defined in that filegroup 58 - ... 59``` 60 61The `libs` collection contains the list of all the libraries we describe. Some may be 62helper libraries for the tests. Some may be installable libraries. Some may be 63helper libraries for installable binaries. 64 65The `targets` array contains the list of all the binary targets we describe. Some may 66be installable binaries. 67 68One `libs` or `targets` entry has the following structure (see below for 69details): 70 71``` 72name: "arbitrary string", # the name of the library 73build: "build type", # in which situation we want that library to be 74 # built and potentially installed (see below). 75language: "...", # the language tag; "c" or "c++" 76public_headers: # list of public headers to install 77headers: # list of headers used by that target 78src: # list of files to compile 79secure: boolean, # see below 80baselib: boolean, # this is a low level library that has system 81 # dependencies 82filegroups: # list of filegroups to merge to that project 83 # note that this will be expanded automatically 84deps: # list of libraries this target depends on 85deps_linkage: "..." # "static" or "dynamic". Used by the Makefile only to 86 # determine the way dependencies are linkned. Defaults 87 # to "dynamic". 88dll: "..." # see below. 89``` 90 91## The `"build"` tag 92 93Currently, the "`build`" tag have these meanings: 94 95* `"all"`: library to build on `"make all"`, and install on the system. 96* `"protoc"`: a protoc plugin to build on `"make all"` and install on the system. 97* `"private"`: a library to only build for tests. 98* `"test"`: a test binary to run on `"make test"`. 99* `"tool"`: a binary to be built upon `"make tools"`. 100 101All of the targets should always be present in the generated project file, if 102possible and applicable. But the build tag is what should group the targets 103together in a single build command. 104 105 106## The `"secure"` tag 107 108This means this target requires OpenSSL one way or another. The values can be 109`"yes"`, `"no"` and `"check"`. The default value is `"check"`. It means that 110the target requires OpenSSL, but that since the target depends on another one 111that is supposed to also import OpenSSL, the import should then be implicitely 112transitive. `"check"` should then only disable that target if OpenSSL hasn't 113been found or is unavailable. 114 115## The `"baselib"` boolean 116 117This means this is a library that will provide most of the features for gRPC. 118In particular, if we're locally building OpenSSL, protobuf or zlib, then we 119should merge OpenSSL, protobuf or zlib inside that library. That effect depends 120on the `"language"` tag. OpenSSL and zlib are for `"c"` libraries, while 121protobuf is for `"c++"` ones. 122 123## The `"dll"` tag 124 125Currently only used by cmake. "true" means the project will be 126built with both static and dynamic runtimes. "false" means it'll only be built 127with static runtime. "only" means it'll only be built with the dll runtime. 128 129 130# The template system 131 132We're currently using the [mako templates](http://www.makotemplates.org/) 133renderer. That choice enables us to simply render text files without dragging 134with us a lot of other features. Feel free to explore the current templates 135in that directory. 136 137## The renderer engine 138 139As mentioned, the renderer is using [mako templates](http://www.makotemplates.org/), 140but some glue is needed to process all of that. See the [buildgen folder](../tools/buildgen) 141for more details. We're mainly loading the build.json file, and massaging it, 142in order to get the list of properties we need, into a Python dictionary, that 143is then passed to the template while rending it. 144 145## The plugins 146 147The file build.json itself isn't passed straight to the template files. It is 148first processed and modified by a few plugins. For example, the `filegroups` 149expander is [a plugin](../tools/buildgen/plugins/expand_filegroups.py). 150 151The structure of a plugin is simple. The plugin must defined the function 152`mako_plugin` that takes a Python dictionary. That dictionary represents the 153current state of the build.json contents. The plugin can alter it to whatever 154feature it needs to add. 155