// Copyright 2014-2021 The Khronos Group Inc. // SPDX-License-Identifier: CC-BY-4.0 = Vulkan^(R)^ Specification Build Instructions and Notes :toc2: :toclevels: 2 ifdef::env-github[] :note-caption: :information_source: endif::[] [[intro]] == Introduction This README describes how to build the Vulkan API specification, reference pages, and/or other related targets. It documents how to set up your build environment, build steps and targets, and contains some troubleshooting advice. [[building]] == Building The Spec First, clone the Khronos Github repository containing the Vulkan specification to your local Linux, Windows, or Mac PC. The repository is located at https://github.com/KhronosGroup/Vulkan-Docs/. Next, install all the necessary build tools (see <> below). If you are using the <>, which we strongly recommend, then one way to build using the image (assuming a Linux docker host) is: $ docker run --user `id -u`:`id -g` -it --rm \ -v :/vulkan \ khronosgroup/docker-images:asciidoctor-spec /bin/bash where is the path to the cloned repository. This runs the image with the cloned repository under /vulkan and accesses it as a specified user (set to your own user and group ID above), so that it doesn't get filled with files owned by another user. Then in the running image, $ cd /vulkan $ ./makeSpec -spec core html which builds an HTML5 specification output for the core Vulkan 1.2 specification, with no extensions included, or $ ./makeSpec -spec all all which builds the spec targets `html`, `pdf`, `styleguide`, `registry`, `manhtmlpages`, and `allchecks`, with *all* registered extensions included. There are many other ways of using the image, including inside a Continuous Integration pipeline; locally with persistent Docker volume storage of the repository; and so on. If you are not using our Docker image to build with, and you have a <> with the entire toolchain installed, you can go to and invoke the same `make` commands there. [NOTE] .Note ==== * While it's possible to invoke `make` directly, this is rarely appropriate or useful. Usually dozens to hundreds of build options must be set to specify the desired set of extensions to include in the specification. The `makeSpec` python script, which is discussed in more detail <>, simplifies this process for common cases. * The `all` target takes a long time to run, and generates outputs that are irrelevant for most users. The `html` target just generates the HTML output, which is often all that's needed for spec bugfixes not involving extensions. * The default `make` options build a Vulkan 1.2 specification with no optional extensions. * The `validusage` target is not built as part of the `all` target, due to it needing to be built with all extensions enabled (`-spec all`). Building the `validusage` target will fail otherwise. ==== These targets generate a variety of output documents in the directory specified by the Makefile variable `$(OUTDIR)` (by default, `out/`). The checked-in file `out/index.html` links to all these targets, or they can individually be found as follows: Vulkan Specification:: * `html` -- Single-file HTML5 in `$(OUTDIR)/html/vkspec.html`, and KaTeX dependency in $(OUTDIR)/katex * `chunked` -- Chunked HTML5 in `$(OUTDIR)/html/chap?.html` * `pdf` -- PDF in `$(OUTDIR)/pdf/vkspec.pdf` "`styleguide`" (Vulkan Documentation and Extensions: Procedures and Conventions):: * `styleguide` -- Single-file HTML5 in `$(OUTDIR)/styleguide.html` XML Registry schema document:: * `registry` -- Single-file HTML5 in `$(OUTDIR)/registry.html` <>:: * `diff_html` -- Single-file HTML5 in `$(OUTDIR)/html/diff.html` <>:: * `manhtmlpages` -- File-per-entry-point HTML in `$(OUTDIR)/man/html/*`. Must be built with all extensions enabled (using `makeSpec -spec all`). <>:: * None at present. The `allchecks` target writes to standard output unless the underlying script is given additional options. Valid usage database:: * `validusage` - json database of all valid usage statements in the specification. Must be built with `./makeAllExts` (for now). Output in `$(OUTDIR)/validation/validusage.json`. A validated schema for the output of this is stored in `$(CURDIR)/config/vu-to-json/vu_schema.json` Once you have the basic build working, an appropriate parallelization option, such as `-j 8`, should significantly speed up the reference page builds. If you encounter problems refer to the <> section. [[building-versions]] === Building Specifications For Different API Versions The `Makefile` defaults to building a Vulkan 1.2 specification. This is controlled by Asciidoctor attributes passed in the Makefile variable `$(VERSIONS)` To instead build a Vulkan 1.1 specification, pass ---- VERSIONS="VK_VERSION_1_0 VK_VERSION_1_1" ---- on the `makeSpec` command line. [[building-extensions]] === Building With Extensions Included Extensions are defined in the same source as the core Specification, but are only conditionally included in the output. http://asciidoctor.org/docs/user-manual/#attributes[Asciidoctor attributes] of the same name as the extension are used to define whether the extension is included or not -- defining such an attribute will cause the output to include the text for that extension. When building the specification, the extensions included are those specified as a space-separated list of extension names (e.g. `VK_KHR_surface`) in the Makefile variable `$(EXTENSIONS)`, usually set on the make command line. When changing the list of extensions, it is critical to remove all generated files using the `clean_generated` Makefile target, as the contents of generated files depends on `$(EXTENSIONS)`. The `makeSpec` wrapper script can clean generated files and then build one or more specification targets for a set of explicitly specified extensions, including all implicit extension dependencies of that set. It accepts these options: * -clean - remove generated targets before building * -v - print actions as well as executing them * -n - print actions without executing them * -genpath *path* - specify path to generated files (default `gen`) * -spec *type* - build with sepcified sets of extensions. *type* may be ** *core* - no extensions added (default if not specified) ** *khr* - all KHR extensions added ** *all* - all registered extensions added * -extension *extname* - build with specified extension included, as well as the set specified by `-spec`. Can be given multiple times. * All remaining targets are arbitrary `make` options or targets in the Makefile. The `target(s)` passed to these scripts are arbitrary `make` options, and can be used to set Makefile variables and options discussed above, as well as specify actual build targets. For example, to build the HTML specification with all KHR extensions included as well as a single vendor extension: ---- $ ./makeSpec -clean -spec khr -extension VK_EXT_debug_report html ---- The scripts `makeAllExts`, `makeKHR`, and `makeExt` set appropriate options and invoke `makeSpec`, for backwards compatibility, but are no longer used by Khronos. The Makefile variable `$(APITITLE)` defines an additional string which is appended to the specification title. When building with extensions enabled, this should be set to something like `(with extension VK_extension_name)`. The `makeSpec` script already does this. The reference pages (the `manhtmlpages` target) must be built using the `-spec all` option; there are markup and scripting issues which will probably cause any more restricted set of refpages to fail to build. [[building-diff]] ==== Building A Highlighted Extension Diff The `diff_html` target in the Makefile can be used to generate a version of the specification which highlights changes made to the specification by the inclusion of a particular set of extensions. Extensions in the Makefile variable `$(EXTENSIONS)` define the base extensions to be enabled by the specification, and these will not be highlighted in the output. Extensions in the Makefile variable `$(DIFFEXTENSIONS)` define the set of extensions whose changes to the text will be highlighted when they are enabled. Any extensions in both variables will be treated as if they were only included in `$(DIFFEXTENSIONS)`. `$(DIFFEXTENSIONS)` can be set when using the `makeSpec` script described above. In the resulting HTML document, content that has been added by one of the extensions will be highlighted with a lime background, and content that was removed will be highlighted with a pink background. Each section has an anchor of `#differenceN`, with an arrow (=>) at the end of each section which links to the next difference section. The first diff section is `#difference1`. [NOTE] .Note ==== This output is not without errors. It may instead result in visible `+++[.added]##content##+++` and `+++[.removed]##content##+++`, and so also highlights not being rendered. But such visible markup still correctly encapsulates the modified content. ==== [[building-test]] === Alternate and Test Builds If you are just testing Asciidoctor formatting, macros, stylesheets, etc., you may want to edit `vkspec.txt` to just include your test code. The asciidoctor HTML build is very fast, even for the whole Specification, but PDF builds take several minutes. === Images Used In The Specification All images used in the specification are in the `images/` directory in the SVG format, and were created with Inkscape. We recommend using Inkscape to modify or create new images, as we've had problems using SVG files created by some other tools; especially in the PDF builds. [[validation-scripts]] === Validation Scripts The `allchecks` Makefile target runs a Python script that looks for markup errors, missing interfaces, macro misuse, and inconsistencies in the specification text. This script is necessarily heuristic, since it's dealing with lots of hand-written material, but it identifies many problems and can suggest solutions. This script is also run as part of the CI tests in the internal Khronos gitlab repository. [[macros]] == Our Asciidoctor Macros We use many custom Ruby macros in the reference pages and API spec Asciidoctor sources. The validator scripts rely on these macros as part of their sanity checks, and you should use the macros whenever referring to an API command, struct, token, or enum name, so the documents are semantically tagged and more easily verifiable. The supported macros are defined in the `config/spec-macros/extension.rb` asciidoctor extension script. The tags used are described in the link:https://www.khronos.org/registry/vulkan/specs/1.1/styleguide.html[style guide] (generated from `styleguide.txt`). We (may) eventually tool up the spec and reference pages to the point that anywhere there's a type or token referred to, clicking on (or perhaps hovering over) it in the HTML view will take reader to the definition of that type/token. That will take some more plumbing work to tag the stuff in the autogenerated include files, and do something sensible in the spec (e.g. resolve links to internal references). Most of these macros deeply need more intuitive names. [[refpages]] == Reference Pages The reference pages are extracted from the API Specification source, which has been tagged to help identify boundaries of language talking about different commands, structures, enumerants, and other types. A set of Python scripts extract and lightly massage the relevant tagged language into corresponding reference page sources. To regenerate the reference page sources from scratch yourself, execute: ---- ./makeSpec -spec all refpages ---- The `genRef.py` script will generate many warnings, but most are just reminders that some pages are automatically generated. If everything is working correctly, all the `$(GENERATED)/refpage/*.txt` files will be regenerated, but their contents will not change. If you add new API features to the Specification in a branch, make sure that the commands have the required tagging and that reference pages are generated for them, and build properly. When executing the `manhtmlpages` target in the Makefile, after building HTML versions of all reference pages extracted from the spec, symbolic links from aliases to the reference page for the API they alias will also be created. [[styles]] == Our stylesheets We use an HTML stylesheet `config/khronos.css` derived from the http://asciidoctor.org/docs/produce-custom-themes-using-asciidoctor-stylesheet-factory/[Asciidoctor stylesheet factory] "`colony`" theme, with the default Arial font family replaced by the sans-serif https://en.wikipedia.org/wiki/Noto_fonts[Noto font family]. [[styleguide]] == Vulkan Style Guide If you're writing new spec language or modifying existing language, see the link:https://www.khronos.org/registry/vulkan/specs/1.2/styleguide.html["`style guide`"] (formally titled "`Vulkan Documentation and Extensions: Procedures and Conventions`") document for details of our asciidoctor macros, extensions, mathematical equation markup, writing style, etc. [[depends]] == Software Dependencies This section describes the software components used by the Vulkan spec toolchain. In the past, we previously specified package versions and instructions for installing the toolchain in multiple desktop environments including Linux, MacOS X, and Microsoft Windows. The underlying components evolve rapidly, and we have not kept those instructions up to date. [[depends-docker]] === Khronos-Provided Docker Image Khronos has published a Docker image containing a Debian Linux distribution with the entire toolchain preinstalled. We will occasionally update this image if needed, and we recommend people needing to build from this repository use the Docker image. Docker installation is beyond the scope of this document. Refer to link:https://docs.docker.com/get-docker/[the Docker website] for information about installing Docker on Linux, Windows, and MacOS X. The name of the build image is khronosgroup/docker-images:asciidoctor-spec It can be pulled from the link:https://hub.docker.com/repository/docker/khronosgroup/docker-images[Dockerhub repository] with the command docker pull khronosgroup/docker-images:asciidoctor-spec Once docker is installed and the image is available, it can be executed as described above under <> to generate Specification output documents or other Makefile targets. [NOTE] .Note ==== The old `vulkan-docs-base` and `vulkan-docs` images continue to be hosted, but the new `asciidoctor-spec` image is preferred - the added functionality of `vulkan-docs`, to set the user/group inside Docker based on environment variables passed into Docker, is no longer needed with the Docker `--user` option. ==== [[depends-nondocker]] === Non-Docker Build Environments We do not actively support building outside of our Docker image, but it is straightforward to reproduce our toolchain in a Debian (or similar APT-based Linux) distribution by executing the same steps as the link:https://github.com/KhronosGroup/DockerContainers/blob/master/asciidoctor-spec.dockerfile[Dockerfile] used to build our Docker image. It should be possible to apply the same steps in a Windows Subsystem for Linux (WSL2) environment on Windows 10, as well. For other native environments, such as MacOS X and older Unix-like environments for Windows such as MinGW and Cygwin, we provided instructions in older versions of this document. While those instructions are out of date and have been removed from current versions of this document, you may be able to make use of link:https://github.com/KhronosGroup/Vulkan-Docs/blob/v1.2.135/BUILD.adoc#depends[the version of BUILD.adoc in the v1.2.135 repository tag] [NOTE] .Note ==== While we have no intention of forcing people to use our Docker image, we cannot support every possible environment. The Docker image is a straightforward way to use the Vulkan-Docs repository with almost all modern desktop environments. ==== [[history]] == Revision History * 2021-03-12 - Use the new Docker image. * 2020-07-15 - Update to use `makeSpec` instead of `makeAllExts`. * 2020-03-23 - Document Khronos' published Docker image for building the spec, and remove all platform-specific instructions. * 2018-12-04 - Update Rbenv and ruby gem installation instructions and package dependencies for Linux and Ubuntu/Windows 10. * 2018-10-25 - Update Troubleshooting, and Windows and Linux build. Plus random editing. * 2018-03-13 - Rename to BUILD.adoc and update for new directory structure. * 2018-03-05 - Update README for Vulkan 1.1 release. * 2017-03-20 - Add description of prawn versioning problem and how to fix it. * 2017-03-06 - Add description of ruby-enum versioning problem and how to fix it. * 2017-02-13 - Move some comments here from ../../../README.md. Tweak asciidoctor markup to more clearly delineate shell command blocks. * 2017-02-10 - Add more Ruby installation guidelines and reflow the document in accordance with the style guide. * 2017-01-31 - Add rbenv instructions and update the README elsewhere. * 2017-01-16 - Modified dependencies for Asciidoctor * 2017-01-06 - Replace MathJax with KaTeX. * 2016-08-25 - Update for the single-branch model. * 2016-07-10 - Update for current state of spec and ref page generation. * 2015-11-11 - Add new can: etc. macros and DBLATEXPREFIX variable. * 2015-09-21 - Convert document to asciidoc and rename to README.md in the hope the gitlab browser will render it in some fashion. * 2015-09-21 - Add descriptions of LaTeX and MathJax math support for all output formats. * 2015-09-02 - Added Cygwin package info. * 2015-09-02 - Initial version documenting macros, required toolchain components and versions, etc.