README.md
1# Arm NN Build Tool
2The **_Arm NN Build Tool_** provides a collection of **parameterized bash scripts** accompanied by a **Dockerfile** for building Arm NN and its dependencies, including **[Arm Compute Library (ACL)](https://github.com/ARM-software/ComputeLibrary/)**.
3This tool replaces/supersedes the majority of the existing Arm NN build guides as a user-friendly way to build Arm NN from scratch.
4The main benefit of building Arm NN from scratch is the ability to **exactly choose which components to build, targeted for your ML project**.<br>
5The Arm NN Build Tool is tested on **x86_64 (Intel) and aarch64 (Arm) build hosts** for the **Ubuntu** platform.
6Other host platforms such as Windows and Mac **should** work (with Docker installed), but have not been officially tested.<br>
7At present, the tool supports **targeting Linux devices (from Ubuntu 18.04 onwards) on x86_64 and aarch64** architectures.<br>
8We recommend using the Arm NN Build Tool through the use of Docker. However, the scripts may be [executed directly on your machine](#build-arm-nn-without-docker) if desired.
9
10**If you already have Docker installed** and want to quickly build the Arm NN Dockerfile with some default build arguments, please follow the [Docker Build Steps](#docker-build-steps).
11
12* [Overview](#overview)
13* [System Requirements](#system-requirements)
14* [Install Docker](#install-docker)
15 * [Ubuntu / Linux](#ubuntu--linux)
16 * [Windows and Mac](#windows-and-mac)
17* [Docker Build Arguments Reference](#docker-build-arguments-reference)
18 * [SETUP_ARGS](#setup_args)
19 * [BUILD_ARGS](#build_args)
20* [Docker Build Steps](#docker-build-steps)
21* [Build Arm NN Without Docker](#build-arm-nn-without-docker)
22* [Advanced Usage](#advanced-usage)
23 * [Use custom Arm NN and ACL repository versions during build](#use-custom-arm-nn-and-acl-repository-versions-during-build)
24 * [Additional Docker Build Arguments](#additional-docker-build-arguments)
25 * [Inspect the contents of the Docker Image](#inspect-the-contents-of-the-docker-image)
26 * [Bind Mounts](#bind-mounts)
27
28## Overview
29The Arm NN Build Tool consists of an official Arm NN **Dockerfile** with associated **scripts** for building Arm NN:
301. **```install-packages.sh```:** builds CMake from source and installs system-wide **apt packages**
312. **```setup-armnn.sh```:** downloads and builds Arm NN **dependencies** (e.g. Flatbuffers, TF Lite, Protobuf)
323. **```build-armnn.sh```:** builds Arm NN and ACL
334. **```common.sh```:** common variables and functions used by ```setup-armnn.sh``` and ```build-armnn.sh```
345. **```validation.sh```:** validation of common command-line arguments to ```setup-armnn.sh``` and ```build-armnn.sh```
356. **```docker-copy-to-host```:** helpful script to copy contents of a Docker Image to the host machine
36
37A **Dockerfile** is a text document which outlines the instructions for building a **Docker Image**. Running ```docker build```
38on the Arm NN Dockerfile will result in a Docker Image that contains:
391. The Ubuntu operating system
402. Installed system-wide packages required by Arm NN
413. Source code for Arm NN and its dependencies
424. The build for Arm NN and its dependencies
43
44All scripts except ```docker-copy-to-host.sh``` are used during the ```docker build``` process.
45
46The ```docker-copy-to-host.sh``` script creates a temporary [**Docker Container**](https://www.docker.com/resources/what-container/) which is required to retrieve files from the
47Docker Image. A Docker Container is a **runtime** version of a Docker Image. It can be interacted with through a command-line
48terminal, like a lightweight virtual machine (VM). The ```docker-copy-to-host.sh``` script is designed to run on the host
49machine after ```docker build``` to copy the Arm NN build to the host.
50
51The **diagram below** outlines the operation of the Arm NN **Dockerfile** and the contents of the resulting **Docker Image** after
52running ```docker build```. During ```docker build```, the provided **```SETUP_ARGS```** determine which of Arm NN's
53dependencies to download and build. The **```BUILD_ARGS```** Docker argument determines which components of Arm NN to build.
54
55The resulting Docker Image contains the Arm NN build (along with everything required to build it) in the home directory ```/home/arm-user```.
56There are two directories: **```source```** which contains the source code of Arm NN and its dependencies, and **```build```**
57which contains the Arm NN build and the builds of each dependency of Arm NN.
58
59The **```armnn_aarch64_build.tar.gz``` tarball** in the home directory is a compressed archive of the Arm NN build which is located at ```/home/arm-user/build/armnn```.
60The home directory also contains the **scripts** mentioned above (not pictured in the diagram), which are copied into the
61Docker Image during the ```docker build``` process.
62
63<div align="center">
64 <img src="dockerfile_armnn.png" class="center" alt="Arm NN Dockerfile" width="800"/>
65</div>
66
67<br>
68
69**Benefits of using Docker:**
701. Run one ```docker build``` command instead of executing multiple scripts, saving developer time
712. Can easily manage multiple different build configurations at once, each in their own Docker Image
723. System packages are installed in the Docker Image only, isolated from the host machine
734. Build in the exact environment the Arm NN Team used during testing, ensuring consistency in builds
745. Allows Arm NN to be built on a wide range of hosts, as long as they support Docker
75
76**Disadvantages of using Docker:**
771. Requires the developer to install Docker on their host machine
782. Requires the developer to get familiar with Docker (this guide should help)
793. Possible incompatibilities/limitations if the host machine's Linux Kernel version is too different than that of the
80base OS in the Docker Image. This is due to the fact that Docker Containers share the Linux Kernel of the host machine
81(unlike VM's which each have their own Kernel).
82
83<br>
84
85## System Requirements
86This guide will not provide the system requirements for installing Docker - please refer to the Docker documentation during installation.
87
88The built Docker Image has disk requirements of **less than 6GB**, depending on the Arm NN components chosen and whether
89the build included a cross compile (i.e. building aarch64 on x86_64). Repeated builds may cause the total disk usage to increase
90beyond 6GB. The ```docker images``` command shows the disk usage of each Docker Image. To view total disk space used by all
91Docker Images and Docker Containers, use the command ```docker system df```.
92
93This Dockerfile was built on a **Ubuntu 18.04 host machine with Docker version 20.10.14**. We recommend using a host machine
94with at least Ubuntu 18.04 and a similar Docker version, if possible. Other Linux distros such as **Debian** should also work as
95long as the Docker version is similar to **20.10.14**.
96
97The build outputs from this Dockerfile have been tested on an **Odroid N2+ target device with Ubuntu 18.04** installed. The build outputs
98should also work on newer versions of Ubuntu (and Debian) but this has not been exhaustively tested.
99
100To view the system packages that are installed during ```docker build```, please refer to the ```install-packages.sh``` script.
101
102<br>
103
104## Install Docker
105Builds of the official Arm NN Dockerfile have been tested on the **Ubuntu 18.04** host platform.
106Whilst other platforms have not been tested, Docker should be able to build the Arm NN Dockerfile on other platforms such as Windows and Mac.
107On **Linux**, we recommend using **Docker Engine** which is used through the command-line interface (CLI).
108For **Windows and Mac** users, **Docker Desktop** is available which allows the use of the CLI plus a graphical interface.<br>
109This guide will **only** provide steps on using Docker through the use of the CLI and not Docker Desktop.
110
111### Ubuntu / Linux
112For Ubuntu users, we recommend installing **Docker Engine** with the following steps from the Docker documentation:<br>
1131. [Uninstall any old versions of Docker](https://docs.docker.com/engine/install/ubuntu/#uninstall-old-versions)
1142. [Install using the repository](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)
1153. [Add user to Docker Unix Group](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user) (**recommended**: allows running of Docker commands **without ```sudo```**)
116
117Similar steps for installing Docker Engine on other Linux distributions can be found by clicking on your distro under the Server table on [this page](https://docs.docker.com/engine/install/#server).
118
119### Windows and Mac
120We do not officially test the building of the Arm NN Dockerfile on Windows or Mac. However, it's likely that it will work if your system supports Docker.
121For **Windows**, use the Docker Desktop download link on [this page](https://docs.docker.com/desktop/install/windows-install/).
122For **Mac** users, please use the Docker Desktop download link on [this page](https://docs.docker.com/desktop/install/mac-install/). <br>
123**Note:** we unfortunately cannot provide support for the installation of Docker and/or the use of the Arm NN Dockerfile on Windows and Mac platforms.
124
125<br>
126
127## Docker Build Arguments Reference
128The following describes the ```docker build``` arguments that can be used to customize the Arm NN build.
129Docker build arguments such as ```SETUP_ARGS``` and ```BUILD_ARGS``` are provided in the ```docker build``` as a ```--build-arg```.
130
131### SETUP_ARGS
132These arguments are eventually passed to ```setup-armnn.sh``` which downloads and builds Arm NN dependencies.
133For ease of use (but longer initial docker build), use ```--all``` to have all Arm NN dependencies available for use during Arm NN builds.
134Repeated docker builds with the same ```SETUP_ARGS``` will skip the setup process (using [caching](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache) of previous Docker build steps).
135The ```SETUP_ARGS``` string should start and end with **double quotes** ```"```.
136
137| SETUP_ARGS | Description |
138|-------------------|:---------------------------------------------------------------------------------|
139| --tflite-delegate | **flag:** setup dependencies for the Arm NN TF Lite Delegate |
140| --tflite-parser | **flag:** setup dependencies for the Arm NN TF Lite Parser |
141| --onnx-parser | **flag:** setup dependencies for the Arm NN ONNX parser |
142| --all | **flag:** setup dependencies for all Arm NN components listed above |
143| --target-arch= | **mandatory option:** specify a target architecture ```aarch64, x86_64``` |
144
145**At least one component** (e.g. ```--tflite-delegate```) must be provided **or** else provide ```--all``` to setup dependencies for all components.
146
147**Examples:**<br>
148Setup for aarch64 with all Arm NN dependencies:<br>
149```SETUP_ARGS="--target-arch=aarch64 --all"```
150
151Setup for aarch64 with TF Lite Delegate and TF Lite Parser dependencies only:<br>
152```SETUP_ARGS="--target-arch=aarch64 --tflite-delegate --tflite-parser"```
153
154<br>
155
156### BUILD_ARGS
157The following arguments are given to ```build-armnn.sh``` and define which components of Arm NN to include in the build.
158The ```BUILD_ARGS``` string should start and end with **double quotes** ```"```.
159
160| BUILD_ARGS | Description |
161|---------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
162| --tflite-delegate | **flag:** build the Arm NN TF Lite Delegate component |
163| --tflite-parser | **flag:** build the Arm NN TF Lite Parser component |
164| --onnx-parser | **flag:** build the Arm NN ONNX parser component |
165| --all | **flag:** build all Arm NN components listed above |
166| --target-arch= | **mandatory option:** specify a target architecture ```aarch64, x86_64``` |
167| --neon-backend | **flag:** build Arm NN with the NEON backend (CPU acceleration from ACL) |
168| --cl-backend | **flag:** build Arm NN with the OpenCL backend (GPU acceleration from ACL) |
169| --ref-backend | **flag:** build Arm NN with the reference backend<br/>**Should be used for verification purposes only.<br/>Does not provide any performance acceleration.** |
170| --debug | **flag:** build Arm NN (and ACL) with debug turned on (optional: defaults to off) |
171| --armnn-cmake-args= | **option:** provide additional comma-separated CMake arguments string for building Arm NN (optional)<br/>String should start and end with **single quotes** ```'```<br/>Please refer to **armnn/cmake/GlobalConfig.cmake** |
172| --acl-scons-params= | **option**: provide additional comma-separated scons parameters string for building ACL (optional)<br/>String should start and end with **single quotes** ```'```<br/>ACL provide [documentation](https://arm-software.github.io/ComputeLibrary/latest/how_to_build.xhtml#S1_1_build_options) for their build options |
173
174
175**At least one component** (i.e. ```--tflite-delegate```, ```--tflite-parser```, ```--onnx-parser```) must be provided or else provide ```--all``` to build all Arm NN components.<br>
176**A component given in ```BUILD_ARGS``` must also have been given in ```SETUP_ARGS``` previously**, or else Arm NN will fail to build.<br>
177**At least one backend** (i.e. ```--neon-backend```, ```--cl-backend```, ```--ref-backend```) must be chosen.<br>
178
179**Examples:**<br>
180Build for aarch64 with all Arm NN components, NEON enabled and OpenCL enabled:<br>
181```BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend"```
182
183Build for aarch64 with TF Lite Delegate, OpenCL enabled and additional ACL scons params:<br>
184```BUILD_ARGS="--target-arch=aarch64 --tflite-delegate --cl-backend --acl-scons-params='compress_kernels=1,benchmark_examples=1'"```
185
186Setup for aarch64 with all Arm NN dependencies, OpenCL enabled and additional Arm NN cmake args:<br>
187```BUILD_ARGS="--target-arch=aarch64 --all --cl-backend --armnn-cmake-args='-DBUILD_SAMPLE_APP=1,-DBUILD_UNIT_TESTS=0'"```
188
189**Example _valid_ combination of SETUP_ARGS and BUILD_ARGS:**<br>
190```
191SETUP_ARGS="--target-arch=aarch64 --all"
192BUILD_ARGS="--target-arch=aarch64 --tflite-delegate --neon-backend --cl-backend"
193```
194
195**Example _invalid_ combination of SETUP_ARGS and BUILD_ARGS:**<br>
196```
197SETUP_ARGS="--target-arch=aarch64 --tflite-delegate"
198BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend"
199```
200
201The example above is invalid since it tries to build all Arm NN components after **only** building the dependencies required by the TF Lite Delegate.
202
203<br>
204
205## Docker Build Steps
206
207First clone the Arm NN repo to obtain the build-tool. This automatically downloads the latest release version of Arm NN.
208```
209git clone https://github.com/ARM-software/armnn.git armnn
210```
211<br>
212
213Enter into the Arm NN build-tool directory where the Dockerfile and associated scripts are located.
214```
215cd armnn/build-tool
216```
217<br>
218
219Run the ```docker build```, which downloads and builds Arm NN and its dependencies. This process is isolated from the host machine's filesystem and the result is a Docker Image being created.<br>
220Dockerfile arguments are provided using ```--build-arg```, the relative path to the Dockerfile is given with ```--file``` and the current directory is given with ```.```<br>
221Provide a descriptive name for the Image with ```--tag``` in the form of ```image_name:tag``` (example given below). The backslashes ```\``` tell Bash to expect a continuation of the command on the next line.
222
223This example's choice of **[SETUP_ARGS](#setup_args)** and **[BUILD_ARGS](#build_args)** builds all Arm NN components with NEON and OpenCL accelerated backends for aarch64 target.
224This process should take **less than an hour** with a modern machine but will vary depending on the arguments chosen and host machine specification.
225
226```
227docker build \
228--build-arg SETUP_ARGS="--target-arch=aarch64 --all" \
229--build-arg BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend" \
230--tag armnn:aarch64 \
231--file docker/Dockerfile .
232```
233
234<br>
235
236Arm NN and all of its dependencies are now built within a Docker Image. Built Docker images can be listed with ```docker images```.
237```
238docker images
239
240REPOSITORY TAG IMAGE ID CREATED SIZE
241armnn aarch64 cc623174fd98 7 seconds ago 5.02GB
242```
243
244<br>
245
246A tarball archive of the Arm NN build is located inside the Docker home directory (```/home/arm-user/```), named ```armnn_<target-arch>_build.tar.gz```.
247If the ```--debug``` build flag is provided in the ```BUILD_ARGS``` above, the tarball will be named ```armnn_<target-arch>_build_debug.tar.gz```.<br>
248The ```docker-copy-to-host.sh``` script will copy a file from the Docker Image (in arm-user's home directory) to the host machine.<br>
249The script copies the tarball into a new directory on the host in ```build-tool/docker_output```. It takes two arguments: ```image_name:tag``` and ```filename```.<br>
250The filename is a relative path from the home directory created inside the Docker Image (```/home/arm-user/```).
251```
252./scripts/docker-copy-to-host.sh armnn:aarch64 armnn_aarch64_build.tar.gz
253```
254
255<br>
256
257This tarball can now be used for integration in an ML application. The method of extraction is given below.<br>
258If the ```--target-arch``` chosen in the arguments above matches the host machine, the build can be tested locally (else copy the tarball to a remote device).
259```
260cd docker_output
261
262# Extract the tarball into a directory called <target_arch>_build
263# If --debug is enabled, the extracted build directory will be called <target_arch>_build_debug
264tar -xzf armnn_aarch64_build.tar.gz
265cd aarch64_build
266
267# Set LD_LIBRARY_PATH to the current aarch64_build directory (.)
268export LD_LIBRARY_PATH=.; ./UnitTests
269
270# If the Arm NN TF Lite Delegate is built, we can also run DelegateUnitTests
271cd delegate
272
273# Set LD_LIBRARY_PATH to the current delegate directory (.) and the aarch64_build directory (..)
274export LD_LIBRARY_PATH=.:..; ./DelegateUnitTests
275```
276
277<br>
278
279**Note:** Repeated Docker builds may result in an accumulation of unwanted, dangling images. **To remove dangling images** (good practice), run the following command: ```docker image prune```
280
281<br>
282
283## Build Arm NN without Docker
284For ease of use, we recommend that the official Arm NN Dockerfile is used to build Arm NN.
285If you'd like to build locally on a Ubuntu host machine without Docker, execute the build-tool scripts directly instead.
286This involves running ```install-packages.sh``` followed by ```setup-armnn.sh``` and ```build-armnn.sh```.
287The arguments for ```setup-armnn.sh``` and ```build-armnn.sh``` are very much the same as ```SETUP_ARGS``` and ```BUILD_ARGS```.
288Please use ```--help``` for more specific information.
289
290```
291cd build-tool/scripts
292
293sudo ./install-packages.sh
294
295./setup-armnn.sh --target-arch=aarch64 --all
296./build-armnn.sh --target-arch=aarch64 --all --neon-backend --cl-backend
297
298./setup-armnn.sh --help
299./build-armnn.sh --help
300```
301
302<br>
303
304## Advanced Usage
305
306### Use custom Arm NN and ACL repository versions during build
307By default, the docker build process (specifically, during ```build-armnn.sh```) will download the latest release versions of Arm NN and ACL.
308If you'd like to use different versions during the build, check them out in the ```build-tool``` directory on the host.<br>
309When providing custom repositories, the following ```docker build``` argument must be provided ```--build-arg BUILD_TYPE=dev```.
310This will trigger Docker to copy the custom repos into the Docker Image during build. The ACL repo is only required if
311supplying the ```--neon-backend``` or ```--cl-backend``` BUILD_ARGS options.
312
313**Note:** the Arm NN version used for build-tool (Dockerfile and scripts) is not the same version of Arm NN that is used during the build.
314This means that separate versions of Arm NN can be used for the build-tool and for building Arm NN itself.<br>
315If you repeat the ```docker build``` process multiple times (e.g. with different ```BUILD_ARGS```), the setup process will be skipped as long as ```SETUP_ARGS``` remains the same.
316```
317cd build-tool
318
319git clone https://github.com/ARM-software/armnn.git armnn
320cd armnn
321git checkout <branch or SHA>
322
323cd ..
324
325# custom ACL repo only required when supplying --neon-backend or --cl-backend BUILD_ARGS options
326git clone https://github.com/ARM-software/ComputeLibrary.git acl
327cd acl
328git checkout <tag or SHA>
329
330cd ..
331
332# Example docker build with BUILD_TYPE=dev, ran inside the build-tool directory
333docker build \
334--build-arg BUILD_TYPE=dev \
335--build-arg SETUP_ARGS="--target-arch=aarch64 --all" \
336--build-arg BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend" \
337--tag armnn:aarch64 \
338--file docker/Dockerfile .
339```
340
341<br>
342
343### Additional Docker Build Arguments
344
345#### UBUNTU_VERSION
346The default base Image used during ```docker build``` is ```ubuntu:18.04```. Building Arm NN with this default image should be sufficient for a wide range of target devices.
347To use a different Ubuntu base Image, provide ```UBUNTU_VERSION``` during ```docker build``` e.g. ```--build-arg UBUNTU_VERSION=20.04```.
348
349#### BUILD_TYPE
350By default, ```BUILD_TYPE``` is set to ```production```. This means that the latest release versions of Arm NN and ACL are used during builds.
351To use custom repositories as explained [above](#Use-custom-Arm-NN-and-ACL-repository-versions-during-build), provide ```--build-arg BUILD_TYPE=dev```.
352
353#### Additional SETUP_ARGS and BUILD_ARGS
354The following are additional ```SETUP_ARGS``` and ```BUILD_ARGS``` that can be provided to the ```docker build```.
355
356| SETUP_ARGS / BUILD_ARGS | Description |
357|-------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------|
358| --num-threads= | **option:** specify number of threads/cores to build Arm NN and its dependencies with (this defaults to number of online cores on the host)<br/>e.g. --num-threads=4 |
359| -x | **flag:** enable shell tracing during setup-armnn.sh and build-armnn.sh script execution |
360
361#### USER_ID and GROUP_ID
362By default, the Docker Image creates a user called ```arm-user``` with user/group ID of 1000. This ensures the build outputs can be used by a wide range of users on target devices.<br>
363To provide a more specific user and group ID to be associated with the build, provide ```--build-arg USER_ID=XXXX``` and ```--build-arg GROUP_ID=YYYY```.
364
365<br>
366
367### Inspect the contents of the Docker Image
368To inspect the full contents of the Docker Image, the Image must be run as a **Docker Container**.
369A Container is a runtime version of the Image, like a **lightweight virtual machine** with its own filesystem.
370
371The following shows how to run a container with an interactive terminal based on a specified Docker Image ```image_name:tag```.
372The ID of the generated container in this example is ```4da7c575a95a```. An ```ls``` command shows the contents of the ```/home/arm-user``` directory.
373```
374docker run --interactive --tty armnn:aarch64
375
376arm-user@4da7c575a95a:~$ ls
377armnn_aarch64_build.tar.gz build build-armnn.sh common.sh setup-armnn.sh source validation.sh
378```
379The ```source``` directory contains the downloaded dependencies for building Arm NN.<br>
380The ```build``` directory contains the built Arm NN dependencies and the Arm NN build itself.<br>
381The ```armnn_aarch64_build.tar.gz``` tarball is an archive of the build located at ```build/armnn/aarch64_build```.<br>
382The scripts (e.g. ```build-armnn.sh```) in this directory can be run as you wish, changes will persist within the container but not the Docker Image.
383To exit the docker container, run ```CTRL-D```. A list of Docker containers can be obtained with the following command:
384```
385# --all shows both running and non-running containers
386docker ps --all
387
388CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3894da7c575a95a armnn:aarch64 "bash" 7 minutes ago Exited (0) 7 seconds ago pedantic_joliot
390```
391To restart the same container (and any changes made to it) in a terminal again, use the following commands:
392```
393# 4da7c575a95a in this case is the CONTAINER ID from above
394
395docker start 4da7c575a95a # restart container in background
396docker attach 4da7c575a95a # re-attach to terminal
397
398# After exiting with CTRL-D
399docker stop 4da7c575a95a
400```
401To save space over time, stopped Docker containers can be deleted with the following command:<br>
402```docker container prune```.
403
404<br>
405
406### Bind Mounts
407**Bind Mounts** may be used to mount directories from the host machine to inside a Docker container. This could be useful when
408frequently changing the contents of the Arm NN or ACL source repositories (in ```/home/arm-user/source```) between runs of
409build-armnn.sh inside the container. Note that bind mounts are a feature that apply to runtime versions of Docker
410images i.e. Docker Containers (```docker run```, not ```docker build```).
411Please refer to the [Docker documentation for more information about bind mounts](https://docs.docker.com/storage/bind-mounts/).
