1Integration guide 2================= 3 4This document describes how to integrate the Arm NN Android NNAPI driver into an Android source tree. 5 6### Prerequisites 7 81. Android source tree for Android P (we have tested against Android P version 9.0.0_r3) , in the directory `<ANDROID_ROOT>` 92. Android source tree for Android Q (we have tested against Android Q version 10.0.0_r39), in the directory `<ANDROID_ROOT>` 102. Android source tree for Android R (we have tested against Android R version 11.0.0_r3), in the directory `<ANDROID_ROOT>` 113. Mali OpenCL driver integrated into the Android source tree 12 13### Procedure 14 151. Place this source directory at `<ANDROID_ROOT>/vendor/arm/android-nn-driver` 162. Run setup.sh 173. Update the Android build environment to add the ArmNN driver. This ensures that the driver service 18is built and copied to the `system/vendor/bin/hw` directory in the Android image. 19To update the build environment, add to the contents of the variable `PRODUCT_PACKAGES` 20within the device-specific makefile that is located in the `<ANDROID_ROOT>/device/<manufacturer>/<product>` 21directory. This file is normally called `device.mk`: 22 23For Android P, Q or R, using NN API version (1.0), the following should be added to `device.mk`: 24<pre> 25PRODUCT_PACKAGES += android.hardware.neuralnetworks@1.0-service-armnn 26</pre> 27 28For Android P, Q or R, a new version of the NN API is available (1.1), 29thus the following should be added to `device.mk` instead: 30<pre> 31PRODUCT_PACKAGES += android.hardware.neuralnetworks@1.1-service-armnn 32</pre> `Android.mk` contains the module definition of both versions of the ArmNN driver. 33 34For Android Q, a new version of the NN API is available (1.2), 35thus the following should be added to `device.mk` instead: 36<pre> 37PRODUCT_PACKAGES += android.hardware.neuralnetworks@1.2-service-armnn 38</pre> `Android.mk` contains the module definition of all three versions (1.0, 1.1 and 1.2) of the ArmNN driver. 39 40For Android R, new version of the NN API is available (1.3), 41thus the following should be added to `device.mk` instead: 42<pre> 43PRODUCT_PACKAGES += android.hardware.neuralnetworks@1.3-service-armnn 44</pre> 45`Android.mk` contains the module definition of all versions (1.0, 1.1, 1.2 and 1.3) of the ArmNN driver. 46 47Similarly, the Neon, CL or reference backend can be enabled/disabled by setting ARMNN_COMPUTE_CL_ENABLE, 48ARMNN_COMPUTE_NEON_ENABLE or ARMNN_REF_ENABLE in `device.mk`: 49<pre> 50ARMNN_COMPUTE_CL_ENABLE := 1 51</pre> 52 53For Android P, Q and R the vendor manifest.xml requires the Neural Network HAL information. 54For Android P use HAL version 1.1 as below. For Android Q substitute 1.2 where necessary. For Android R substitute 1.3 where necessary. 55```xml 56<hal format="hidl"> 57 <name>android.hardware.neuralnetworks</name> 58 <transport>hwbinder</transport> 59 <version>1.1</version> 60 <interface> 61 <name>IDevice</name> 62 <instance>armnn</instance> 63 </interface> 64 <fqname>@1.1::IDevice/armnn</fqname> 65</hal> 66``` 67 684. Build Android as normal, i.e. run `make` in `<ANDROID_ROOT>` 695. To confirm that the ArmNN driver has been built, check for driver service executable at 70 71Android P 72<pre> 73<ANDROID_ROOT>/out/target/product/<product>/system/vendor/bin/hw 74</pre> 75For example, if the ArmNN driver has been built with the NN API 1.0, check for the following file: 76<pre> 77<ANDROID_ROOT>/out/target/product/<product>/system/vendor/bin/hw/android.hardware.neuralnetworks@1.0-service-armnn 78</pre> 79 80Android Q and later has a different path: 81<pre> 82<ANDROID_ROOT>/out/target/product/<product>/vendor/bin/hw 83</pre> 84 85### Testing 86 871. Run the ArmNN driver service executable in the background. 88Use the corresponding version of the driver for the Android version you are running. 89i.e 90android.hardware.neuralnetworks@1.1-service-armnn for Android P, 91android.hardware.neuralnetworks@1.2-service-armnn for Android Q and 92android.hardware.neuralnetworks@1.3-service-armnn for Android R 93<pre> 94It is also possible to use a specific backend by using the -c option. 95The following is an example of using the CpuAcc backend for Android Q: 96adb shell /system/vendor/bin/hw/android.hardware.neuralnetworks@1.2-service-armnn -c CpuAcc & 97</pre> 982. Run some code that exercises the Android Neural Networks API, for example Android's 99`NeuralNetworksTest` unit tests (note this is an optional component that must be built). 100<pre> 101adb shell /data/nativetest/NeuralNetworksTest_static/NeuralNetworksTest_static > NeuralNetworkTest.log 102</pre> 1033. To confirm that the ArmNN driver is being used to service the Android Neural Networks API requests, 104check for messages in logcat with the `ArmnnDriver` tag. 105 106### Using the GPU tuner 107 108The GPU tuner is a feature of the Compute Library that finds optimum values for GPU acceleration tuning parameters. 109There are three levels of tuning: exhaustive, normal and rapid. 110Exhaustive means that all lws values are tested. 111Normal means that a reduced number of lws values are tested, but that generally is sufficient to have a performance close enough to the exhaustive approach. 112Rapid means that only 3 lws values should be tested for each kernel. 113The recommended way of using it with ArmNN is to generate the tuning data during development of the Android image for a device, and use it in read-only mode during normal operation: 114 1151. Run the ArmNN driver service executable in tuning mode. The path to the tuning data must be writable by the service. 116The following examples assume that the 1.0 version of the driver is being used: 117<pre> 118adb shell /system/vendor/bin/hw/android.hardware.neuralnetworks@1.0-service-armnn --cl-tuned-parameters-file <PATH_TO_TUNING_DATA> --cl-tuned-parameters-mode UpdateTunedParameters --cl-tuning-level exhaustive & 119</pre> 1202. Run a representative set of Android NNAPI testing loads. In this mode of operation, each NNAPI workload will be slow the first time it is executed, as the tuning parameters are being selected. Subsequent executions will use the tuning data which has been generated. 1213. Stop the service. 1224. Deploy the tuned parameters file to a location readable by the ArmNN driver service (for example, to a location within /vendor/etc). 1235. During normal operation, pass the location of the tuning data to the driver service (this would normally be done by passing arguments via Android init in the service .rc definition): 124<pre> 125adb shell /system/vendor/bin/hw/android.hardware.neuralnetworks@1.0-service-armnn --cl-tuned-parameters-file <PATH_TO_TUNING_DATA> & 126</pre> 127