1/// Copyright (c) 2020 ARM Limited. 2/// 3/// SPDX-License-Identifier: MIT 4/// 5/// Permission is hereby granted, free of charge, to any person obtaining a copy 6/// of this software and associated documentation files (the "Software"), to deal 7/// in the Software without restriction, including without limitation the rights 8/// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9/// copies of the Software, and to permit persons to whom the Software is 10/// furnished to do so, subject to the following conditions: 11/// 12/// The above copyright notice and this permission notice shall be included in all 13/// copies or substantial portions of the Software. 14/// 15/// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16/// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17/// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18/// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19/// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20/// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21/// SOFTWARE. 22/// 23 24namespace armnn 25{ 26/** 27@page backends Backend Developer Guides 28@tableofcontents 29 30@section S12_backend_developer_guide Backend Developer Guide 31 32Arm NN allows adding new backends through the `Pluggable Backend` mechanism. 33 34@subsection S12_1_backend_developer_guide How to add a new backend 35 36Backends reside under [src/backends](./), in separate subfolders. For Linux builds they must have a `backend.cmake` file, 37which is read automatically by [src/backends/backends.cmake](backends.cmake). The `backend.cmake` file 38under the backend-specific folder is then included by the main CMakeLists.txt file at the root of the 39Arm NN source tree. 40 41### The backend.cmake file 42 43The `backend.cmake` has three main purposes: 44 451. It makes sure the artifact (a cmake OBJECT library) is linked into the Arm NN shared library by appending the name of the library to the `armnnLibraries` list. 462. It makes sure that the subdirectory where backend sources reside gets included into the build. 473. To include backend-specific unit tests, the object library for the unit tests needs to be added to the `armnnUnitTestLibraries` list. 48 49Example `backend.cmake` file taken from [reference/backend.cmake](reference/backend.cmake): 50 51~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.cmake 52# 53# Make sure the reference backend is included in the build. 54# By adding the subdirectory, cmake requires the presence of CMakeLists.txt 55# in the reference (backend) folder. 56# 57add_subdirectory(${PROJECT_SOURCE_DIR}/src/backends/reference) 58 59# 60# Add the cmake OBJECT libraries built by the reference backend to the 61# list of libraries linked against the Arm NN shared library. 62# 63list(APPEND armnnLibraries armnnRefBackend armnnRefBackendWorkloads) 64 65# 66# Backend specific unit tests can be integrated through the 67# armnnUnitTestLibraries variable. This makes sure that the 68# UnitTests executable can run the backend-specific unit 69# tests. 70# 71list(APPEND armnnUnitTestLibraries armnnRefBackendUnitTests) 72~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 73 74### The CMakeLists.txt file 75 76As described in the previous section, adding a new backend will require creating a `CMakeLists.txt` in 77the backend folder. This follows the standard cmake conventions, and is required to build a static cmake OBJECT library 78to be linked into the Arm NN shared library. As with any cmake build, the code can be structured into 79subfolders and modules as the developer sees fit. 80 81Example can be found under [reference/CMakeLists.txt](reference/CMakeLists.txt). 82 83### The backend.mk file 84 85Arm NN on Android uses the native Android build system. New backends are integrated by creating a 86`backend.mk` file, which has a single variable called `BACKEND_SOURCES` listing all cpp 87files to be built by the Android build system for the Arm NN shared library. 88 89Optionally, backend-specific unit tests can be added similarly, by 90appending the list of cpp files to the `BACKEND_TEST_SOURCES` variable. 91 92Example taken from [reference/backend.mk](reference/backend.mk): 93 94~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.make 95BACKEND_SOURCES := \ 96 RefLayerSupport.cpp \ 97 RefWorkloadFactory.cpp \ 98 workloads/Activation.cpp \ 99 workloads/ElementwiseFunction.cpp \ 100 workloads/Broadcast.cpp \ 101 ... 102 103BACKEND_TEST_SOURCES := \ 104 test/RefCreateWorkloadTests.cpp \ 105 test/RefEndToEndTests.cpp \ 106 test/RefJsonPrinterTests.cpp \ 107 ... 108~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 109 110@subsection S12_2_backend_developer_guide How to Add Common Code Across Backends 111 112For multiple backends that need common code, there is support for including them in the build 113similarly to the backend code. This requires adding three files under a subfolder at the same level 114as the backends folders. These are: 115 1161. common.cmake 1172. common.mk 1183. CMakeLists.txt 119 120They work the same way as the backend files. The only difference between them is that 121common code is built first, so the backend code can depend on them. 122 123[aclCommon](aclCommon) is an example for this concept and you can find the corresponding files: 124 1251. [aclCommon/common.cmake](aclCommon/common.cmake) 1262. [aclCommon/common.mk](aclCommon/common.mk) 1273. [aclCommon/CMakeLists.txt](aclCommon/CMakeLists.txt) 128 129@subsection S12_3_backend_developer_guide Identifying Backends 130 131Backends are identified by a string that must be unique across backends. This string is 132wrapped in the [BackendId](../../include/armnn/BackendId.hpp) object for backward compatibility 133with previous Arm NN versions. 134 135@subsection S12_4_backend_developer_guide The IBackendInteral Interface 136 137All backends need to implement the [IBackendInternal](../../include/armnn/backends/IBackendInternal.hpp) interface. 138The interface functions to be implemented are: 139 140~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.cpp 141 virtual IMemoryManagerUniquePtr CreateMemoryManager() const = 0; 142 virtual IWorkloadFactoryPtr CreateWorkloadFactory( 143 const IMemoryManagerSharedPtr& memoryManager = nullptr) const = 0; 144 virtual IBackendContextPtr CreateBackendContext(const IRuntime::CreationOptions&) const = 0; 145 virtual IBackendProfilingContextPtr CreateBackendProfilingContext(const IRuntime::CreationOptions& creationOptions, 146 armnn::profiling::IBackendProfiling& backendProfiling) const = 0; 147 virtual ILayerSupportSharedPtr GetLayerSupport() const = 0; 148 virtual Optimizations GetOptimizations() const = 0; 149 virtual SubgraphUniquePtr OptimizeSubgraph(const SubgraphView& subgraph, bool& optimizationAttempted) const; 150 virtual OptimizationViews OptimizeSubgraphView(const SubgraphView& subgraph) const; 151~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 152 153Note that `GetOptimizations()` and `SubgraphViewUniquePtr OptimizeSubgraphView(const SubgraphView& subgraph, bool& optimizationAttempted)` 154have been deprecated. 155The method `OptimizationViews OptimizeSubgraph(const SubgraphView& subgraph)` should be used instead to 156apply specific optimizations to a given sub-graph. 157 158The Arm NN framework then creates instances of the IBackendInternal interface with the help of the 159[BackendRegistry](../../include/armnn/BackendRegistry.hpp) singleton. 160 161**Important:** the `IBackendInternal` object is not guaranteed to have a longer lifetime than 162the objects it creates. It is only intended to be a single entry point for the factory functions it has. 163The best use of this is to be a lightweight, stateless object and make no assumptions between 164its lifetime and the lifetime of the objects it creates. 165 166For each backend one needs to register a factory function that can 167be retrieved using a [BackendId](../../include/armnn/BackendId.hpp). 168The Arm NN framework creates the backend interfaces dynamically when 169it sees fit and it keeps these objects for a short period of time. Examples: 170 171- During optimization Arm NN needs to decide which layers are supported by the backend. 172 To do this, it creates a backends and calls the `GetLayerSupport()` function and creates 173 an `ILayerSupport` object to help deciding this. 174- During optimization Arm NN can run backend-specific optimizations. After splitting the graph into 175 sub-graphs based on backends, it calls the `OptimizeSubgraphView()` function on each of them and, if possible, 176 substitutes the corresponding sub-graph in the original graph with its optimized version. 177- When the Runtime is initialized it creates an optional `IBackendContext` object and keeps this context alive 178 for the Runtime's lifetime. It notifies this context object before and after a network is loaded or unloaded. 179- When the LoadedNetwork creates the backend-specific workloads for the layers, it creates a backend 180 specific workload factory and calls this to create the workloads. 181 182@subsection S12_5_backend_developer_guide The BackendRegistry 183 184As mentioned above, all backends need to be registered through the BackendRegistry so Arm NN knows 185about them. Registration requires a unique backend ID string and a lambda function that 186returns a unique pointer to the [IBackendInternal interface](../../include/armnn/backends/IBackendInternal.hpp). 187 188For registering a backend only this lambda function needs to exist, not the actual backend. This 189allows dynamically creating the backend objects when they are needed. 190 191The BackendRegistry has a few convenience functions, like we can query the registered backends and 192are able to tell if a given backend is registered or not. 193 194Dynamic backends are registered during the runtime creation. 195 196@subsection S12_6_backend_developer_guide The ILayerSupport Interface 197 198Arm NN uses the [ILayerSupport](../../include/armnn/ILayerSupport.hpp) interface to decide if a layer 199with a set of parameters (i.e. input and output tensors, descriptor, weights, filter, kernel if any) are 200supported on a given backend. The backends need a way to communicate this information by implementing 201the `GetLayerSupport()` function on the `IBackendInternal` interface. 202 203Examples of this can be found in the [RefLayerSupport header](reference/RefLayerSupport.hpp) 204and the [RefLayerSupport implementation](reference/RefLayerSupport.cpp). 205 206@subsection S12_7_backend_developer_guide The IWorkloadFactory Interface 207 208The [IWorkloadFactory interface](backendsCommon/WorkloadFactory.hpp) is used for creating the backend 209specific workloads. The factory function that creates the IWorkloadFactory object in the IBackendInterface 210takes an IMemoryManager object. 211 212To create a workload object the `IWorkloadFactory` takes a `WorkloadInfo` object that holds 213the input and output tensor information and a workload specific queue descriptor. 214 215@subsection S12_8_backend_developer_guide The IMemoryManager Interface 216 217Backends may choose to implement custom memory management. Arm NN supports this concept through the following 218mechanism: 219 220- the `IBackendInternal` interface has a `CreateMemoryManager()` function, which is called before 221 creating the workload factory 222- the memory manager is passed to the `CreateWorkloadFactory(...)` function so the workload factory can 223 use it for creating the backend-specific workloads 224- the LoadedNetwork calls `Acquire()` on the memory manager before it starts executing the network and 225 it calls `Release()` in its destructor 226 227@subsection S12_9_backend_developer_guide The Optimizations 228 229The backends may choose to implement backend-specific optimizations. 230This is supported through the method `OptimizationViews OptimizeSubgraph(const SubgraphView& subgraph)` of 231the backend interface that allows the backends to apply their specific optimizations to a given sub-graph. 232 233The `OptimizeSubgraph(...)` method returns an OptimizationViews object containing three lists: 234 235- A list of the sub-graph substitutions: a "substitution" is a pair of sub-graphs, the first is the "substitutable" sub-graph, 236 representing the part of the original graph that has been optimized by the backend, while the second is the "replacement" sub-graph, 237 containing the actual optimized layers that will be replaced in the original graph correspondingly to the "substitutable" sub-graph 238- A list of the failed sub-graphs: these are the parts of the original sub-graph that are not supported by the backend, 239 thus have been rejected. Arm NN will try to re-allocate these parts on other backends if available. 240- A list of the untouched sub-graphs: these are the parts of the original sub-graph that have not been optimized, 241 but that can run (unoptimized) on the backend. 242 243The previous way backends had to provide a list optimizations to the Optimizer (through the `GetOptimizations()` method) 244is still in place for backward compatibility, but it's now considered deprecated and will be remove in a future release. 245 246@subsection S12_10_backend_developer_guide The IBackendContext Interface 247 248Backends may need to be notified whenever a network is loaded or unloaded. To support that, one can implement the optional 249[IBackendContext](../../include/armnn/backends/IBackendContext.hpp) interface. The framework calls the `CreateBackendContext(...)` 250method for each backend in the Runtime. If the backend returns a valid unique pointer to a backend context, then the 251runtime will hold this for its entire lifetime. It then calls the following interface functions for each stored context: 252 253- `BeforeLoadNetwork(NetworkId networkId)` 254- `AfterLoadNetwork(NetworkId networkId)` 255- `BeforeUnloadNetwork(NetworkId networkId)` 256- `AfterUnloadNetwork(NetworkId networkId)` 257 258@subsection S12_11_backend_developer_guide Dynamic Backends 259 260Backends can also be loaded by Arm NN dynamically at runtime. 261To be properly loaded and used, the backend instances must comply to the standard interface for dynamic backends and to the versioning 262rules that enforce ABI compatibility. 263 264@subsection S12_12_backend_developer_guide Dynamic Backends Base Interface 265 266The dynamic backend shared object must expose the following interface for Arm NN to handle it correctly: 267 268~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.cpp 269extern "C" 270{ 271const char* GetBackendId(); 272void GetVersion(uint32_t* outMajor, uint32_t* outMinor); 273void* BackendFactory(); 274} 275~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 276 277Interface details: 278 279- `extern "C"` is needed to use avoid C++ name mangling, necessary to allow Arm NN to dynamically load the symbols. 280- `GetBackendId()`: must return the unique id of the dynamic backends. 281 If at the time of the loading the id already exists in the internal Arm NN's backend registry, the backend will be skipped and 282 not loaded in Arm NN 283- `GetVersion()`: must return the version of the dynamic backend. 284 The version must indicate the version of the Backend API the dynamic backend has been built with. 285 The current Backend API version can be found by inspecting the IBackendInternal interface. 286 At the time of loading, the version of the backend will be checked against the version of the Backend API Arm NN is built with. 287 If the backend version is not compatible with the current Backend API, the backend will not be loaded as it will be assumed that 288 it is not ABI compatible with the current Arm NN build. 289- `BackendFactory()`: must return a valid instance of the backend. 290 The backend instance is an object that must inherit from the version of the IBackendInternal interface declared by GetVersion(). 291 It is the backend developer's responsibility to ensure that the backend implementation correctly reflects the version declared by 292 GetVersion(), and that the object returned by the BackendFactory() function is a valid and well-formed instance of the IBackendInternal 293 interface. 294 295@subsection S12_13_backend_developer_guide Dynamic Backend Versioning and ABI Compatibility 296 297Dynamic backend versioning policy: 298 299Updates to Arm NN's Backend API follow these rules: changes to the Backend API (the IBackendInternal interface) that break 300ABI compatibility with the previous API version will be indicated by a change of the API's major version, while changes 301that guarantee ABI compatibility with the previous API version will be indicated by a change in API's the minor version. 302 303For example: 304 305- Dynamic backend version 2.4 (i.e. built with Backend API version 2.4) is compatible with Arm NN's Backend API version 2.4 306 (same version, backend built against the same Backend API) 307- Dynamic backend version 2.1 (i.e. built with Backend API version 2.1) is compatible with Arm NN's Backend API version 2.4 308 (same major version, backend built against earlier compatible API) 309- Dynamic backend version 2.5 (i.e. built with Backend API version 2.5) is not compatible with Arm NN's Backend API version 2.4 310 (same major version, backend built against later incompatible API, backend might require update to the latest compatible backend API) 311- Dynamic backend version 2.0 (i.e. built with Backend API version 2.0) is not compatible with Arm NN's Backend API version 1.0 312 (backend requires a completely new API version) 313- Dynamic backend version 2.0 (i.e. built with Backend API version 2.0) is not compatible with Arm NN's Backend API version 3.0 314 (backward compatibility in the Backend API is broken) 315 316@subsection S12_13_backend_developer_guide Dynamic Backend Loading Paths 317 318During the creation of the Runtime, Arm NN will scan a given set of paths searching for suitable dynamic backend objects to load. 319A list of (absolute) paths can be specified at compile-time by setting a define named `DYNAMIC_BACKEND_PATHS` in the form of a colon-separated list of strings. 320 321~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.sh 322-DDYNAMIC_BACKEND_PATHS="PATH_1:PATH_2...:PATH_N" 323~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 324 325The paths will be processed in the same order as they are indicated in the macro. 326 327It is also possible to override those paths at runtime when creating the Runtime object by setting the value of the `m_DynamicBackendsPath` member in the CreationOptions class. 328Only one path is allowed for the override via the CreationOptions class. 329By setting the value of the `m_DynamicBackendsPath` to a path in the filesystem, Arm NN will entirely ignore the list of paths passed via the 330`DYNAMIC_BACKEND_PATHS` compiler directive. 331 332All the specified paths are validated before processing (they must exist, must be directories, and must be absolute paths), 333in case of error a warning message will be added to the log, but Arm NN's execution will not be stopped. 334If all paths are not valid, then no dynamic backends will be loaded in the Arm sNN's runtime. 335 336Passing an empty list of paths at compile-time and providing no path override at runtime will effectively disable the 337dynamic backend loading feature, and no dynamic backends will be loaded into Arm NN's runtime. 338 339@subsection S12_14_backend_developer_guide Dynamic Backend File Naming Convention 340 341During the creation of a Runtime object, Arm NN will scan the paths specified for dynamic backend loading searching for suitable backend objects. 342Arm NN will try to load only the files that match the following accepted naming scheme: 343 344~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.sh 345<vendor>_<name>_backend.so[<version>] (e.g. "Arm_GpuAcc_backend.so" or "Arm_GpuAcc_backend.so.1.2.3") 346~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 347 348Only alphanumeric characters are allowed for both the `<vendor>` and the `<name>` fields, namely lowercase and/or uppercase characters, 349and/or numerical digits (see the table below for examples). 350Only dots and numbers are allowed for the optional `<version>` field. 351 352Symlinks to other files are allowed to support the standard linux shared object versioning: 353 354~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.sh 355Arm_GpuAcc_backend.so -> Arm_GpuAcc_backend.so.1.2.3 356Arm_GpuAcc_backend.so.1 -> Arm_GpuAcc_backend.so.1.2.3 357Arm_GpuAcc_backend.so.1.2 -> Arm_GpuAcc_backend.so.1.2.3 358Arm_GpuAcc_backend.so.1.2.3 359~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 360 361Files are identified by their full canonical path, so it is allowed to have files with the same name in different directories. 362However, if those are actually the same dynamic backend, only the first in order of parsing will be loaded. 363 364Examples: 365 366| Filename | Description | 367| -------------------------------------------------------- | ------------------------------------------------- | 368| Arm_GpuAcc_backend.so | valid: basic backend name | 369| Arm_GpuAcc_backend.so.1 | valid: single field version number | 370| Arm_GpuAcc_backend.so.1.2 | valid: multiple field version number | 371| Arm_GpuAcc_backend.so.1.2.3 | valid: multiple field version number | 372| Arm_GpuAcc_backend.so.10.1.27 | valid: Multiple digit version | 373| Arm_GpuAcc_backend.so.10.1.33. | not valid: dot not followed by version number | 374| Arm_GpuAcc_backend.so.3.4..5 | not valid: dot not followed by version number | 375| Arm_GpuAcc_backend.so.1,1.1 | not valid: comma instead of dot in the version | 376| Arm123_GpuAcc_backend.so | valid: digits in vendor name are allowed | 377| Arm_GpuAcc456_backend.so | valid: digits in backend id are allowed | 378| Arm%Co_GpuAcc_backend.so | not valid: invalid character in vendor name | 379| Arm_Gpu.Acc_backend.so | not valid: invalid character in backend id | 380| GpuAcc_backend.so | not valid: missing vendor name | 381| _GpuAcc_backend.so | not valid: missing vendor name | 382| Arm__backend.so | not valid: missing backend id | 383| Arm_GpuAcc.so | not valid: missing "backend" at the end | 384| __backend.so | not valid: missing vendor name and backend id | 385| __.so | not valid: missing all fields | 386| Arm_GpuAcc_backend | not valid: missing at least ".so" at the end | 387| Arm_GpuAcc_backend_v1.2.so | not valid: extra version info at the end | 388| Arm_CpuAcc_backend.so | valid: basic backend name | 389| Arm_CpuAcc_backend.so.1 -> Arm_CpuAcc_backend.so | valid: symlink to valid backend file | 390| Arm_CpuAcc_backend.so.1.2 -> Arm_CpuAcc_backend.so.1 | valid: symlink to valid symlink | 391| Arm_CpuAcc_backend.so.1.2.3 -> Arm_CpuAcc_backend.so.1.2 | valid: symlink to valid symlink | 392| Arm_no_backend.so -> nothing | not valid: symlink resolves to non-existent file | 393| pathA/Arm_GpuAcc_backend.so | valid: basic backend name | 394| pathB/Arm_GpuAcc_backend.so | valid: but duplicated from pathA/ | 395 396Arm NN will try to load the dynamic backends in the same order as they are parsed from the filesystem. 397 398@subsection S12_15_backend_developer_guide Dynamic Backend Examples 399 400The source code includes an example that is used to generate some mock dynamic backends for testing purposes. The source files are: 401 402- TestDynamicBackend.hpp 403- TestDynamicBackend.cpp 404 405This example is useful for going through all the use cases that constitute an invalid dynamic backend object, such as 406an invalid/malformed implementation of the shared object interface, or an invalid value returned by any of the interface methods 407that would prevent Arm NN from making use of the dynamic backend. 408 409A dynamic implementation of the reference backend is also provided. The source files are: 410 411- RefDynamicBackend.hpp 412- RefDynamicBackend.cpp 413 414The implementation itself is quite simple and straightforward. Since an implementation of this particular backend was already available, 415the dynamic version is just a wrapper around the original code that simply returns the backend id, version and an instance of the 416backend itself via the factory function. 417For the sake of the example, the source code of the reference backend is used to build the dynamic version (as you would for any new 418dynamic backend), while all the other symbols needed are provided by linking the dynamic backend against Arm NN. 419 420The makefile used for building the reference dynamic backend is also provided: [CMakeLists.txt](dynamic/reference/CMakeLists.txt) 421 422A unit test that loads the reference backend dynamically and that exercises it is also included in the file 423[DynamicBackendTests.cpp](dynamic/backendsCommon/test/DynamicBackendTests.cpp), by the test case `CreateReferenceDynamicBackend`. 424In the test, a path on the filesystem is scanned for valid dynamic backend files (using the override option in `CreationOptions`) 425where only the reference dynamic backend is. 426In this example the file is named `Arm_CpuRef_backend.so`, which is compliant with the expected naming scheme for dynamic backends. 427A `DynamicBackend` is created in the runtime to represent the newly loaded backend, then the backend is registered in the Backend 428Registry with the id "CpuRef" (returned by `GetBackendId()`). 429The unit test makes sure that the backend is actually registered in Arm NN, before trying to create an instance of the backend by 430calling the factory function provided through the shared object interface (`BackendFactory()`). 431The backend instance is used to verify that everything is in order, testing basic 2D convolution support by making use of the 432Layer Support API and the Workload Factory. 433At the end of test, the runtime object goes out of scope and the dynamic backend instance is automatically destroyed, and the handle to 434the shared object is closed. 435 436<br/><br/><br/><br/> 437 438@section S13_dynamic_backend_guide Standalone Dynamic Backend Developer Guide 439 440Arm NN allows adding new dynamic backends. Dynamic Backends can be compiled as standalone against Arm NN 441and can be loaded by Arm NN dynamically at runtime. 442 443To be properly loaded and used, the backend instances must comply to the standard interface for dynamic backends 444and to the versioning rules that enforce ABI compatibility. 445The details of how to add dynamic backends can be found in [src/backends/README.md](../backends/README.md). 446 447@subsection S13_1_dynamic_backend_guide Standalone Dynamic Backend Example 448 449The source code includes an example that is used to generate a dynamic implementation of the reference backend 450is provided at 451- RefDynamicBackend.hpp 452- RefDynamicBackend.cpp 453 454The makefile used for building the standalone reference dynamic backend is also provided: 455CMakeLists.txt 456 457@subsection S13_2_dynamic_backend_guide Dynamic Backend Loading Paths 458 459During the creation of the Runtime, Arm NN will scan a given set of paths searching for suitable dynamic backend objects to load. 460A list of (absolute) paths can be specified at compile-time by setting a define named `DYNAMIC_BACKEND_PATHS` 461 in the form of a colon-separated list of strings. 462 463~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.sh 464-DDYNAMIC_BACKEND_PATHS="PATH_1:PATH_2...:PATH_N" 465~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 466 467The paths will be processed in the same order as they are indicated in the macro. 468 469**/ 470}