README.md
1# Sound Trigger Middleware 2 3## Overview 4Sound Trigger Middleware is a system service that exposes sound trigger functionality (low-power 5detection of acoustic events) to applications and higher-level system service. 6 7It has the following roles: 8- Isolating the soundtrigger HAL from potentially untrusted clients. 9- Enforcing correct behavior of the clients. 10- Enforcing correct behavior of the HAL and attempting to recover from failures. 11- Enforcing permissions for using soundtrigger functionality. 12- Serializing access to the HAL. 13- Logging soundtrigger usage in a comprehensive and consistent manner. 14- Generating a dumpsys report including current state and history of operations. 15- Providing a standard interface regardless which version of the HAL is implemented and gracefully 16 degrading operation whenever necessary. 17 18## Structure 19 20The service implementation can be divided into three main layers: 21 22- The "bottom layer" is concerned with HAL compatibility - making all HAL versions look and behave 23 the same. 24- The "middle layer" is concerned with the business logic of the service. 25- The "top layer" is concerned with exposing this functionality as a System Service and integrating 26 with other parts of the system. 27 28### HAL Compatibility Layer 29 30This layer implements the `ISoundTriggerHal` interface, which is the version-agnostic representation 31of the sound trigger HAL driver. It has two main implementations, `SoundTriggerHw2Compat` and 32`SoundTriggerHw3Compat` responsible for adapting to V2.x and V3 HAL drivers, respectively, including 33supporting their respective minor-version differences. 34 35This layer also includes several `ISoundTriggerHal` decorators, such as `SoundTriggerHalWatchdog` 36that enforces deadlines on calls into the HAL, and `SoundTriggerHalEnforcer` which enforces that 37the HAL respects the expected protocol. 38 39The decorator-based design is an effective tool for separation of aspects and modularity, thus 40keeping classes relatively small and focused on one concern. It is also very effective for 41testability by following dependency injection principles. 42 43### Business Logic Layer 44 45This layer also uses a decorator-based design for separation of concerns. The main interface being 46decorated is `ISoundTriggerMiddlwareInternal`, which closely follows the external-facing AIDL 47interface, `ISoundTriggerMiddlewareService`. 48 49Each of the decorators serves a focused purpose: for example, `SoundTriggerMiddlwarePermission` 50deals with enforcing permissions required for the various methods, `SoundTriggerMiddlewareLogging` 51logs all API usage, `SoundTriggerMiddlewareValidation` enforces correct usage of the protocol and 52isolates client errors from internal server errors. 53 54At the bottom of this decorator stack is `SoundTriggerMiddlewareImpl` / `SoundTriggerModule`, which 55are the adapter between `ISoundTriggerHal` and `ISoundTriggerMiddlwareInternal`, introducing the 56notion of having separate client sessions sharing the same HAL. 57 58### Service Layer 59 60This layer ties everything together. It instantiates the actual system service and the decorator 61stack. It also provides concrete connections to the Audio service (for negotiating sessions shared 62between Audio and Sound Trigger and for notifications about audio recording) and to the various HAL 63factories. 64 65This is the only layer that makes strong assumptions about the environment instead of relying on 66abstractions. 67 68## Error Handling and Exception Conventions 69 70We follow conventions for usage of exceptions in the service, in order to correctly and consistently 71distinguish the following cases: 72 731. The client has done something wrong. 742. The service implementation has done something wrong. 753. The HAL has done something wrong. 764. Nobody has done anything wrong, but runtime conditions prevent an operation from being fulfilled 77 as intended. 78 79The `SoundTriggerMiddlewarePermission` class would reject any calls from unauthorized clients, 80responding with the appropriate exception. 81 82The `SoundTriggerMiddlewareValidation` class does much of this separation. By validating the 83client's data and state, it would throw a relevant `RuntimeException` exception to the client 84without passing the requests down to the lower layers. Once that is done, any exception thrown from 85the underlying implementation can be assumed to be not the client's fault. If caught, they will be 86classified according to the following rule: 87 88- If they are `RecoverableException`s, they represent category #4 above, and will be presented to 89 the client as `ServiceSpecificException`s with the same error code. 90- Otherwise, they are considered an internal error (including HAL malfunction) and will be 91 presented to the client as `ServiceSpecificException(Status.INTERNAL_ERROR)`. 92 93Internally, we would throw `RecoverableException` whenever appropriate. Whenever a HAL malfunctions, 94`SoundTriggerHalEnforcer` is responsible for rebooting it and throwing an exception. A HAL death is 95considered a valid failure mode, and thus result in `RecoverableException(Status.DEAD_OBJECT)`, 96which ends up as a `ServiceSpecificException(Status.DEAD_OBJECT)` on the client side. 97 98## Notes About Thread Synchronization 99This component has some tricky thread synchronization considerations due to its layered design and 100due to the fact that it is involved in both in-bound and out-bound calls from / to 101external components. 102 103The following mutexes need to be considered: 104- Typically, a one or more mutexes that exist in every layer of the sound trigger middleware stack 105 to serialize access to its internal state or to external components. 106- Audio Policy Service lock. This one is external - it should be assumed to be held whenever we're 107 inside the `ExternalCaptureStateTracker.setCaptureState()` call stack *AND* to be acquired from 108 within our calls into `AudioSessionProvider.acquireSession()` / 109 `AudioSessionProvider.releaseSession()`. 110 111To avoid potential deadlocks, a strict locking order must be ensured whenever nesting locks. The 112order is: 113- Upper layers of the stack, starting from the top (i.e. may not attempt to acquire a higher-layer 114 mutex while a lower-layer mutex is being held) until `ISoundTriggerHw2`. 115- Audio Policy Service lock. 116- Lower layers of the stack, starting from `ISoundTriggerHw2` all the way down to the HAL. 117 118In order to enforce this order, some conventions are established around when it is safe for a module 119to call another module, while having its local mutex(es) held: 120- Most calls (see exceptions below) originating from SoundTriggerMiddlewareService simply propagate 121 down the decorator stack. It is legal to call into the next layer down while holding a local 122 mutex. It is illegal to invoke a callback with a local mutex held. 123- Callbacks propagate from the lower layers up to the upper layers. It is legal to hold a local 124 mutex within a callback, but **not** while call to an upper layer. 125- In order to be able to synchronize, despite the asynchronous nature of callbacks, 126 `stopRecognition()` and `unloadModel()` work differently. They guarantee that once they return, 127 the callbacks associated with them will no longer be called. This implies that they have to block 128 until any pending callbacks are done processing and since these callbacks are potentially holding 129 locks of higher-order mutexes, we must not be holding a local mutex while calling down. The proper 130 sequence for these calls is: 131 - Obtain the local lock if needed. Update/check local state as necessary. 132 - Call the respective method of the delegate ("downwards"). Once it returns, not more callbacks 133 related to this operation will be called. 134 - Obtain the local lock if needed. Update local state as necessary. Assume that state might have 135 changed while the lock has been released. 136 - Release the local lock. 137 - Invoke any synchronous callbacks if needed. 138- Calling from `SoundTriggerMiddlewareImpl` / `SoundTriggerModule` into the audio policy service via 139 `acquireSession()` / `releaseSession()` while holding the local lock is legal. 140- `setCaptureState()` calls, originating from Audio Policy Service, into the lower layers of the 141 stack may call into the HAL (specifically, they must invoke `stopRecognition()`, but must not 142 block on callbacks. For this reason, `SoundTriggerHw2ConcurrentCaptureHandler`, which is the 143 recipient of these calls, features a buffer and an additional thread, which allows the actual 144 stopping to be synchronous, as required, without having to block the call upon higher layers 145 processing the callbacks. 146