• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Multimodal Input
2
3## Description
4
5OpenHarmony provides a Natural User Interface (NUI) for you to interact with your users. Unlike conventional categorization of input methods, OpenHarmony combines input methods of different dimensions into multimodal inputs, so you can easily arm your application with multi-dimensional, natural interaction features by using the application framework and system built-in UI components or APIs.
6
7Specifically, the Multimodal Input subsystem receives input events from various input devices, such as the keyboard, mouse, touchscreen, and touchpad, via the Linux native driver and HDF driver. After that, it normalizes and standardizes the input events, and distributes the events to the Inner SDK. The Inner SDK distributes the events to the ArkUI framework, which then encapsulates the input events and passes them to applications. Alternatively, the Inner SDK can also directly distribute input events to applications through JsKit APIs.
8
9![](figures/multimodal-architecture.png)
10
11## Contents
12
13```
14/foundation/multimodalinput/input
15├── common                       # Common code
16├── interfaces                   # External APIs
17│   └── native                   # Native APIs
18│       └── innerkits            # Native APIs provided for internal subsystems
19├── service                      # Service framework code
20├── sa_profile                   # Service startup configuration file
21├── uinput                       # Input event injection module
22```
23
24## Usage
25
26### Available APIs
27
28Available APIs of the Multimodal Input subsystem are event injection ones, which are open only to system applications currently.
29
30The **inputEventClient** class provides APIs for event injection.
31
32**Table 1** Major APIs in inputEventClient
33
34| Category|                      API                      |        Description       |
35| :------: | :-----------------------------------------------: | :----------------: |
36| Key injection| function injectEvent(keyEvent: KeyEvent): number; | Injects events.|
37
38### How to Use
39
40Currently, only the **BACK** key event can be injected.
41
42When a system app needs to return to the previous directory, you can call the API to inject the **BACK** key event to Multimodal Input, which then transfers this event to the system app, thereby achieving the return effect. The example code is as follows:
43
44```
45// Import the required JavaScript API library.
46import inputEventClient from '@ohos.multimodalInput.inputEventClient';
47
48// Call the API for injecting events.
49let keyEvent = {
50    isPressed: true,        // Action type of the key event. true indicates that the key is being pressed down, and false indicates that the key is being released.
51    keyCode: 2,             // Keycode for the key, for example, 2 for the BACK key.
52    keyDownDuration: 0,     // Duration in which the current key is pressed down before it is released, in milliseconds.
53    isIntercepted: false    // Whether to intercept key pressing events.
54}
55res = inputEventClient.injectEvent({KeyEvent: keyEvent});
56```
57
58> **NOTE**<br>New APIs must be downward compatible.
59
60### Supported Devices
61
62|    Device    | Touchscreen| Touchpad| Mouse| Keyboard|
63| :---------: | :----: | :----: | :--: | :--: |
64|   rk3568    |   Yes  |   Yes  |  Yes |  Yes |
65| hi3516dv300 |   Yes  |   No  |  No |  No |
66
67## Repositories Involved
68
69**Multimodal Input Subsystem**
70
71[multimodalinput_input](https://gitee.com/openharmony/multimodalinput_input)
72