• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# ROI Video Encoding
2
3<!--Kit: AVCodec Kit-->
4<!--Subsystem: Multimedia-->
5<!--Owner: @xhjgc-->
6<!--Designer: @dpy2650--->
7<!--Tester: @cyakee-->
8<!--Adviser: @zengyawen-->
9
10## Basic Concepts
11
12Region of Interest (ROI) video encoding is an advanced capability built on existing hardware encoding. It allows you to specify ROI areas and optimize encoding effects by adjusting bit rate allocation for these areas.
13- ROI area: ROI typically refers to the eye-catching areas in a video, such as a face in a live broadcast.
14- ROI encoding: This allows for higher bit rate allocation to the ROI areas in the frame while reducing the bit rate for non-ROI areas like the background, thereby enhancing the overall visual experience under limited bandwidth conditions.
15
16## When to Use
17
18Video encoding scenarios with distinct ROI areas, such as:
19- **Live streaming scenarios**: Faces or products are usually the focal points. ROI encoding can effectively enhance the subjective clarity of these areas.
20- **Surveillance scenarios**: License plates or faces are usually the focal points. ROI encoding can preserve clearer useful information.
21
22## Constraints
23
24- **Supported platforms**: Kirin 9000 and later
25- **Supported systems**: OpenHarmony 6.0 and later
26- **Supported encoders**: H.264 8-bit hardware encoding, H.265 8-bit hardware encoding, and H.265 10-bit hardware encoding
27- **Supported bit rate control modes**: VBR, CBR, and SQR
28- **Supported encoding modes**: surface mode and buffer mode
29- A maximum of 6 ROI areas can be configured per frame, and the total ROI area cannot exceed 1/5 of the image area.
30- The module only provides the underlying ROI encoding capability and does not include ROI area detection.
31
32## Available APIs
33
34The ROI encoding API enables you to send configuration parameters in string format. The parameters must follow the format "Top1,Left1-Bottom1,Right1=Offset1;Top2,Left2-Bottom2,Right2=Offset2;".<br>
35- **Top**, **Left**, **Bottom**, and **Right** specify the top, left, bottom, and right boundaries of the ROI area, respectively.
36- **Offset** specifies deltaQp. The **"=Offset"** part can be omitted. If it is omitted, the default parameter (**deltaQp=-3**) is used.
37- Multiple ROI parameters are separated by semicolons (;). All parameters are integers.
38- Before using the ROI encoding APIs, ensure that the input parameters are valid and avoid overlapping between multiple ROI areas as much as possible.
39
40|Parameter|Description|Format|
41|------- |------- |------- |
42|OH_MD_KEY_VIDEO_ENCODER_ROI_PARAMS |ROI parameters.|String|
43
44**NOTE**
451. For ease of use, ROI parameters support per-frame sending and real-time effectiveness. You do not need to perform capability queries or configure global switches.
462. If the underlying hardware encoder does not support ROI encoding, the encoder ignores the ROI parameters and performs normal encoding.
473. The valid range for deltaQp is [-51, 51]. The encoder overlays deltaQp on the block-level QP, and then clips it to the [minQp, maxQp] range to obtain the final QP.
484. If the parameters are not sent through **OH_MD_KEY_VIDEO_ENCODER_ROI_PARAMS**, the frame will reuse historical ROI information for encoding.
495. If an empty string is sent through **OH_MD_KEY_VIDEO_ENCODER_ROI_PARAMS**, ROI encoding is disabled, historical ROI information is cleared, and normal encoding is performed.
506. If the string sent through **OH_MD_KEY_VIDEO_ENCODER_ROI_PARAMS** cannot be parsed into any valid ROI information, ROI encoding does not take effect, historical ROI information is cleared, and normal encoding is performed.
517. If the total ROI area exceeds 1/5 of the image size, ROI encoding does not take effect, historical ROI information is cleared, and normal encoding is performed.
528. If more than 6 valid ROI parameters are input, the excess ROI parameters are ignored.
539. If multiple ROI areas overlap, only the first ROI parameter takes effect in the overlapping area.
54
55## How to Develop
56
57For details about basic encoding functionality, see [Video Encoding](video-encoding.md). The following are specific instructions for ROI encoding.
58
59### Surface Mode
60
61
621. Call **OH_VideoEncoder_RegisterParameterCallback()** to register the frame-specific parameter callback.
63
64    ```c++
65    // 1. Implement the OH_VideoEncoder_OnNeedInputParameter callback function.
66    static void OnNeedInputParameter(OH_AVCodec *codec, uint32_t index, OH_AVFormat *parameter, void *userData)
67    {
68        /**
69        * Configure two ROI areas and specify the corresponding deltaQp.
70        * ROI1 has its top-left corner at (0, 0) and bottom-right corner at (64, 128), with a QP adjustment of -4.
71        * ROI2 has its top-left corner at (200, 100) and bottom-right corner at (400, 300), with a QP adjustment of +3.
72        **/
73        const char *roiInfo = "0,0-128,64=-4;100,200-300,400=3";
74        OH_AVFormat_SetStringValue(parameter, OH_MD_KEY_VIDEO_ENCODER_ROI_PARAMS, roiInfo);
75        OH_VideoEncoder_PushInputParameter(codec, index);
76    }
77    // 2. Register the frame channel callback functions.
78    OH_VideoEncoder_OnNeedInputParameter inParaCb = OnNeedInputParameter;
79    OH_VideoEncoder_RegisterParameterCallback(videoEnc, inParaCb, nullptr); // nullptr: userData is null.
80    ```
81
82### Buffer Mode
83
841. Call **OH_AVBuffer_SetParameter()** to configure frame-specific parameters.
85
86    ```c++
87    void OnNeedInputBuffer(OH_AVCodec *codec, uint32_t index, OH_AVBuffer *buffer, void *userData)
88    {
89        /**
90        * Configure two ROI areas and specify the corresponding deltaQp.
91        * ROI1 has its top-left corner at (0, 0) and bottom-right corner at (64, 128), with a QP adjustment of -4.
92        * ROI2 has its top-left corner at (200, 100) and bottom-right corner at (400, 300), with a QP adjustment of +3.
93        **/
94        const char *roiInfo = "0,0-128,64=-4;100,200-300,400=3";
95        OH_AVFormat *parameter = OH_AVBuffer_GetParameter(buffer);
96        OH_AVFormat_SetStringValue(parameter, OH_MD_KEY_VIDEO_ENCODER_ROI_PARAMS, roiInfo);
97        OH_AVBuffer_SetParameter(buffer, parameter);
98        OH_AVFormat_Destroy(parameter);
99
100        inQueue.Enqueue(std::make_shared<CodecBufferInfo>(index, buffer));
101    }
102    ```
103