• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Using ImageEffect to Edit Images
2
3## When to Use
4
5The **ImageEffect** class provides a series of APIs for editing an image. You can use the APIs to implement various filter effects on images of different input types (pixel map, native window, native buffer, or URI).
6With the native APIs provided by **ImageEffect**, you can:
7
8- Add a filter or filter chain, set an input image, and make the filter effective on the image.
9- Customize a filter.
10- Use a system defined filter.
11
12## **Available APIs**
13
14For details about the APIs, see [ImageEffect](../../reference/apis-image-kit/capi-imageeffect.md).
15
16## How to Develop
17
18**Adding Dynamic Link Libraries**
19
20Add the following libraries to **CMakeLists.txt**.
21
22```txt
23target_link_libraries(entry PUBLIC
24    libace_ndk.z.so
25    libimage_effect.so
26    libpixelmap.so
27    libnative_window.so
28    libnative_buffer.so
29)
30```
31
32Add the following dynamic link libraries based on the image type: **libpixelmap.so** for the pixel map type, **libnative_window.so** for the native window type, and **libnative_buffer.so** for the native buffer type.
33
34**Adding the Header Files**
35
36```c++
37#include <multimedia/image_effect/image_effect.h>
38#include <multimedia/image_effect/image_effect_filter.h>
39#include <multimedia/image_effect/image_effect_errors.h>
40```
41
42### Making a Filter Effective
43
441. Create an ImageEffect instance.
45
46    ```c++
47    // Create an ImageEffect instance, with the alias set to ImageEdit.
48    OH_ImageEffect *imageEffect = OH_ImageEffect_Create("ImageEdit");
49    ```
50
512. Add a filter.
52
53    ```c++
54    // Add a filter to obtain an OH_EffectFilter instance. You can call this API multiple times to add multiple filters to form a filter chain.
55    OH_EffectFilter *filter = OH_ImageEffect_AddFilter(imageEffect, OH_EFFECT_BRIGHTNESS_FILTER);
56    CHECK_AND_RETURN_LOG(filter != nullptr, "OH_ImageEffect_AddFilter fail!");
57
58    // Set a filter parameter. For example, set the intensity to 50.
59    ImageEffect_Any value = { .dataType = ImageEffect_DataType::EFFECT_DATA_TYPE_FLOAT, .dataValue.floatValue = 50.f };
60    ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, OH_EFFECT_FILTER_INTENSITY_KEY, &value);
61    ```
62
633. Set the data to be processed.
64
65    **Scenario 1: Set the OH_PixelmapNative input type.**
66
67    For details about how to use OH_PixelmapNative, see [PixelMap Data Processing (C/C++)](image-pixelmap-operation-native.md).
68
69    ```c++
70    // Set an input pixel map.
71    errorCode = OH_ImageEffect_SetInputPixelmap(imageEffect, inputPixelmap);
72    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputPixelmap fail!");
73
74    // (Optional) Set an output pixel map. If this API is not called, the filter effect directly takes effect on the input pixel map.
75    errorCode = OH_ImageEffect_SetOutputPixelmap(imageEffect, outputPixelmap);
76    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputPixelmap fail!");
77    ```
78
79    **Scenario 2: Set the OH_NativeBuffer input type.**
80
81    For details about how to use OH_NativeBuffer, see [Native Buffer Development (C/C++)](../../graphics/native-buffer-guidelines.md).
82
83    ```c++
84    // Set an input native buffer.
85    errorCode = OH_ImageEffect_SetInputNativeBuffer(imageEffect, inputNativeBuffer);
86    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputNativeBuffer fail!");
87
88    // (Optional) Set an output native buffer. If this API is not called, the filter effect directly takes effect on the input native buffer.
89    errorCode = OH_ImageEffect_SetOutputNativeBuffer(imageEffect, outputNativeBuffer);
90    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputNativeBuffer fail!");
91    ```
92
93    **Scenario 3: Set the URI input type.**
94
95    ```c++
96    // Set an input URI.
97    errorCode = OH_ImageEffect_SetInputUri(imageEffect, inputUri);
98    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputUri fail!");
99
100    // (Optional) Set an output URI. If this API is not called, the filter effect directly takes effect on the input URI.
101    errorCode = OH_ImageEffect_SetOutputUri(imageEffect, outputUri);
102    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputUri fail!");
103    ```
104    **Scenario 4: Set the texture input type.**
105
106    In the texture input scenario, the GPU is used for rendering. In this scenario, the application needs to provide a valid OpenGL context environment, set parameters, and perform rendering operations in the correct environment.
107    ```c++
108    // Set the input texture ID.
109    errorCode = OH_ImageEffect_SetInputTextureId(imageEffect, inputTex, ColorSpaceName::SRGB);
110    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputTextureId fail!");
111
112    // Set the output texture ID. Note that the output texture ID cannot be the same as the input texture ID. Otherwise, rendering exceptions may occur.
113    errorCode = OH_ImageEffect_SetOutputTextureId(imageEffect, outputTex);
114    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputTextureId fail!");
115    ```
116
117    **Scenario 5: Set the OHNativeWindow input type.**
118
119    The following uses camera preview as an example to describe the scenario. The surface ID provided by the **XComponent** for camera preview streams can be converted into an OHNativeWindow instance at the native C++ layer.
120    For details about how to use the **XComponent**, see [XComponent](../../reference/apis-arkui/arkui-ts/ts-basic-components-xcomponent.md).
121    For details about how to use the native window module, see [OHNativeWindow](../../reference/apis-arkgraphics2d/capi-nativewindow.md).
122    For details about how to use the camera, see [Camera Preview (C/C++)](../camera/native-camera-preview.md).
123
124    (1) Add an **XComponent** to the .ets file.
125
126     ```ts
127     XComponent({
128         id: 'xcomponentId',
129         type: 'surface',
130         controller: this.mXComponentController,
131         libraryname: 'entry'
132     })
133     .onLoad(() => {
134         // Obtain the surface ID of the XComponent.
135         this.mSurfaceId = this.mXComponentController.getXComponentSurfaceId()
136
137         // Obtain the input surface ID.
138         this.mSurfaceId = imageEffect.getSurfaceId(this.mSurfaceId)
139
140         // Call the camera API to start preview and transfer the input surface ID to the camera framework.
141         // ...
142     })
143     .width('100%')
144     .height('100%')
145     ```
146
147    (2) Implement imageEffect.getSurfaceId at the native C++ layer.
148
149     ```c++
150     // Create a NativeWindow instance based on the surface ID. Note that the instance must be released by calling OH_NativeWindow_DestoryNativeWindow when it is no longer needed.
151     uint64_t outputSurfaceId;
152     std::istrstream iss(outputSurfaceIdStr);
153     issue >> outputSurfaceId;
154     OHNativeWindow *outputNativeWindow = nullptr;
155     int32_t res = OH_NativeWindow_CreateNativeWindowFromSurfaceId(outputSurfaceId, &outputNativeWindow);
156     CHECK_AND_RETURN_LOG(res == 0, "OH_NativeWindow_CreateNativeWindowFromSurfaceId fail!");
157
158     // Set an output surface.
159     ImageEffect_ErrorCode errorCode = OH_ImageEffect_SetOutputSurface(imageEffect, outputNativeWindow);
160     CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputSurface fail!");
161
162     // Obtain the input surface. Note that the obtained inputNativeWindow instance must be released by calling OH_NativeWindow_DestoryNativeWindow when it is no longer needed.
163     OHNativeWindow *inputNativeWindow = nullptr;
164     errorCode = OH_ImageEffect_GetInputSurface(imageEffect, &inputNativeWindow);
165     CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_GetInputSurface fail!");
166
167     // Obtain the surface ID from the inputNativeWindow instance.
168     uint64_t inputSurfaceId = 0;
169     res = OH_NativeWindow_GetSurfaceId(inputNativeWindow, &inputSurfaceId);
170     CHECK_AND_RETURN_LOG(res == 0, "OH_NativeWindow_GetSurfaceId fail!");
171
172     // Convert the surface ID to a string and return the string.
173     std::string inputSurfaceIdStr = std::to_string(inputSurfaceId);
174     ```
175
1764. Start the image effector.
177
178    ```c++
179    // Start the image effector.
180    errorCode = OH_ImageEffect_Start(imageEffect);
181    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Start fail!");
182    ```
183
1845. (Optional) Stop the image effector. This operation is required only in the input surface scenario.
185
186    ```c++
187    // Stop the image effector.
188    errorCode = OH_ImageEffect_Stop(imageEffect);
189    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Stop fail!");
190    ```
191
1926. (Optional) Serialize the image effector.
193
194    ```c++
195    char *info = nullptr;
196    errorCode = OH_ImageEffect_Save(imageEffect, &info);
197    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Save fail!");
198    ```
199
2007. Destroy the ImageEffect instance.
201
202    ```c++
203    // Release the ImageEffect instance.
204    errorCode = OH_ImageEffect_Release(imageEffect);
205    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result, "OH_ImageEffect_Release fail!");
206    ```
207
208### Customizing a Filter
209
210To implement and register a custom filter, perform the following steps:
211
2121. Define **ImageEffect_FilterDelegate**.
213
214    ```c++
215    // Image information struct.
216    struct EffectBufferInfo {
217        void *addr = nullptr;
218        int32_t width = 0;
219        int32_t height = 0;
220        int32_t rowSize = 0;
221        ImageEffect_Format format = ImageEffect_Format::EFFECT_PIXEL_FORMAT_UNKNOWN;
222    };
223
224    // Implement a custom filter.
225    ImageEffect_FilterDelegate filterDelegate = {
226        .setValue = [](OH_EffectFilter *filter, const char *key, const ImageEffect_Any *value) {
227            // Verify the parameters. If the verification is successful, true is returned. Otherwise, false is returned.
228            // ...
229            return true;
230        },
231        .render = [](OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData) {
232            return Render(filter, info, pushData);
233        },
234        .save = [](OH_EffectFilter *filter, char **info) {
235            // Obtain the custom filter parameter. In this example, Brightness is the key of the custom filter.
236            ImageEffect_Any value;
237            ImageEffect_ErrorCode errorCode = OH_EffectFilter_GetValue(filter, "Brightness", &value);
238            CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, false, "OH_EffectFilter_GetValue fail!");
239
240            // Generate a key-value pair.
241            json values;
242            values["Brightness"] = value.dataValue.floatValue;
243            json root;
244            root["name"] = "CustomBrightness";
245            root["values"] = values;
246
247        　　 // Convert the JSON object into the string infoStr.
248            // ...
249
250            // Assign a serialized string address to *info.
251            *info = infoStr;
252            return true;
253        },
254        .restore = [](const char *info) {
255            // Create an OH_EffectFilter instance, in which CustomBrightness is the name of the custom filter.
256            OH_EffectFilter *filter = OH_EffectFilter_Create("CustomBrightness");
257            // Parse the JSON string info to obtain the key and value.
258            // ...
259
260            // Set a filter parameter. value is the parameter parsed based on the JSON string in info.
261            ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, "Brightness", &value);
262
263            // ...
264            return filter;
265        }
266    };
267    ```
268
269    You can implement the **Render** API in two scenarios.
270
271    **Scenario 1: The custom algorithm can directly modify the pixel data in info (for example, the brightness filter).**
272
273    ```c++
274    bool Render(OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData)
275    {
276        // Obtain the image information.
277        EffectBufferInfo inputBufferInfo;
278        OH_EffectBufferInfo_GetAddr(info, &inputBufferInfo.addr);
279        OH_EffectBufferInfo_GetWidth(info, &inputBufferInfo.width);
280        OH_EffectBufferInfo_GetHeight(info, &inputBufferInfo.height);
281        OH_EffectBufferInfo_GetRowSize(info, &inputBufferInfo.rowSize);
282        OH_EffectBufferInfo_GetEffectFormat(info, &inputBufferInfo.format);
283
284        // Invoke the custom filter algorithm.
285        ApplyCustomAlgo(inputBufferInfo);
286
287        // After the editing is complete, call pushData to directly transfer the original image.
288        pushData(filter, info);
289        return true;
290    }
291    ```
292
293    **Scenario 2: The custom algorithm cannot directly modify the pixel data in info (for example, the crop filter).**
294
295    ```c++
296    bool Render(OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData)
297    {
298        // Obtain the image information.
299        EffectBufferInfo inputBufferInfo;
300        OH_EffectBufferInfo_GetAddr(info, &inputBufferInfo.addr);
301        OH_EffectBufferInfo_GetWidth(info, &inputBufferInfo.width);
302        OH_EffectBufferInfo_GetHeight(info, &inputBufferInfo.height);
303        OH_EffectBufferInfo_GetRowSize(info, &inputBufferInfo.rowSize);
304        OH_EffectBufferInfo_GetEffectFormat(info, &inputBufferInfo.format);
305
306        // Create output pixel information.
307        EffectBufferInfo outputBufferInfo = CreateOutputBufferInfo(inputBufferInfo);
308
309        // Invoke the custom filter algorithm.
310        ApplyCustomAlgo(inputBufferInfo, outputBufferInfo);
311
312        // Generate outputOhInfo.
313        OH_EffectBufferInfo *outputOhInfo = OH_EffectBufferInfo_Create();
314        OH_EffectBufferInfo_SetAddr(outputOhInfo, outputBufferInfo.addr);
315        OH_EffectBufferInfo_SetWidth(outputOhInfo, outputBufferInfo.width);
316        OH_EffectBufferInfo_SetHeight(outputOhInfo, outputBufferInfo.height);
317        OH_EffectBufferInfo_SetRowSize(outputOhInfo, outputBufferInfo.rowSize);
318        OH_EffectBufferInfo_SetEffectFormat(outputOhInfo, outputBufferInfo.format);
319
320        // Call pushData to transfer outputOhInfo after editing.
321        pushData(filter, outputOhInfo);
322
323        // Release resources.
324        OH_EffectBufferInfo_Release(outputOhInfo);
325        ReleaseOutputBuffer(outputBufferInfo.addr);
326
327        return true;
328    }
329    ```
330
3312. Generate custom filter information.
332
333    ```c++
334    // Create an OH_EffectFilterInfo instance.
335    OH_EffectFilterInfo *customFilterInfo = OH_EffectFilterInfo_Create();
336    CHECK_AND_RETURN_LOG(customFilterInfo != nullptr, "OH_EffectFilter_GetValue fail!");
337
338    // Set the name of the custom filter.
339    OH_EffectFilterInfo_SetFilterName(customFilterInfo, "CustomBrightness");
340
341    // Set the buffer types supported by the custom filter.
342    ImageEffect_BufferType bufferTypeArray[] = { ImageEffect_BufferType::EFFECT_BUFFER_TYPE_PIXEL };
343    OH_EffectFilterInfo_SetSupportedBufferTypes(customFilterInfo, sizeof(bufferTypeArray) / sizeof(ImageEffect_BufferType), bufferTypeArray);
344
345    // Set the pixel formats supported by the custom filter.
346    ImageEffect_Format formatArray[] = { ImageEffect_Format::EFFECT_PIXEL_FORMAT_RGBA8888 };
347    OH_EffectFilterInfo_SetSupportedFormats(customFilterInfo, sizeof(formatArray) / sizeof(ImageEffect_Format), formatArray);
348    ```
349
3503. Register **ImageEffect_FilterDelegate** with the image effector.
351
352    ```c++
353    // Register the custom filter.
354    ImageEffect_ErrorCode errorCode = OH_EffectFilter_Register(customFilterInfo, &filterDelegate);
355    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_Register fail!");
356    ```
357
358### Using a System Defined Filter
359
3601. Create a filter.
361
362    ```c++
363    // Create a filter, for example, a contrast filter.
364    OH_EffectFilter *filter = OH_EffectFilter_Create(OH_EFFECT_CONTRAST_FILTER);
365    ```
366
3672. Set a filter parameter.
368
369    ```c++
370    // Set a filter parameter. For example, set the intensity to 50.
371    ImageEffect_Any value = {.dataType = ImageEffect_DataType::EFFECT_DATA_TYPE_FLOAT, .dataValue.floatValue = 50.f};
372    ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, OH_EFFECT_FILTER_INTENSITY_KEY, &value);
373    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_SetValue fail!");
374    ```
375
3763. Make the filter take effect.
377
378    ```c++
379    // Start rendering with the filter.
380    errorCode = OH_EffectFilter_Render(filter, inputPixelmap, outputPixelmap);
381    ```
382
3834. Destroy the filter instance.
384
385    ```c++
386    // Destroy the filter instance.
387    errorCode = OH_EffectFilter_Release(filter);
388    ```
389
390### Query Capabilities
391
392- Query filter information based on the filter name.
393
394    ```c++
395    // Create an OH_EffectFilterInfo instance.
396    OH_EffectFilterInfo *filterInfo = OH_EffectFilterInfo_Create();
397    CHECK_AND_RETURN_LOG(filterInfo != nullptr, "OH_EffectFilterInfo_Create fail!");
398
399    // Query the filter information based on the filter name.
400    ImageEffect_ErrorCode errorCode = OH_EffectFilter_LookupFilterInfo(OH_EFFECT_BRIGHTNESS_FILTER, filterInfo);
401    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_LookupFilterInfo fail!");
402
403    // Obtain the filter name from the filter information.
404    char *name = nullptr;
405    OH_EffectFilterInfo_GetFilterName(filterInfo, &name);
406
407    // Obtain the supported buffer types.
408    uint32_t supportedBufferTypesCnt = 0;
409    ImageEffect_BufferType *bufferTypeArray = nullptr;
410    OH_EffectFilterInfo_GetSupportedBufferTypes(filterInfo, &supportedBufferTypesCnt, &bufferTypeArray);
411
412    // Obtain the supported pixel formats.
413    uint32_t supportedFormatsCnt = 0;
414    ImageEffect_Format *formatArray = nullptr;
415    OH_EffectFilterInfo_GetSupportedFormats(filterInfo, supportedFormatsCnt, &formatArray);
416
417    // Destroy the OH_EffectFilterInfo instance.
418    OH_EffectFilterInfo_Release(filterInfo);
419    ```
420
421- Query filters that meet given conditions.
422
423    ```c++
424    // Query all filters. Resources need to be released.
425    ImageEffect_FilterNames *filterNames = OH_EffectFilter_LookupFilters("Default");
426
427    // ...
428
429    // Release virtual memory resources by filter names.
430    OH_EffectFilter_ReleaseFilterNames();
431    ```
432
433<!--no_check-->