• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Secondary Processing of Preview Streams (C/C++)
2<!--Kit: Camera Kit-->
3<!--Subsystem: Multimedia-->
4<!--Owner: @qano-->
5<!--SE: @leo_ysl-->
6<!--TSE: @xchaosioda-->
7
8You can use the APIs in the **ImageReceiver** class to create a PreviewOutput instance and obtain real-time data of the preview stream for secondary processing. For example, you can add a filter algorithm to the preview stream.
9
10## How to Develop
11
12Read [Camera](../../reference/apis-camera-kit/capi-oh-camera.md) for the API reference.
13
141. Import the NDK, which provides camera-related attributes and methods.
15
16   ```c++
17   // Include the NDK header files.
18   #include <cstdlib>
19   #include <hilog/log.h>
20   #include <memory>
21   #include <multimedia/image_framework/image/image_native.h>
22   #include <multimedia/image_framework/image/image_receiver_native.h>
23   #include "ohcamera/camera.h"
24   #include "ohcamera/camera_input.h"
25   #include "ohcamera/capture_session.h"
26   #include "ohcamera/preview_output.h"
27   #include "ohcamera/camera_manager.h"
28   ```
29
302. Link the dynamic library in the CMake script.
31
32   ```txt
33   target_link_libraries(entry PUBLIC
34       libace_napi.z.so
35       libhilog_ndk.z.so
36       libohimage.so
37       libimage_receiver.so
38       libnative_image.so
39       libohcamera.so
40       libnative_buffer.so
41   )
42   ```
43
443. Initialize an [ImageReceiver](../image/image-receiver-c.md) instance and obtain a surface ID.
45
46   Call **OH_ImageReceiverNative_Create** of the image module to create an OH_ImageReceiverNative instance, and call **OH_ImageReceiverNative_GetReceivingSurfaceId** of the instance to obtain a surface ID.
47
48   ```c++
49   void InitImageReceiver() {
50       OH_ImageReceiverOptions* options = nullptr;
51       // Capture exceptions and check whether the instance is null. This example shows only the API call process.
52       // Set image parameters.
53       Image_ErrorCode errCode = OH_ImageReceiverOptions_Create(&options);
54       Image_Size imgSize;
55       imgSize.width = 1080; // Width of the created preview stream.
56       imgSize.height = 1080; // Height of the created preview stream.
57       int32_t capacity = 8; // Maximum number of images in BufferQueue. The recommended value is 8.
58       errCode = OH_ImageReceiverOptions_SetSize(options, imgSize);
59       errCode = OH_ImageReceiverOptions_SetCapacity(options, capacity);
60       // Create an OH_ImageReceiverNative instance.
61       OH_ImageReceiverNative* receiver = nullptr;
62       errCode = OH_ImageReceiverNative_Create(options, &receiver);
63       // Obtain the Surface ID of the OH_ImageReceiverNative instance.
64       uint64_t receiverSurfaceID = 0;
65       errCode = OH_ImageReceiverNative_GetReceivingSurfaceId(receiver, &receiverSurfaceID);
66       OH_LOG_INFO(LOG_APP, "receiver surfaceID:%{public}lu", receiverSurfaceID);
67   }
68   ```
69
704. Create a preview stream based on the surface ID obtained. (Note that you must convert the surface ID type to char * before the creation of the preview stream.) For details, see step 4 in [Camera Preview (C/C++)](./native-camera-preview.md).
71
725. Create a session and enable it. For details, see [Camera Session Management (C/C++)](./native-camera-session-management.md).
73
746. Register an ImageReceiver callback to listen for the image content reported by each frame.
75
76   ```c++
77   OH_ImageReceiverNative* receiver; // Instance created in step 3.
78
79   // Image callback. For details, see Media/Image Kit.
80   static void OnCallback(OH_ImageReceiverNative* receiver) {
81       OH_LOG_INFO(LOG_APP, "ImageReceiverNativeCTest buffer available.");
82       // Capture exceptions and check whether the instance is null. This example shows only the API call process.
83       OH_ImageNative* image = nullptr;
84       // Obtain the image from the bufferQueue.
85       Image_ErrorCode errCode = OH_ImageReceiverNative_ReadNextImage(receiver, &image);
86       // Read the image width and height.
87       Image_Size size;
88       errCode = OH_ImageNative_GetImageSize(image, &size);
89       OH_LOG_INFO(LOG_APP, "OH_ImageNative_GetImageSize errCode:%{public}d width:%{public}d height:%{public}d", errCode,
90           size.width, size.height);
91
92       // Obtain the image's component type.
93       size_t typeSize = 0;
94       OH_ImageNative_GetComponentTypes(image, nullptr, &typeSize);
95       uint32_t* types = new uint32_t[typeSize];
96       OH_ImageNative_GetComponentTypes(image, &types, &typeSize);
97       uint32_t component = types[0];
98       // Obtain the image buffer.
99       OH_NativeBuffer* imageBuffer = nullptr;
100       errCode = OH_ImageNative_GetByteBuffer(image, component, &imageBuffer);
101       size_t bufferSize = 0;
102       errCode = OH_ImageNative_GetBufferSize(image, component, &bufferSize);
103       OH_LOG_INFO(LOG_APP, "ImageReceiverNativeCTest buffer component: %{public}d size:%{public}zu", component, bufferSize);
104       // Obtain the row stride of the image.
105       int32_t stride = 0;
106       errCode = OH_ImageNative_GetRowStride(image, component, &stride);
107       OH_LOG_INFO(LOG_APP, "ImageReceiverNativeCTest buffer stride: %{public}d.", stride);
108       void* srcVir = nullptr;
109       OH_NativeBuffer_Map(imageBuffer, &srcVir);
110       uint8_t* srcBuffer = static_cast<uint8_t*>(srcVir);
111       // Check whether the row stride is the same as the width of the preview stream. If they are different, consider the impact of the stride on buffer reading.
112       if (stride == size.width) {
113           // Process the buffer by calling the API that does not support stride.
114       } else {
115           // Process the buffer by calling the API that supports stride or remove the stride data.
116           // The following uses the operation of removing the stride data as an example. Specifically, remove the stride data from the byteBuffer and obtain a new dstBuffer by means of copy.
117           size_t dstBufferSize = size.width * size.height * 1.5; // Camera preview stream in NV21 format.
118           std::unique_ptr<uint8_t[]> dstBuffer = std::make_unique<uint8_t[]>(dstBufferSize);
119           uint8_t* dstPtr = dstBuffer.get();
120           for (int j = 0; j < size.height * 1.5; j++) {
121               memcpy(dstPtr, srcBuffer, size.width);
122               dstPtr += size.width;
123               srcBuffer += stride;
124           }
125           // Process the buffer by calling the API that does not support stride.
126       }
127       // Release resources.
128       OH_NativeBuffer_Unmap(imageBuffer); // Release the buffer to ensure that the bufferQueue is rotated properly.
129       errCode = OH_ImageNative_Release(image);
130	   delete[] types;
131   }
132
133   void OnImageReceiver() {
134       // Register the callback to listen for the image content reported by each frame.
135       Image_ErrorCode errCode = OH_ImageReceiverNative_On(receiver, OnCallback);
136   }
137   ```
138