• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Using ImageReceiver to Receive Images
2<!--Kit: Image Kit-->
3<!--Subsystem: Multimedia-->
4<!--Owner: @aulight02-->
5<!--SE: @liyang_bryan-->
6<!--TSE: @xchaosioda-->
7
8You can use the **ImageReceiver** class to obtain the surface ID of a component, read the latest image or the next image, and release ImageReceiver instances.
9
10> **NOTE**
11>
12> The ImageReceiver works as a consumer and needs a matching producer to provide data to work properly. Typical producers are the camera's photo or preview streams. The ImageReceiver merely serves as the recipient and consumer of images. The properties set in ImageReceiver, such as size and format, do not actually take effect. The parameters passed when creating the ImageReceiver object do not have a practical impact. Image properties need to be configured on the sending side (the producer), such as when setting up the [profile](../../reference/apis-camera-kit/arkts-apis-camera-i.md#profile) for a [camera preview stream](../../reference/apis-camera-kit/arkts-apis-camera-CameraManager.md#createpreviewoutput).
13
14The ImageReceiver can receive images from the camera's preview stream, allowing for [dual-channel preview](../camera/camera-dual-channel-preview.md).
15
16Read [Image](../../reference/apis-image-kit/arkts-apis-image-ImageReceiver.md) for APIs related to ImageReceiver.
17
18## How to Develop
19
20Create an ImageReceiver object, obtain the surface ID to create a preview stream, register image listeners, and process each frame of the image in the preview stream as required.
21
221. Create an ImageReceiver object, through which you can obtain the surface ID of the preview stream.
23
24    ```ts
25    import { image } from '@kit.ImageKit';
26    let imageWidth: number = 1920; // Use the width in the profile size supported by the device.
27    let imageHeight: number = 1080; // Use the height in the profile size supported by the device.
28
29    async function initImageReceiver():Promise<void>{
30      // Create an ImageReceiver object. The parameters in createImageReceiver do not have any impact on the received data.
31      let size: image.Size = { width: imageWidth, height: imageHeight };
32      let imageReceiver = image.createImageReceiver(size, image.ImageFormat.JPEG, 8);
33      // Obtain the preview stream surface ID.
34      let imageReceiverSurfaceId = await imageReceiver.getReceivingSurfaceId();
35      console.info(`initImageReceiver imageReceiverSurfaceId:${imageReceiverSurfaceId}`);
36    }
37    ```
38
392. Register a listener to process each frame of image data in the preview stream. Specifically, use the **imageArrival** event in the ImageReceiver object to obtain the image data returned by the bottom layer. For details, see [Image API Reference](../../reference/apis-image-kit/arkts-apis-image-ImageReceiver.md).
40
41    ```ts
42    import { BusinessError } from '@kit.BasicServicesKit';
43    import { image } from '@kit.ImageKit';
44
45    function onImageArrival(receiver: image.ImageReceiver): void {
46      // Subscribe to the imageArrival event.
47      receiver.on('imageArrival', () => {
48        // Obtain an image.
49        receiver.readNextImage((err: BusinessError, nextImage: image.Image) => {
50          if (err || nextImage === undefined) {
51            console.error('readNextImage failed');
52            return;
53          }
54          // Parse the image.
55          nextImage.getComponent(image.ComponentType.JPEG, async (err: BusinessError, imgComponent: image.Component) => {
56            if (err || imgComponent === undefined) {
57              console.error('getComponent failed');
58            }
59            if (imgComponent.byteBuffer) {
60              // For details, see the description of parsing the image buffer data below. This example uses method 1.
61              let width = nextImage.size.width; // Obtain the image width.
62              let height = nextImage.size.height; // Obtain the image height.
63              let stride = imgComponent.rowStride; // Obtain the image stride.
64              console.debug(`getComponent with width:${width} height:${height} stride:${stride}`);
65              // The value of stride is the same as that of width.
66              if (stride == width) {
67                let pixelMap = await image.createPixelMap(imgComponent.byteBuffer, {
68                  size: { height: height, width: width },
69                  srcPixelFormat: 8,
70                })
71              } else {
72                // The value of stride is different from that of width.
73                const dstBufferSize = width * height * 1.5
74                const dstArr = new Uint8Array(dstBufferSize)
75                for (let j = 0; j < height * 1.5; j++) {
76                  const srcBuf = new Uint8Array(imgComponent.byteBuffer, j * stride, width)
77                  dstArr.set(srcBuf, j * width)
78                }
79                let pixelMap = await image.createPixelMap(dstArr.buffer, {
80                  size: { height: height, width: width },
81                  srcPixelFormat: 8,
82                })
83              }
84            } else {
85              console.error('byteBuffer is null');
86            }
87            // Release the resource when the buffer is not in use.
88            // If an asynchronous operation is performed on the buffer, call nextImage.release() to release the resource after the asynchronous operation is complete.
89            nextImage.release();
90          })
91        })
92      })
93    }
94    ```
95
96
97    The following methods are available for parsing the image buffer data by using [image.Component](../../reference/apis-image-kit/arkts-apis-image-i.md#component9).
98
99    > **NOTE**
100    >
101    > Check whether the width of the image is the same as **rowStride**. If they are different, perform the following operations:
102
103    Method 1: Remove the stride data from **imgComponent.byteBuffer**, obtain a new buffer by means of copy, and process the buffer by calling the API that does not support stride.
104
105    ```ts
106    // For example, for NV21 (images in YUV_420_SP format), the formula for calculating the YUV_420_SP memory is as follows: YUV_420_SP memory = Width * Height + (Width * Height)/2.
107    const dstBufferSize = width * height * 1.5;
108    const dstArr = new Uint8Array(dstBufferSize);
109    // Read the buffer data line by line.
110    for (let j = 0; j < height * 1.5; j++) {
111      // Copy the first width bytes of each line of data in imgComponent.byteBuffer to dstArr.
112      const srcBuf = new Uint8Array(imgComponent.byteBuffer, j * stride, width);
113      dstArr.set(srcBuf, j * width);
114    }
115    let pixelMap = await image.createPixelMap(dstArr.buffer, {
116      size: { height: height, width: width }, srcPixelFormat: 8
117    });
118    ```
119
120    Method 2: Create a PixelMap based on the value of stride * height, and call **cropSync** of the PixelMap to crop redundant pixels.
121
122    ```ts
123    // Create a PixelMap, with width set to the value of stride.
124    let pixelMap = await image.createPixelMap(imgComponent.byteBuffer, {
125      size:{height: height, width: stride}, srcPixelFormat: 8});
126    // Crop extra pixels.
127    pixelMap.cropSync({size:{width:width, height:height}, x:0, y:0});
128    ```
129
130    Method 3: Pass **imgComponent.byteBuffer** and **stride** to the API that supports stride.
131