1# NativeImage Development 2 3## When to Use 4 5**NativeImage** is a module for associating a surface with a OpenGL external texture. It functions as the consumer of a graphics queue. It provides APIs for you to obtain and use a buffer, and output the buffer content to a OpenGL external texture. 6 7The following scenario is common for **NativeImage** development: 8 9Use the native APIs provided by **NativeImage** to create a **NativeImage** instance as the consumer and obtain the corresponding **NativeWindow** instance (functioning as the producer). Use the APIs provided by **NativeWindow** to fill in and flush the buffer, and then use the APIs provided by **NativeImage** to update the buffer content to a OpenGL ES texture. The **NativeImage** module must be used together with the **NativeWindow**, **NativeBuffer**, **EGL**, and **GLES3** modules. 10 11## Available APIs 12 13| API| Description| 14| -------- | -------- | 15| OH_NativeImage_Create (uint32_t textureId, uint32_t textureTarget) | Creates an **OH_NativeImage** instance to be associated with the OpenGL ES texture ID and target.| 16| OH_NativeImage_AcquireNativeWindow (OH_NativeImage \*image) | Obtains a **NativeWindow** instance associated with an **OH_NativeImage** instance. You need to call **OH_NativeWindow_DestroyNativeWindow** to release the **NativeWindow** instance when it is no longer needed.| 17| OH_NativeImage_AttachContext (OH_NativeImage \*image, uint32_t textureId) | Attaches an **OH_NativeImage** instance to the current OpenGL ES context. The OpenGL ES texture will be bound to an **GL_TEXTURE_EXTERNAL_OES** instance and updated through the **OH_NativeImage** instance.| 18| OH_NativeImage_DetachContext (OH_NativeImage \*image) | Detaches an **OH_NativeImage** instance from the current OpenGL ES context.| 19| OH_NativeImage_UpdateSurfaceImage (OH_NativeImage \*image) | Updates the OpenGL ES texture associated with the latest frame through an **OH_NativeImage** instance.| 20| OH_NativeImage_GetTimestamp (OH_NativeImage \*image) | Obtains the timestamp of the texture image that recently called the **OH_NativeImage_UpdateSurfaceImage** function.| 21| OH_NativeImage_GetTransformMatrix (OH_NativeImage \*image, float matrix[16]) | Obtains the transform matrix of the texture image that recently called the **OH_NativeImage_UpdateSurfaceImage** function.| 22| OH_NativeImage_Destroy (OH_NativeImage \*\*image) | Destroys an **OH_NativeImage** instance created by calling **OH_NativeImage_Create**. After the instance is destroyed, the pointer to the **OH_NativeImage** instance is assigned **NULL**.| 23 24For details about the APIs, see [native_image](../reference/native-apis/_o_h___native_image.md). 25 26## How to Develop 27 28The following steps describe how to use the native APIs provided by **NativeImage** to create an **OH_NativeImage** instance as the consumer and update the data to a OpenGL external texture. 29 30**Adding Dynamic Link Libraries** 31 32Add the following libraries to **CMakeLists.txt**: 33```txt 34libEGL.so 35libGLESv3.so 36libnative_image.so 37libnative_window.so 38libnative_buffer.so 39``` 40 41**Header File** 42```c++ 43#include <EGL/egl.h> 44#include <EGL/eglext.h> 45#include <GLES3/gl3.h> 46#include <native_image/native_image.h> 47#include <native_window/external_window.h> 48#include <native_buffer/native_buffer.h> 49``` 50 511. Initialize the EGL environment. 52 53 Refer to the code snippet below. 54 ```c++ 55 #include <iostream> 56 #include <string> 57 #include <EGL/egl.h> 58 #include <EGL/eglext.h> 59 60 using GetPlatformDisplayExt = PFNEGLGETPLATFORMDISPLAYEXTPROC; 61 constexpr const char* EGL_EXT_PLATFORM_WAYLAND = "EGL_EXT_platform_wayland"; 62 constexpr const char* EGL_KHR_PLATFORM_WAYLAND = "EGL_KHR_platform_wayland"; 63 constexpr int32_t EGL_CONTEXT_CLIENT_VERSION_NUM = 2; 64 constexpr char CHARACTER_WHITESPACE = ' '; 65 constexpr const char* CHARACTER_STRING_WHITESPACE = " "; 66 constexpr const char* EGL_GET_PLATFORM_DISPLAY_EXT = "eglGetPlatformDisplayEXT"; 67 EGLContext eglContext_ = EGL_NO_CONTEXT; 68 EGLDisplay eglDisplay_ = EGL_NO_DISPLAY; 69 static inline EGLConfig config_; 70 71 // Check the EGL extension. 72 static bool CheckEglExtension(const char* extensions, const char* extension) 73 { 74 size_t extlen = strlen(extension); 75 const char* end = extensions + strlen(extensions); 76 77 while (extensions < end) { 78 size_t n = 0; 79 if (*extensions == CHARACTER_WHITESPACE) { 80 extensions++; 81 continue; 82 } 83 n = strcspn(extensions, CHARACTER_STRING_WHITESPACE); 84 if (n == extlen && strncmp(extension, extensions, n) == 0) { 85 return true; 86 } 87 extensions += n; 88 } 89 return false; 90 } 91 92 // Obtain the display. 93 static EGLDisplay GetPlatformEglDisplay(EGLenum platform, void* native_display, const EGLint* attrib_list) 94 { 95 static GetPlatformDisplayExt eglGetPlatformDisplayExt = NULL; 96 97 if (!eglGetPlatformDisplayExt) { 98 const char* extensions = eglQueryString(EGL_NO_DISPLAY, EGL_EXTENSIONS); 99 if (extensions && 100 (CheckEglExtension(extensions, EGL_EXT_PLATFORM_WAYLAND) || 101 CheckEglExtension(extensions, EGL_KHR_PLATFORM_WAYLAND))) { 102 eglGetPlatformDisplayExt = (GetPlatformDisplayExt)eglGetProcAddress(EGL_GET_PLATFORM_DISPLAY_EXT); 103 } 104 } 105 106 if (eglGetPlatformDisplayExt) { 107 return eglGetPlatformDisplayExt(platform, native_display, attrib_list); 108 } 109 110 return eglGetDisplay((EGLNativeDisplayType)native_display); 111 } 112 113 static void InitEGLEnv() 114 { 115 // Obtain the display. 116 eglDisplay_ = GetPlatformEglDisplay(EGL_PLATFORM_OHOS_KHR, EGL_DEFAULT_DISPLAY, NULL); 117 if (eglDisplay_ == EGL_NO_DISPLAY) { 118 std::cout << "Failed to create EGLDisplay gl errno : " << eglGetError() << std::endl; 119 } 120 121 EGLint major, minor; 122 // Initialize the EGL display. 123 if (eglInitialize(eglDisplay_, &major, &minor) == EGL_FALSE) { 124 std::cout << "Failed to initialize EGLDisplay" << std::endl; 125 } 126 127 // Bind the OpenGL ES API. 128 if (eglBindAPI(EGL_OPENGL_ES_API) == EGL_FALSE) { 129 std::cout << "Failed to bind OpenGL ES API" << std::endl; 130 } 131 132 unsigned int ret; 133 EGLint count; 134 EGLint config_attribs[] = { EGL_SURFACE_TYPE, EGL_WINDOW_BIT, EGL_RED_SIZE, 8, EGL_GREEN_SIZE, 8, EGL_BLUE_SIZE, 8, 135 EGL_ALPHA_SIZE, 8, EGL_RENDERABLE_TYPE, EGL_OPENGL_ES3_BIT, EGL_NONE }; 136 137 // Obtain a valid system configuration. 138 ret = eglChooseConfig(eglDisplay_, config_attribs, &config_, 1, &count); 139 if (!(ret && static_cast<unsigned int>(count) >= 1)) { 140 std::cout << "Failed to eglChooseConfig" << std::endl; 141 } 142 143 static const EGLint context_attribs[] = { EGL_CONTEXT_CLIENT_VERSION, EGL_CONTEXT_CLIENT_VERSION_NUM, EGL_NONE }; 144 145 // Create a context. 146 eglContext_ = eglCreateContext(eglDisplay_, config_, EGL_NO_CONTEXT, context_attribs); 147 if (eglContext_ == EGL_NO_CONTEXT) { 148 std::cout << "Failed to create egl context %{public}x, error:" << eglGetError() << std::endl; 149 } 150 151 // Associate the context. 152 eglMakeCurrent(eglDisplay_, EGL_NO_SURFACE, EGL_NO_SURFACE, eglContext_); 153 154 // The EGL environment initialization is complete. 155 std::cout << "Create EGL context successfully, version" << major << "." << minor << std::endl; 156 } 157 ``` 158 1592. Create an **OH_NativeImage** instance. 160 ```c++ 161 // Create an OpenGL ES texture. 162 GLuint textureId; 163 glGenTextures(1, &textureId); 164 // Create an OH_NativeImage instance, which will be associated with an OpenGL ES texture. 165 OH_NativeImage* image = OH_NativeImage_Create(textureId, GL_TEXTURE_2D); 166 ``` 167 1683. Obtain a **NativeWindow** instance that functions as the producer. 169 ```c++ 170 // Obtain a NativeWindow instance. 171 OHNativeWindow* nativeWindow = OH_NativeImage_AcquireNativeWindow(image); 172 ``` 173 1744. Sets the width and height of the native window. 175 ```c++ 176 int code = SET_BUFFER_GEOMETRY; 177 int32_t width = 800; 178 int32_t height = 600; 179 int32_t ret = OH_NativeWindow_NativeWindowHandleOpt(nativeWindow, code, width, height); 180 ``` 181 1825. Write the produced content to a **NativeWindowBuffer** instance. 183 1. Obtain a NativeWindowBuffer instance from the NativeWindow instance. 184 ```c++ 185 OHNativeWindowBuffer* buffer = nullptr; 186 int fenceFd; 187 // Obtain a NativeWindowBuffer instance by calling OH_NativeWindow_NativeWindowRequestBuffer. 188 OH_NativeWindow_NativeWindowRequestBuffer(nativeWindow, &buffer, &fenceFd); 189 190 BufferHandle *handle = OH_NativeWindow_GetBufferHandleFromNative(buffer); 191 ``` 192 2. Write the produced content to the **NativeWindowBuffer** instance. 193 ```c++ 194 #include <sys/mman.h> 195 196 // Use mmap() to obtain the memory virtual address of buffer handle. 197 void* mappedAddr = mmap(handle->virAddr, handle->size, PROT_READ | PROT_WRITE, MAP_SHARED, handle->fd, 0); 198 if (mappedAddr == MAP_FAILED) { 199 // mmap failed 200 } 201 static uint32_t value = 0x00; 202 value++; 203 uint32_t *pixel = static_cast<uint32_t *>(mappedAddr); 204 for (uint32_t x = 0; x < width; x++) { 205 for (uint32_t y = 0; y < height; y++) { 206 *pixel++ = value; 207 } 208 } 209 // Unmap the memory when the memory is no longer required. 210 int result = munmap(mappedAddr, handle->size); 211 if (result == -1) { 212 // munmap failed 213 } 214 ``` 215 3. Flush the **NativeWindowBuffer** to the **NativeWindow**. 216 ```c++ 217 // Set the refresh region. If Rect in Region is a null pointer or rectNumber is 0, all contents in the NativeWindowBuffer are changed. 218 Region region{nullptr, 0}; 219 // Flush the buffer to the consumer through OH_NativeWindow_NativeWindowFlushBuffer, for example, by displaying it on the screen. 220 OH_NativeWindow_NativeWindowFlushBuffer(nativeWindow, buffer, fenceFd, region); 221 ``` 222 4. Destroy the **NativeWindow** instance when it is no longer needed. 223 ```c++ 224 OH_NativeWindow_DestroyNativeWindow(nativeWindow); 225 ``` 226 2276. Update the content to the OpenGL texture. 228 ```c++ 229 // Update the content to the OpenGL texture. 230 ret = OH_NativeImage_UpdateSurfaceImage(image); 231 if (ret != 0) { 232 std::cout << "OH_NativeImage_UpdateSurfaceImage failed" << std::endl; 233 } 234 // Obtain the timestamp and transform matrix of the texture image that recently called the **OH_NativeImage_UpdateSurfaceImage** function. 235 int64_t timeStamp = OH_NativeImage_GetTimestamp(image); 236 float matrix[16]; 237 ret = OH_NativeImage_GetTransformMatrix(image, matrix); 238 if (ret != 0) { 239 std::cout << "OH_NativeImage_GetTransformMatrix failed" << std::endl; 240 } 241 ``` 242 2437. Unbind the OpenGL texture and bind it to a new external texture. 244 ```c++ 245 // Detach an OH_NativeImage instance from the current OpenGL ES context. 246 ret = OH_NativeImage_DetachContext(image); 247 if (ret != 0) { 248 std::cout << "OH_NativeImage_DetachContext failed" << std::endl; 249 } 250 // Attach the OH_NativeImage instance to the current OpenGL ES context. The OpenGL ES texture will be bound to an GL_TEXTURE_EXTERNAL_OES instance and updated through the OH_NativeImage instance. 251 GLuint textureId2; 252 glGenTextures(1, &textureId2); 253 ret = OH_NativeImage_AttachContext(image, textureId2); 254 ``` 255 2568. Destroy the **OH_NativeImage** instance when it is no longer needed. 257 ```c++ 258 // Destroy the OH_NativeImage instance. 259 OH_NativeImage_Destroy(&image); 260 ``` 261