Name NV_stream_consumer_eglimage Name Strings EGL_NV_stream_consumer_eglimage Contributors Mukund Keshava James Jones Daniel Kartch Sandeep Shinde Pyarelal Knowles Leo Xu Contacts Mukund Keshava, NVIDIA (mkeshava 'at' nvidia.com) Status Draft Version Version 3 - November 27, 2019 Number EGL Extension #139 Extension Type EGL display extension Dependencies Requires the EGL_KHR_stream extension. Requires the EGL_KHR_reusable_sync extension. This extension is written against the wording of the EGL 1.5 Specification Overview An EGLStream consists of a sequence of image frames. This extension allows these frames to be acquired as EGLImages. Frames from the stream would be used as the content for the EGLImage. New Procedures and Functions EGLBoolean eglStreamImageConsumerConnectNV( EGLDisplay dpy, EGLStreamKHR stream, EGLint num_modifiers, EGLuint64KHR *modifiers, EGLAttrib* attrib_list); EGLint eglQueryStreamConsumerEventNV( EGLDisplay dpy, EGLStreamKHR stream, EGLTime timeout, EGLenum *event, EGLAttrib *aux); EGLBoolean eglStreamAcquireImageNV( EGLDisplay dpy, EGLStreamKHR stream, EGLImage *pImage, EGLSync sync); EGLBoolean eglStreamReleaseImageNV( EGLDisplay dpy, EGLStreamKHR stream, EGLImage image, EGLSync sync); New Tokens Accepted by the parameter of eglCreateImage: EGL_STREAM_CONSUMER_IMAGE_NV 0x3373 Returned as an from eglQueryStreamConsumerEventNV: EGL_STREAM_IMAGE_ADD_NV 0x3374 EGL_STREAM_IMAGE_REMOVE_NV 0x3375 EGL_STREAM_IMAGE_AVAILABLE_NV 0x3376 Add to section "3.9 EGLImage Specification and Management" of the EGL 1.5 Specification, in the description of eglCreateImage: "Values accepted for are listed in Table 3.10, below. +-------------------------------+-----------------------------------+ | | Notes | +-------------------------------+-----------------------------------+ | EGL_STREAM_CONSUMER_IMAGE_NV | Used with EGLStream objects | +-------------------------------+-----------------------------------+ Table 3.10: Legal values for eglCreateImage target parameter. If is EGL_STREAM_CONSUMER_IMAGE_NV, a new EGLImage will be created for the next consumer image frame in the EGLStream referenced by which is not currently bound to an EGLImage. If the stream's producer reuses memory buffers for multiple image frames, then an EGLImage obtained in this way will persist for the next image frame that uses the same buffer, unless destroyed in the interim. Otherwise, the user must create a new EGLImage for every frame. Creating the EGLImage does not guarantee that the image contents will be ready for use. The EGLImage must first be acquired from the stream after creation. If the EGLImage created for a consumer image frame is destroyed via eglDestroyImage, a new EGLImage needs to be created via eglCreateImage for the same consumer image frame. must be a valid initialized display. must be EGL_NO_CONTEXT. must be a handle to a valid EGLStream object, cast into the type EGLClientBuffer. Add to the list of error conditions for eglCreateImage: "* If is EGL_STREAM_CONSUMER_IMAGE_NV and is not a valid stream handle associated with , the error EGL_BAD_STREAM_KHR is generated. * If is EGL_STREAM_CONSUMER_IMAGE_NV, and is not EGL_NO_CONTEXT, the error EGL_BAD_PARAMETER is generated. * If is EGL_STREAM_CONSUMER_IMAGE_NV, and there are no buffers in the currently or if there are no buffers associated with the stream that are not already bound to EGLImages EGL_BAD_ACCESS is generated. eglCreateImage needs to be called with EGL_STREAM_CONSUMER_IMAGE_NV as the for every valid buffer in the EGLStream. Add section "3.10.2 Connecting an EGLStream to a consumer" in the EGL_KHR_stream extension with this: 3.10.2.2 EGLImage consumer Call EGLBoolean eglStreamImageConsumerConnectNV( EGLDisplay dpy, EGLStreamKHR stream, EGLint num_modifiers, EGLuint64KHR *modifiers, EGLAttrib* attrib_list); to connect the EGLImage consumer to the . An EGLImage consumer allows image frames inserted in the stream to be received as EGLImages, which can then be bound to any other object which supports EGLImage. For each image frame, an EGLImage must first be created as described in section "3.9 EGLImage Specification and Management" of the EGL 1.5 Specification, and then the frame contents must be latched to the EGLImage as described below. In the consumer can advertise an optional list of supported DRM modifiers as described in EXT_image_dma_buf_import_modifiers. This information could be used by the producer to generate consumer supported image frames. If not NULL, points to an array of name/value pairs, terminated by EGL_NONE. Currently no attributes are supported. On success, EGL_TRUE is returned. - state is set to EGL_STREAM_STATE_CONNECTING_KHR allowing the producer to be connected. On failure, EGL_FALSE is returned and an error is generated. - EGL_BAD_DISPLAY is generated if is not the handle of a valid EGLDisplay object. - EGL_BAD_STREAM_KHR is generated if is not a valid valid stream handle associated with . - EGL_BAD_STATE_KHR is generated if the state is not EGL_STREAM_STATE_CREATED_KHR before eglStreamImageConsumerConnectNV is called. Call EGLint eglQueryStreamConsumerEventNV( EGLDisplay dpy, EGLStreamKHR stream, EGLTime timeout, EGLenum *event, EGLAttrib *aux); to query the for the next pending event. eglQueryStreamConsumerEventNV returns in the event type and returns in additional data associated with some events. If no event is pending at the time eglQueryStreamConsumerEventNV is called, it will wait up to nanoseconds for one to arrive before returning. If is EGL_FOREVER, the function will not time out and will only return if an event arrives or the stream becomes disconnected. On success, EGL_TRUE is returned. A new event will be returned. The valid events are as follows: - EGL_STREAM_IMAGE_ADD_NV is returned if a buffer is present in the stream which has not yet been bound to an EGLImage with eglCreateImage. - EGL_STREAM_IMAGE_REMOVE_NV indicates that a buffer has been removed from the stream and its EGLImage, whose handle is returned in , can be destroyed when the consumer application no longer requires it. - EGL_STREAM_IMAGE_AVAILABLE_NV indicates that there is a new frame available in the stream that can be acquired via eglStreamAcquireImageNV. On failure, EGL_FALSE is returned and an error is generated and and are not modified. - EGL_BAD_STREAM_KHR is generated if is not a valid valid stream handle associated with . EGL_TIMEOUT_EXPIRED is returned if the duration is complete, and there are no valid events that occured in this duration. The and parameters are not modified. Call EGLBoolean eglStreamAcquireImageNV( EGLDisplay dpy, EGLStreamKHR stream, EGLImage *pImage, EGLSync sync); to "latch" the next image frame in the image stream from into an EGLImage. The consumer application needs to create a reusable EGLSync object using eglCreateSync with EGL_SYNC_STATUS set to EGL_SIGNALED prior to to this command. eglStreamAcquireImageNV accepts a handle to a previously created object. eglStreamAcquireImageNV will write into the object and this indicates when the producer will be done writing to the frame. It also resets the state of the object to unsignaled. If is EGL_NO_SYNC, the consumer ignores the sync object. On success, EGL_TRUE is returned. - will have the most recent frame from the On failure, eglStreamAcquireImageNV returns EGL_FALSE, and an error is generated. - EGL_BAD_DISPLAY is generated if is not a valid, initialized EGLDisplay. - EGL_BAD_STREAM_KHR is generated if is not a valid valid stream handle associated with . - EGL_BAD_ACCESS is generated if there are no frames in the that are available to acquire. Call EGLBoolean eglStreamReleaseImageNV( EGLDisplay dpy, EGLStreamKHR stream, EGLImage image, EGLSync sync); to release the frame back to the stream. This takes a that indicates when the consumer will be done using the frame. Before calling eglStreamReleaseImageNV, the needs to have previously been acquired with eglStreamAcquireImageNV. If is EGL_NO_SYNC, then the sync object is ignored. The eglStreamReleaseImageNV call makes a copy of the , so the caller is free to delete or reuse the as it chooses. On success, EGL_TRUE is returned, and the frame is successfully returned back to the stream. On failure, eglStreamReleaseImageNV returns EGL_FALSE, and an error is generated. - EGL_BAD_DISPLAY is generated if is not a valid, initialized EGLDisplay. - EGL_BAD_STATE_KHR is generated if is not in state EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR or EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR. - EGL_BAD_STREAM_KHR is generated if is not a valid EGLStream created for . - EGL_BAD_PARAMETER is generated if is either invalid, or is not held by the consumer. - EGL_BAD_PARAMETER is generated if is not a valid EGLSync object or EGL_NO_SYNC. If an acquired EGLImage has not yet released when eglDestroyImage is called, then, then an implicit eglStreamReleaseImageNV will be called. Add a new subsection 3.10.4.3.1 at the end of section "3.10.4.3 EGL_STREAM_STATE_KHR Attribute" in the EGL_KHR_stream extension spec: 3.10.4.3.1 Interaction with EGL_STREAM_STATE_KHR Image frames that have been presented to the stream on the producer side, but have not been bound to an EGLImage on the consumer side yet, do not affect the EGLStream state. If a new frame is presented to the stream, the stream state goes into EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR only if this frame is bound to an EGLImage on the consumer, and if it has not already been acquired. If an EGLImage bound on the consumer side has been destroyed via eglDestroyImage, then the stream goes into EGL_STREAM_STATE_EMPTY_KHR if there are no consumer frames left, that are bound to an EGLImage. Issues Revision History #3 (November 27, 2019) Mukund Keshava - Refined some subsections with more details #2 (November 22, 2019) Mukund Keshava - Refined some subsections with more details - Added new subsection 3.10.4.3.1 #1 (November 13, 2019) Mukund Keshava - initial draft