1Name 2 3 KHR_stream_producer_eglsurface 4 5Name Strings 6 7 EGL_KHR_stream_producer_eglsurface 8 9Contributors 10 11 Acorn Pooley 12 Jamie Gennis 13 Marcus Lorentzon 14 15Contacts 16 17 Acorn Pooley, NVIDIA (apooley 'at' nvidia.com) 18 19Notice 20 21 Copyright (c) 2011-2013 The Khronos Group Inc. Copyright terms at 22 http://www.khronos.org/registry/speccopyright.html 23 24Status 25 26 Complete. 27 Approved by the Khronos Board of Promoters on December 2, 2011. 28 29Version 30 31 Version 11, June 18, 2012 32 33Number 34 35 EGL Extension #34 36 37Dependencies 38 39 Requires EGL 1.2. 40 Requires OpenGL ES 1.1 or OpenGL ES 2.0. 41 42 Requires the EGL_KHR_stream extension. 43 44Overview 45 46 This extension allows an EGLSurface to be created as a producer of 47 images to an EGLStream. Each call to eglSwapBuffers posts a new 48 image frame into the EGLStream. 49 50New Procedures and Functions 51 52 EGLSurface eglCreateStreamProducerSurfaceKHR( 53 EGLDisplay dpy, 54 EGLConfig config, 55 EGLStreamKHR stream, 56 const EGLint *attrib_list) 57 58New Tokens 59 60 Bit that can appear in the EGL_SURFACE_TYPE of an EGLConfig: 61 62 EGL_STREAM_BIT_KHR 0x0800 63 64 65 66 67Add a row to "Table 3.2: Types of surfaces supported by an EGLConfig" 68in the EGL spec, right after the EGL_PBUFFER_BIT row: 69 70 EGL Token Name Description 71 -------------- -------------------------- 72 EGL_STREAM_BIT_KHR EGLConfig supports streams 73 74 75In the second paragraph of section "Other EGLConfig Attribute 76Description" in the EGL spec, replace 77 EGL_WINDOW_BIT | EGL_PIXMAP_BIT | EGL_PBUFFER_BIT 78with 79 EGL_WINDOW_BIT | EGL_PIXMAP_BIT | EGL_PBUFFER_BIT | EGL_STREAM_BIT_KHR 80and replace 81 "...cannot be used to create a pbuffer or pixmap." 82with 83 "...cannot be used to create a pbuffer, pixmap, or stream." 84 85 86Replace section "3.10.3.1 No way to connect producer to EGLStream" in 87the EGL_KHR_stream extension with this: 88 89 3.10.3.1 Stream Surface Producer 90 91 Call 92 93 EGLSurface eglCreateStreamProducerSurfaceKHR( 94 EGLDisplay dpy, 95 EGLConfig config, 96 EGLStreamKHR stream, 97 const EGLint *attrib_list) 98 99 to create an EGLSurface and connect it as the producer of 100 <stream>. 101 102 <attrib_list> specifies a list of attributes for <stream>. The 103 list has the same structure as described for eglChooseConfig. The 104 attributes EGL_WIDTH and EGL_HEIGHT must both be specified in the 105 <attrib_list>. 106 107 EGL_WIDTH and EGL_HEIGHT indicate the width and height 108 (respectively) of the images that makes up the stream. 109 110 The EGLSurface producer inserts an image frame into <stream> once 111 for each time it is passed to eglSwapBuffers(). The image frame 112 is inserted after the GL has finished previous rendering commands. 113 Refer to section "3.10.5 EGLStream operation" in the 114 EGL_KHR_stream extension specification for operation of the 115 EGLStream when an image frame is inserted into it. 116 117 If <stream> is not in the EGL_STREAM_STATE_EMPTY_KHR, 118 EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR, or 119 EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR when passed to 120 eglSwapBuffers(), then eglSwapBuffers will return EGL_FALSE and 121 generate an EGL_BAD_CURRENT_SURFACE error. 122 123 If the application would like to have the results of rendering 124 appear on the screen at a particular time then it must query the 125 value of EGL_CONSUMER_LATENCY_USEC_KHR after calling 126 eglCreateStreamProducerSurfaceKHR. This is the estimated time that 127 will elapse between the time the image frame is inserted into the 128 EGLStream and the time that the image frame will appear to the 129 user. 130 131 The image frame is not inserted into the EGLStream until the GL 132 has finished rendering it. Therefore predicting exactly when the 133 image frame will be inserted into the stream is nontrivial. 134 135 If it is critical that this frame of data reach the screen at a 136 particular point in time, then the application can 137 - render the frame (using GL/GLES commands) 138 - call glFinish (or use other synchronization techniques to 139 ensure rendering has completed). 140 - wait until the time that the frame should appear to the user 141 MINUS the value of EGL_CONSUMER_LATENCY_USEC_KHR. 142 - call eglSwapBuffers 143 This will allow the image frame to be inserted into the EGLStream 144 at the correct time ("Image Frame Intended Display Time" minus 145 "Consumer Latency") so that it will be displayed ("Image Frame 146 Actual Display Time" as close as possible to the desired time. 147 148 However, this will cause the GPU to operate in lockstep with the 149 CPU which can cause poor performance. In most cases it will be 150 more important for the image frame to appear to the user "as soon 151 as possible" rather than at a specific point in time. So in most 152 cases the application can ignore the value of 153 EGL_CONSUMER_LATENCY_USEC_KHR, not call glFinish, and not wait 154 before calling eglSwapBuffers. 155 156 On failure eglCreateStreamProducerSurfaceKHR returns EGL_NO_SURFACE 157 and generates an error. 158 159 - EGL_BAD_PARAMETER if EGL_WIDTH is not specified or is specified 160 with a value less than 1. 161 162 - EGL_BAD_PARAMETER if EGL_HEIGHT is not specified or is specified 163 with a value less than 1. 164 165 - EGL_BAD_STATE_KHR is generated if <stream> is not in state 166 EGL_STREAM_STATE_CONNECTING_KHR. 167 168 - EGL_BAD_MATCH is generated if <config> does not have the 169 EGL_STREAM_BIT_KHR set in EGL_SURFACE_TYPE. 170 171 - EGL_BAD_MATCH is generated if the implementation is not able to 172 convert color buffers described by <config> into image frames 173 that are acceptable by the consumer that is connected to 174 <stream>. 175 176 - EGL_BAD_STREAM_KHR is generated if <stream> is not a valid 177 EGLStream created for <dpy>. 178 179 - EGL_BAD_DISPLAY is generated if <dpy> is not a valid, 180 initialized EGLDisplay. 181 182Add a section preceding "3.9.3 Posting Semantics" in the EGL 183specification: 184 185 3.9.x Posting to a Stream 186 187 To post the color buffer to an EGLStream with an EGLSurface 188 producer, call 189 190 EGLBoolean eglSwapBuffers( 191 EGLDisplay dpy, 192 EGLSurface surface); 193 194 If <surface> is the producer of an EGLStream then the 195 contents of the color buffer are inserted as a new image frame 196 into the EGLStream. 197 198 When eglSwapBuffers returns the contents of the color buffer will 199 have been inserted into the EGLStream as described in section 200 "3.10.5 EGLStream operation" in the EGL_KHR_stream extension 201 specification, and the EGL_PRODUCER_FRAME_KHR attribute and 202 EGL_STREAM_STATE_KHR attribute values will reflect this. 203 204 The contents of the color buffer and all ancillary buffers are 205 always undefined after calling eglSwapBuffers. 206 207 eglSwapBuffers is never synchronized to a video frame when 208 <surface> is the producer for an EGLStream (it is as if the 209 swapinterval (set by eglSwapInterval, see below section "3.9.3 210 Posting Semantics") is 0). 211 212 It is implementation dependent whether eglSwapBuffers actually 213 waits for rendering to the color buffer to complete before 214 returning, but except for timing it must appear to the application 215 that all rendering to the EGLSurface (e.g. all previous gl 216 commands) completed before the image frame was inserted into the 217 EGLStream and eglSwapBuffers returned (as described below in 218 section "3.9.3 Posting Semantics"). 219 220 221Add to section "3.9.4 Posting Errors" in the EGL specification a new 222sentence as the 2nd to last sentence in the first paragraph: 223 224 If eglSwapBuffers is called and the EGLStream associated with 225 surface is no longer valid, an EGL_BAD_STREAM_KHR error is 226 generated. 227 228 229Issues 230 1. How many image frame buffers should be used? 231 232 DISCUSSION: 233 - leave up to implementation? 234 - leave up to producer? 235 - need hints from consumer? 236 - In practice 1, 2, and 3 buffers mean different semantics 237 which are visible to both the producer and consumer. Each 238 may be useful. I cannot think of a use for more than 3 239 buffers for EGL_KHR_stream_surface. (For a video producer 240 more than 3 often does make sense, but that is a different 241 extension.) 242 243 One possibility: expose EGL_BUFFER_COUNT_KHR to application. 244 245 It probably does not make sense to ever use more or less than 246 3 buffers. One that is the EGLSurface back buffer. One that 247 is waiting for the consumer to acquire. And one that the 248 consumer has acquired and is actively consuming. 249 250 RESOLVED: remove the EGL_BUFFER_COUNT_KHR parameter and always 251 use 3 buffers. This attribute can be added back with a 252 layered extension later if needed. 253 254 2. How is the resolution (width/height) of image frames set? 255 256 RESOLVED: The width and height are set with the required 257 EGL_WIDTH and EGL_HEIGHT attributes. These do not change for 258 the life of <stream>. 259 260 3. How is the image format, zbuffering, etc set? 261 262 RESOLVED: These are all determined by the <config>. These do 263 not change for the life of <stream>. 264 265 4. How does eglSwapBuffers act if there are already image frames 266 in the EGLStream when it is called. 267 268 RESOLVED: Frames are inserted into the EGLStream as described 269 in section "3.10.5 EGLStream operation" in the EGL_KHR_stream 270 extension specification. In particular: 271 272 If the value of EGL_STREAM_FIFO_LENGTH_KHR is 0 or if the 273 EGL_KHR_stream_fifo extension is not supported then the 274 new frame replaces any frames that already exist in the 275 EGLStream. If the consumer is already consuming a frame 276 then it continues to consume that same frame, but the next 277 time the consumer begins to consume a frame (e.g. the 278 next time eglStreamConsumerAcquireKHR() is called for a 279 gltexture consumer) the newly rendered image frame will be 280 consumed. (This is the standard behavior for ANY producer 281 when EGL_STREAM_FIFO_LENGTH_KHR is 0, described as "mailbox 282 mode"). 283 284 If the EGL_KHR_stream_fifo extension is supported and the 285 value of EGL_STREAM_FIFO_LENGTH_KHR is greater than 0 then 286 the newly rendered frame will be inserted into the 287 EGLStream. If the EGLStream is full (already contains 288 EGL_STREAM_FIFO_LENGTH_KHR frames) then eglSwapBuffers 289 will block until there is room in the fifo. Note that 290 this can deadlock if the consumer is running in the same 291 thread as the producer since the consumer will never be 292 able to consume a frame if the thread is blocked waiting 293 for room in the fifo. This fifo-related behavior is 294 described in the EGL_KHR_stream_fifo specification (this 295 behavior is not specific to this producer; it works the 296 same for all producers and all consumers). 297 298 All rendering commands must complete before the color 299 buffer is inserted into the EGLStream, or at least this is how 300 the behavior must appear to the application. 301 302 To be precise: when eglSwapBuffers returns the rendering 303 commands may or may not actually be complete, but the 304 following must all be true: 305 - The EGL_PRODUCER_FRAME_KHR value reflects the frame that 306 was just swapped by eglSwapBuffers 307 - The EGL_STREAM_STATE_KHR indicates that the image frame 308 is available (i.e. its value is 309 EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR) 310 - In mailbox mode if the consumer consumes a new frame it 311 will get this new frame (not an older frame). (For 312 example, with a EGL_NV_stream_consumer_gltexture 313 consumer, a call to eglStreamConsumerAcquireKHR() will 314 latch this new frame.) 315 - In fifo mode (see EGL_KHR_stream_fifo extension) if the 316 consumer consumes a new frame and all previous frames 317 have been consumed it will get this new frame (not an 318 older frame). (For example, with a 319 EGL_NV_stream_consumer_gltexture consumer, a call to 320 eglStreamConsumerAcquireKHR() will latch this new 321 frame.) 322 - If a consumer consumes the swapped frame, all GL (and 323 other API) commands called prior to eglSwapBuffers 324 will take effect on the image frame before the 325 consumer consumes it. In other words, the consumer 326 will never consume a partially rendered frame. (For 327 example, with EGL_NV_stream_consumer_gltexture 328 consumer, if the app does this: 329 eglSwapBuffers() // swap the producer EGLSurface 330 eglStreamConsumerAcquireKHR() // acquire the swapped image 331 glDrawArrays() // draw something using the texture 332 then the texture used in the glDrawArrays() command 333 will contain the image rendered by all gl (and/or 334 other API) commands preceding the eglSwapBuffers call 335 as if the app had called glFinish and/or eglWaitClient 336 just before calling eglSwapBuffers (but note that this 337 is implicit in eglSwapBuffers; the app does NOT need 338 to actually call glFinish or any other synchronization 339 functions in order to get this effect, and in fact 340 explicitly calling glFinish and/or eglWaitClient there 341 may significantly and negatively affect performance).) 342 343Revision History 344 345 #11 (June 18. 2012) Acorn Pooley 346 - Replace EGLStream with EGLStreamKHR in function prototypes. 347 348 #10 (June 15, 2012) Acorn Pooley 349 - Fix eglCreateStreamProducerSurfaceKHR name (was missing KHR) 350 351 #9 (October 17, 2011) Acorn Pooley 352 - Clarify issue 4 353 354 #8 (October 12, 2011) Acorn Pooley 355 - remove interactions with EGL_KHR_stream_fifo extension (they 356 are already decribed in that extension). 357 358 #7 (October 11, 2011) Acorn Pooley 359 - Add issue 4 360 - add changes to section 3.9 of the EGL spec to clarify 361 eglSwapBuffer behavior 362 363 #6 (October 4, 2011) Acorn Pooley 364 - Convert from an NV extension to a KHR extension 365 366 #5 (September 30, 2011) Acorn Pooley 367 - Remove EGL_BUFFER_COUNT_NV (0x321D) attribute and resolve issue 1. 368 369 #4 (September 27, 2011) Acorn Pooley 370 - Assign enum values (bug 8064) 371 372 #3 (July 6, 2011) Acorn Pooley 373 - Rename EGL_KHR_image_stream to EGL_KHR_stream 374 375 #2 (June 30, 2011) Acorn Pooley 376 - remove dependence on EGLImage 377 - clarify overview 378 - remove glossary (it can be seen in EGL_KHR_stream ext) 379 - Add EGL_STREAM_BIT 380 - clarify description 381 - describe attribute 382 383 #1 (April 20, 2011) Acorn Pooley 384 - initial draft 385 386# vim:ai:ts=4:sts=4:expandtab:textwidth=70 387