1Name 2 3 KHR_stream_fifo 4 5Name Strings 6 7 EGL_KHR_stream_fifo 8 9Contributors 10 11 Acorn Pooley 12 13Contacts 14 15 Acorn Pooley, NVIDIA (apooley 'at' nvidia.com) 16 17Notice 18 19 Copyright (c) 2011-2013 The Khronos Group Inc. Copyright terms at 20 http://www.khronos.org/registry/speccopyright.html 21 22Status 23 24 Complete. 25 Approved by the Khronos Board of Promoters on December 2, 2011. 26 27Version 28 29 Version 6, October 12, 2011 30 31Number 32 33 EGL Extension #36 34 35Dependencies 36 37 Requires EGL 1.2. 38 Requires the EGL_KHR_stream extension. 39 40 This extension is written based on the wording of the EGL 1.2 41 specification. 42 43 The EGL_KHR_stream_producer_eglsurface and 44 EGL_NV_stream_producer_eglsurface extensions affect the wording of 45 this extension. 46 47 The EGL_KHR_stream_producer_aldatalocator and 48 EGL_NV_stream_producer_aldatalocator extensions affect the wording 49 of this extension. 50 51 The EGL_KHR_stream_consumer_gltexture and 52 EGL_NV_stream_consumer_gltexture extensions affect the wording 53 of this extension. 54 55Overview 56 57 This extension allows an EGLStream to operate as a fifo rather 58 than as a mailbox. 59 60 The EGL_KHR_stream extension defines the EGLStream object. 61 The EGLStream object works like a 1 entry mailbox, allowing the 62 consumer to consume the frame that the producer most recently 63 inserted. If the consumer requests image frames faster than the 64 producer creates them then it gets the most recent one over and 65 over until a new one is inserted. If the producer inserts frames 66 faster than the consumer can consume them then the extra frames 67 are discarded. The producer is never stalled. 68 69 This extension allows an EGLStream to be placed into fifo mode. 70 In fifo mode no images are discarded. If the producer attempts to 71 insert a frame and the fifo is full then the producer will stall 72 until there is room in the fifo. When the consumer retrieves an 73 image frame from the EGLStream it will see the image frame that 74 immediately follows the image frame that it last retrieved (unless 75 no such frame has been inserted yet in which case it retrieves the 76 same image frame that it retrieved last time). 77 78 Timing of the EGLStream in mailbox mode, as described by the 79 EGL_KHR_stream extension, is the responsibility of the 80 producer (with help from the consumer in the form of the 81 EGL_CONSUMER_LATENCY_USEC_KHR hint). 82 83 In contrast, timing of an EGLStream in fifo mode is the 84 responsibility of the consumer. Each image frame in the fifo has 85 an associated timestamp set by the producer. The consumer can use 86 this timestamp to determine when the image frame is intended to be 87 displayed to the user. 88 89 90New Types 91 92 This type represents an absolute time in nanoseconds. 93 94 typedef khronos_utime_nanoseconds_t EGLTimeKHR 95 96New functions 97 98 EGLBoolean eglQueryStreamTimeKHR( 99 EGLDisplay dpy, 100 EGLStreamKHR stream, 101 EGLenum attribute, 102 EGLTimeKHR *value); 103 104New Tokens 105 106 Accepted as an attribute in the <attrib_list> parameter of 107 eglCreateStreamKHR and as the <attribute> parameter of 108 eglQueryStreamKHR. 109 110 EGL_STREAM_FIFO_LENGTH_KHR 0x31FC 111 112 These enums are accepted the <attribute> parameter of 113 eglQueryStreamTimeKHR. 114 115 EGL_STREAM_TIME_NOW_KHR 0x31FD 116 EGL_STREAM_TIME_CONSUMER_KHR 0x31FE 117 EGL_STREAM_TIME_PRODUCER_KHR 0x31FF 118 119Add 4 new entries to "Table 3.10.4.4 EGLStream Attributes" in the 120EGL_KHR_stream extension spec: 121 122 Attribute Read/Write Type Section 123 -------------------------- ---------- ---------- -------- 124 EGL_STREAM_FIFO_LENGTH_KHR io EGLint 3.10.4.xx 125 EGL_STREAM_TIME_NOW_KHR ro EGLTimeKHR 3.10.4.xx 126 EGL_STREAM_TIME_CONSUMER_KHR ro EGLTimeKHR 3.10.4.xx 127 EGL_STREAM_TIME_PRODUCER_KHR ro EGLTimeKHR 3.10.4.xx 128 129Add a new paragraph to the end of section "3.10.4.2 Querying EGLStream 130Attributes" in the EGL_KHR_stream extension. 131 132 Call 133 134 EGLBoolean eglQueryStreamTimeKHR( 135 EGLDisplay dpy, 136 EGLStreamKHR stream, 137 EGLenum attribute, 138 EGLTimeKHR *value); 139 140 to query <attribute> from <stream> for attributes whose type is 141 EGLTimeKHR. 142 143 If an error occurs EGL_FALSE is returned and an error is 144 generated. 145 146 - EGL_BAD_STREAM_KHR is generated if <stream> is not a valid 147 EGLStream created for <dpy>. 148 149 - EGL_BAD_ATTRIBUTE is generated if <attribute> is not a valid 150 EGLStream attribute with type EGLTimeKHR. 151 152 153Add new sections 3.1.4.xx at the end of section "3.10.4 EGLStream 154Attributes" in the EGL_KHR_stream extension. 155 156 3.1.4.x EGL_STREAM_FIFO_LENGTH_KHR Attribute 157 158 The EGL_STREAM_FIFO_LENGTH_KHR may be set in the <attrib_list> 159 parameter of eglCreateStreamKHR(), but is read-only once the 160 stream is created. It can be queried with eglQueryStreamKHR(). 161 Its default value is 0. Setting it to a value less than 0 162 generates an EGL_BAD_PARAMETER error. 163 164 When EGL_STREAM_FIFO_LENGTH_KHR is 0 the EGLStream operates in 165 mailbox mode as described in section "3.10.5.1 EGLStream operation 166 in mailbox mode" 167 168 When EGL_STREAM_FIFO_LENGTH_KHR is greater than 0 then the 169 EGLStream operates in fifo mode as described in section "3.10.5.2 170 EGLStream operation in fifo mode". 171 172 In fifo mode the EGLStream contains up to N image frames, where N 173 is the value of the EGL_STREAM_FIFO_LENGTH_KHR attribute. 174 175 The value of EGL_STREAM_FIFO_LENGTH_KHR is independent from the 176 number of internal buffers used by the producer. The producer may 177 require some number of internal buffers, but those are in addition 178 to the fifo buffers described by EGL_STREAM_FIFO_LENGTH_KHR. 179 180 3.1.4.x+1 EGL_STREAM_TIME_NOW_KHR Attribute 181 182 This indicates the current time. It is measured as the number of 183 nanoseconds since some arbitrary event (e.g. the last time the 184 system rebooted). 185 186 3.1.4.x+2 EGL_STREAM_TIME_CONSUMER_KHR Attribute 187 188 This indicates the timestamp of the image frame that the consumer 189 most recently consumed (i.e. frame number EGL_CONSUMER_FRAME_KHR). 190 The frame should first be displayed to the user when 191 EGL_STREAM_TIME_NOW_KHR matches this value. 192 193 In mailbox mode the timestamp for an image frame is always equal 194 to the time that the producer inserted the image frame into the 195 EGLStream, minus the value of EGL_CONSUMER_LATENCY_USEC_KHR. 196 197 In fifo mode the timestamp for an image frame is set by the 198 producer when it is inserted into the EGLStream. 199 200 The timestamp uses the same time units as EGL_STREAM_TIME_NOW_KHR. 201 202 3.1.4.x+3 EGL_STREAM_TIME_PRODUCER_KHR Attribute 203 204 This indicates the timestamp of the image frame that the producer 205 most recently inserted into the EGLStream (i.e. frame number 206 EGL_PRODUCER_FRAME_KHR). 207 208 209Modify the first sentence of section "3.10.5.1 EGLStream operation in 210mailbox mode" in the EGL_KHR_stream extension to: 211 212 When the EGL_STREAM_FIFO_LENGTH_KHR attribute is 0 213 then the EGLStream conceptually operates as a mailbox. 214 215 216Add a new section after section "3.10.5.1 EGLStream operation in 217mailbox mode" in the EGL_KHR_stream extension. 218 219 3.10.5.2 EGLStream operation in fifo mode 220 221 When the EGL_STREAM_FIFO_LENGTH_KHR attribute is greater than 0 222 then the EGLStream operates in fifo mode. The length of the fifo 223 is the value of the EGL_STREAM_FIFO_LENGTH_KHR attribute. 224 225 In fifo mode the EGLStream conceptually operates as a fifo. 226 227 When the consumer wants to consume a new image frame, behavior 228 depends on the state of the EGLStream. If the state is 229 EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR then the fifo is not 230 empty and the image frame to consume is removed from the tail of 231 the fifo. If the state is 232 EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR then the fifo is empty 233 and the consumer consumes the same frame that it most recently 234 consumed. Otherwise there are no image frames available to 235 consume (behavior in this case is described in the documentation 236 for each type of consumer - see section "3.10.2 Connecting an 237 EGLStream to a consumer"). 238 239 If the fifo is empty when the consumer is finished consuming an 240 image frame then the consumer holds on to the image frame in case 241 it needs to be consumed again later (this happens if the consumer 242 wants to consume another image frame before the producer has 243 inserted a new image frame into the fifo). In this case the state 244 of the EGLStream will be EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR 245 until the producer inserts a new image frame (or until the state 246 becomes EGL_STREAM_STATE_DISCONNECTED_KHR). 247 248 The producer inserts image frames at the head of the fifo. If the 249 fifo is full (already contains <L> image frames, where <L> is the 250 value of the EGL_STREAM_FIFO_LENGTH_KHR attribute) then producer 251 is stalled until the fifo is no longer full. When the fifo is not 252 empty the EGLStream state is 253 EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR. 254 255 This operation implies: 256 257 - Frames are never discarded until the consumer has examined 258 them. 259 260 - If the consumer consumes frames slower than the producer 261 inserts frames, then the producer will stall. 262 263 - If the consumer consumes frames faster than the producer 264 inserts frames, then the consumer may see some frames more 265 than once. 266 267 - The consumer can see each frame exactly once if it always 268 waits until the stream is in the 269 EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR state before 270 retrieving an image from the stream. 271 272 In mailbox mode the producer is responsible for timing. In fifo 273 mode the consumer is responsible for timing. 274 275 In fifo mode the producer marks each image frame with a timestamp. 276 The timestamp indicates at what time the image frame should first 277 be visible to the user. Exactly how a producer sets the timestamp 278 is described in the documentation for each type of producer. If 279 the value of an image frame's timestamp is T then the producer 280 must insert that image frame *before* time 281 T - EGL_CONSUMER_LATENCY_USEC_KHR 282 Image frames must be inserted in increasing timestamp order. 283 284 The consumer is responsible for presenting each image frame to the 285 user at the time indicated by its timestamp. The consumer should 286 indicate its minimum latency to the producer by setting the 287 EGL_CONSUMER_LATENCY_USEC_KHR attribute. 288 289 290If the EGL_KHR_stream_producer_eglsurface or 291EGL_NV_stream_producer_eglsurface extension is present then add a 292paragraph to the end of section "3.10.3.1 Stream Surface Producer" 293from that extension: 294 295 If <stream>'s EGL_STREAM_FIFO_LENGTH_KHR value is nonzero then 296 <stream> operates in fifo mode. Each time the EGLSurface is 297 passed to eglSwapBuffers() an image frame is inserted into the 298 fifo. The eglSwapBuffers call sets the timestamp of the image 299 frame to the time that eglSwapBuffers was called PLUS the value of 300 the EGL_CONSUMER_LATENCY_USEC_KHR attribute. 301 302If the EGL_KHR_stream_producer_eglsurface or 303EGL_NV_stream_producer_eglsurface extension is present then add a 304paragraph to section "3.9.x Posting to a Stream" 305from that extension, between the 2nd paragraph (which begins "If 306<surface> is the producer of an EGLStream...") and the 3rd paragraph 307(which begins "When eglSwapBuffers returns the contents..."): 308 309 If the value of the EGL_STREAM_FIFO_LENGTH_KHR attribute, <L> is 310 greater than zero, and there are already <L> image frames in the 311 EGLStream fifo, then the eglSwapBuffers function blocks (does not 312 return and does not insert the new image frame) until there is 313 room in the EGLStream fifo (i.e. there are less than <L> image 314 frames in the fifo). 315 316If the EGL_KHR_stream_producer_aldatalocator or 317EGL_NV_stream_producer_aldatalocator extension is present then replace 318the 2nd to last paragraph (the one that starts "The OpenMAX AL object 319will use the value of...") of section "3.10.3.1 OpenMAX AL Stream 320Producer" from that extension with the following 2 paragraphs: 321 322 If <stream>'s EGL_STREAM_FIFO_LENGTH_KHR value is zero then the 323 stream operates in mailbox mode. The OpenMAX AL object will use 324 the value of the EGL_CONSUMER_LATENCY_USEC_KHR attribute of 325 <stream> to determine when to insert each image frame. If the 326 EGL_CONSUMER_LATENCY_USEC_KHR attribute is modified (by the 327 consumer and/or by the application) then then OpenMAX AL object 328 will adjust its timing within 500 milliseconds of the change. If 329 an image frame is intended to appear to the user at time T (e.g. 330 so that it is synchronized with audio) then the OpenMAX AL object 331 must insert the image frame at time 332 T - EGL_CONSUMER_LATENCY_USEC_KHR 333 and set the image frame's timestamp to T. 334 335 If the <stream>'s EGL_STREAM_FIFO_LENGTH_KHR value is nonzero then 336 <stream> operates in fifo mode. If an image frame is intended to 337 appear to the user at time T then the OpenMAX AL object will 338 insert the image frame into the fifo before time 339 T - EGL_CONSUMER_LATENCY_USEC_KHR 340 and set the image frame's timestamp to T. 341 342If the EGL_KHR_stream_consumer_gltexture or 343EGL_NV_stream_consumer_gltexture extension is present then replace the 3443rd to last paragraph (the one that starts "If the producer has not 345inserted any new image frames...") of section "3.10.2.1 GL Texture 346External consumer" from that extension with the following 2 347paragraphs: 348 349 When <stream>'s EGL_STREAM_FIFO_LENGTH_KHR value is zero then the 350 stream operates in mailbox mode. If the producer has not inserted 351 any new image frames since the last call to 352 eglStreamConsumerAcquireNV then eglStreamConsumerAcquireNV will 353 "latch" the same image frame it latched last time 354 eglStreamConsumerAcquireNV was called. If the producer has 355 inserted one new image frame since the last call to 356 eglStreamConsumerAcquireNV then the eglStreamConsumerAcquireNV 357 will "latch" the newly inserted image frame. If the producer has 358 inserted more than one new image frame since the last call to 359 eglStreamConsumerAcquireNV then all but the most recently inserted 360 image frames are discarded and the producer will "latch" the most 361 recently inserted image frame. 362 363 When <stream>'s EGL_STREAM_FIFO_LENGTH_KHR value is nonzero then 364 <stream> operates in fifo mode. Each call to 365 eglStreamConsumerAcquireNV "latches" the next image frame in the 366 fifo into the OpenGL texture, removing that image frame from the 367 fifo. If there are no new image frames in the fifo then 368 eglStreamConsumerAcquireNV will "latch" the same image frame it 369 latched last time eglStreamConsumerAcquireNV was called. 370 371 372Issues 373 1. Is this extension useful? 374 375 RESOLVED: Yes. Browser vendors and others have expressed 376 interest. 377 378 2. Why not include this functionality in the base EGL_KHR_stream 379 extension? 380 381 RESOLVED: Including it there was confusing. Several 382 developers interested in EGLStream have thought at first that 383 they want to use EGLStreams in fifo mode. Later after 384 thinking about it they realize standard mode (non-fifo or 385 "mailbox" mode) is more useful. 386 387 Mailbox mode is easier to use and is less confusing for 388 aldatalocator-producer, gltexture-consumer usecase which was 389 the primary usecase for the extension at the time it was 390 devised. 391 392 Trying to describe both mailbox mode and fifo mode in 393 the same extension made the extension complicated. It was 394 confusing when the timestamps were useful (only in fifo mode). 395 It was confusing how the EGL_CONSUMER_LATENCY_USEC_KHR 396 attribute worked in different modes. 397 398 these problems the fifo functionality was split into this 399 separate extension. This also allows existing consumer and 400 producer extensions to be defined in terms of mailbox mode, 401 simplifying them and making them easier to understand. Then 402 interactions with fifo mode can be described separately. 403 404 Also, the fifo mode is more complicated to use and implement than 405 the mailbox mode. It was thought that there might be problems 406 with the fifo mode that could lead to a new extension 407 replacing the fifo mode extension. By keeping the fifo mode 408 functionality segregated into its own extension this would be 409 easier to accomplish. 410 411Revision History 412 413 #6 (October 12, 2011) Acorn Pooley 414 - Clarify fifo mode operation. (Does not change behavior.) 415 416 417 #5 (October 11, 2011) Acorn Pooley 418 - Resolve issue 1 419 - fix typos 420 - add issue 2 421 422 #4 (September 27, 2011) Acorn Pooley 423 - Assign enum values (bug 8064) 424 425 #3 (July 6, 2011) Acorn Pooley 426 - Rename EGL_KHR_image_stream to EGL_KHR_stream 427 428 #2 (version #2 skipped) 429 430 #1 (July 1, 2011) Acorn Pooley 431 - Initial draft 432 433# vim:ai:ts=4:sts=4:expandtab:textwidth=70 434