1Name 2 3 EXT_image_dma_buf_import 4 5Name Strings 6 7 EGL_EXT_image_dma_buf_import 8 9Contributors 10 11 Jesse Barker 12 Rob Clark 13 Tom Cooksey 14 15Contacts 16 17 Jesse Barker (jesse 'dot' barker 'at' linaro 'dot' org) 18 Tom Cooksey (tom 'dot' cooksey 'at' arm 'dot' com) 19 20Status 21 22 Complete. 23 24Version 25 26 Version 7, December 13, 2013 27 28Number 29 30 EGL Extension #53 31 32Dependencies 33 34 EGL 1.2 is required. 35 36 EGL_KHR_image_base is required. 37 38 The EGL implementation must be running on a Linux kernel supporting the 39 dma_buf buffer sharing mechanism. 40 41 This extension is written against the wording of the EGL 1.2 Specification. 42 43Overview 44 45 This extension allows creating an EGLImage from a Linux dma_buf file 46 descriptor or multiple file descriptors in the case of multi-plane YUV 47 images. 48 49New Types 50 51 None 52 53New Procedures and Functions 54 55 None 56 57New Tokens 58 59 Accepted by the <target> parameter of eglCreateImageKHR: 60 61 EGL_LINUX_DMA_BUF_EXT 0x3270 62 63 Accepted as an attribute in the <attrib_list> parameter of 64 eglCreateImageKHR: 65 66 EGL_LINUX_DRM_FOURCC_EXT 0x3271 67 EGL_DMA_BUF_PLANE0_FD_EXT 0x3272 68 EGL_DMA_BUF_PLANE0_OFFSET_EXT 0x3273 69 EGL_DMA_BUF_PLANE0_PITCH_EXT 0x3274 70 EGL_DMA_BUF_PLANE1_FD_EXT 0x3275 71 EGL_DMA_BUF_PLANE1_OFFSET_EXT 0x3276 72 EGL_DMA_BUF_PLANE1_PITCH_EXT 0x3277 73 EGL_DMA_BUF_PLANE2_FD_EXT 0x3278 74 EGL_DMA_BUF_PLANE2_OFFSET_EXT 0x3279 75 EGL_DMA_BUF_PLANE2_PITCH_EXT 0x327A 76 EGL_YUV_COLOR_SPACE_HINT_EXT 0x327B 77 EGL_SAMPLE_RANGE_HINT_EXT 0x327C 78 EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT 0x327D 79 EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT 0x327E 80 81 Accepted as the value for the EGL_YUV_COLOR_SPACE_HINT_EXT attribute: 82 83 EGL_ITU_REC601_EXT 0x327F 84 EGL_ITU_REC709_EXT 0x3280 85 EGL_ITU_REC2020_EXT 0x3281 86 87 Accepted as the value for the EGL_SAMPLE_RANGE_HINT_EXT attribute: 88 89 EGL_YUV_FULL_RANGE_EXT 0x3282 90 EGL_YUV_NARROW_RANGE_EXT 0x3283 91 92 Accepted as the value for the EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT & 93 EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT attributes: 94 95 EGL_YUV_CHROMA_SITING_0_EXT 0x3284 96 EGL_YUV_CHROMA_SITING_0_5_EXT 0x3285 97 98 99Additions to Chapter 2 of the EGL 1.2 Specification (EGL Operation) 100 101 Add to section 2.5.1 "EGLImage Specification" (as defined by the 102 EGL_KHR_image_base specification), in the description of 103 eglCreateImageKHR: 104 105 "Values accepted for <target> are listed in Table aaa, below. 106 107 +-------------------------+--------------------------------------------+ 108 | <target> | Notes | 109 +-------------------------+--------------------------------------------+ 110 | EGL_LINUX_DMA_BUF_EXT | Used for EGLImages imported from Linux | 111 | | dma_buf file descriptors | 112 +-------------------------+--------------------------------------------+ 113 Table aaa. Legal values for eglCreateImageKHR <target> parameter 114 115 ... 116 117 If <target> is EGL_LINUX_DMA_BUF_EXT, <dpy> must be a valid display, <ctx> 118 must be EGL_NO_CONTEXT, and <buffer> must be NULL, cast into the type 119 EGLClientBuffer. The details of the image is specified by the attributes 120 passed into eglCreateImageKHR. Required attributes and their values are as 121 follows: 122 123 * EGL_WIDTH & EGL_HEIGHT: The logical dimensions of the buffer in pixels 124 125 * EGL_LINUX_DRM_FOURCC_EXT: The pixel format of the buffer, as specified 126 by drm_fourcc.h and used as the pixel_format parameter of the 127 drm_mode_fb_cmd2 ioctl. 128 129 * EGL_DMA_BUF_PLANE0_FD_EXT: The dma_buf file descriptor of plane 0 of 130 the image. 131 132 * EGL_DMA_BUF_PLANE0_OFFSET_EXT: The offset from the start of the 133 dma_buf of the first sample in plane 0, in bytes. 134 135 * EGL_DMA_BUF_PLANE0_PITCH_EXT: The number of bytes between the start of 136 subsequent rows of samples in plane 0. May have special meaning for 137 non-linear formats. 138 139 For images in an RGB color-space or those using a single-plane YUV format, 140 only the first plane's file descriptor, offset & pitch should be specified. 141 For semi-planar YUV formats, that first plane (plane 0) holds only the luma 142 samples and chroma samples are stored interleaved in a second plane (plane 143 1). For fully planar YUV formats, the first plane (plane 0) continues to 144 hold the luma samples however the chroma samples are stored seperately in 145 two additional planes (plane 1 & plane 2). If present, planes 1 & 2 are 146 specified by the following attributes, which have the same meanings as 147 defined above for plane 0: 148 149 * EGL_DMA_BUF_PLANE1_FD_EXT 150 * EGL_DMA_BUF_PLANE1_OFFSET_EXT 151 * EGL_DMA_BUF_PLANE1_PITCH_EXT 152 * EGL_DMA_BUF_PLANE2_FD_EXT 153 * EGL_DMA_BUF_PLANE2_OFFSET_EXT 154 * EGL_DMA_BUF_PLANE2_PITCH_EXT 155 156 The ordering of samples within a plane is taken from the drm_fourcc 157 pixel_format specified for EGL_LINUX_DRM_FOURCC_EXT. For example, if 158 EGL_LINUX_DRM_FOURCC_EXT is set to DRM_FORMAT_NV12, the chroma plane 159 specified by EGL_DMA_BUF_PLANE1* contains samples in the order V, U, 160 whereas if EGL_LINUX_DRM_FOURCC_EXT is DRM_FORMAT_NV21, the order is U, 161 V. Similarly, the ordering of planes for fully-planar formats is also taken 162 from the pixel_format specified as EGL_LINUX_DRM_FOURCC_EXT. For example, 163 if EGL_LINUX_DRM_FOURCC_EXT is set to DRM_FORMAT_YUV410, the luma plane is 164 specified by EGL_DMA_BUF_PLANE0*, the plane containing U-samples is 165 specified by EGL_DMA_BUF_PLANE1* and the plane containing the V-samples is 166 specified by EGL_DMA_BUF_PLANE2*, whereas if EGL_LINUX_DRM_FOURCC_EXT is 167 set to DRM_FORMAT_YVU410, plane 1 contains the V-samples and plane 2 168 contains the U-samples. 169 170 In addition to the above required attributes, the application may also 171 provide hints as to how the data should be interpreted by the GL. If any of 172 these hints are not specified, the GL will guess based on the pixel format 173 passed as the EGL_LINUX_DRM_FOURCC_EXT attribute or may fall-back to some 174 default value. Not all GLs will be able to support all combinations of 175 these hints and are free to use whatever settings they choose to achieve 176 the closest possible match. 177 178 * EGL_YUV_COLOR_SPACE_HINT_EXT: The color-space the data is in. Only 179 relevant for images in a YUV format, ignored when specified for an 180 image in an RGB format. Accepted values are: 181 EGL_ITU_REC601_EXT, EGL_ITU_REC709_EXT & EGL_ITU_REC2020_EXT. 182 183 * EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT & 184 EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT: Where chroma samples are 185 sited relative to luma samples when the image is in a sub-sampled 186 format. When the image is not using chroma sub-sampling, the luma and 187 chroma samples are assumed to be co-sited. Siting is split into the 188 vertical and horizontal and is in a fixed range. A siting of zero 189 means the first luma sample is taken from the same position in that 190 dimension as the chroma sample. This is best illustrated in the 191 diagram below: 192 193 (0.5, 0.5) (0.0, 0.5) (0.0, 0.0) 194 + + + + + + + + * + * + 195 x x x x 196 + + + + + + + + + + + + 197 198 + + + + + + + + * + * + 199 x x x x 200 + + + + + + + + + + + + 201 202 Luma samples (+), Chroma samples (x) Chrome & Luma samples (*) 203 204 Note this attribute is ignored for RGB images and non sub-sampled 205 YUV images. Accepted values are: EGL_YUV_CHROMA_SITING_0_EXT (0.0) 206 & EGL_YUV_CHROMA_SITING_0_5_EXT (0.5) 207 208 * EGL_SAMPLE_RANGE_HINT_EXT: The numerical range of samples. Only 209 relevant for images in a YUV format, ignored when specified for 210 images in an RGB format. Accepted values are: EGL_YUV_FULL_RANGE_EXT 211 (0-256) & EGL_YUV_NARROW_RANGE_EXT (16-235). 212 213 214 If eglCreateImageKHR is successful for a EGL_LINUX_DMA_BUF_EXT target, the 215 EGL will take a reference to the dma_buf(s) which it will release at any 216 time while the EGLDisplay is initialized. It is the responsibility of the 217 application to close the dma_buf file descriptors." 218 219 220 Add to the list of error conditions for eglCreateImageKHR: 221 222 "* If <target> is EGL_LINUX_DMA_BUF_EXT and <buffer> is not NULL, the 223 error EGL_BAD_PARAMETER is generated. 224 225 * If <target> is EGL_LINUX_DMA_BUF_EXT, and the list of attributes is 226 incomplete, EGL_BAD_PARAMETER is generated. 227 228 * If <target> is EGL_LINUX_DMA_BUF_EXT, and the EGL_LINUX_DRM_FOURCC_EXT 229 attribute is set to a format not supported by the EGL, EGL_BAD_MATCH 230 is generated. 231 232 * If <target> is EGL_LINUX_DMA_BUF_EXT, and the EGL_LINUX_DRM_FOURCC_EXT 233 attribute indicates a single-plane format, EGL_BAD_ATTRIBUTE is 234 generated if any of the EGL_DMA_BUF_PLANE1_* or EGL_DMA_BUF_PLANE2_* 235 attributes are specified. 236 237 * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for 238 EGL_YUV_COLOR_SPACE_HINT_EXT is not EGL_ITU_REC601_EXT, 239 EGL_ITU_REC709_EXT or EGL_ITU_REC2020_EXT, EGL_BAD_ATTRIBUTE is 240 generated. 241 242 * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for 243 EGL_SAMPLE_RANGE_HINT_EXT is not EGL_YUV_FULL_RANGE_EXT or 244 EGL_YUV_NARROW_RANGE_EXT, EGL_BAD_ATTRIBUTE is generated. 245 246 * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for 247 EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT or 248 EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT is not 249 EGL_YUV_CHROMA_SITING_0_EXT or EGL_YUV_CHROMA_SITING_0_5_EXT, 250 EGL_BAD_ATTRIBUTE is generated. 251 252 * If <target> is EGL_LINUX_DMA_BUF_EXT and one or more of the values 253 specified for a plane's pitch or offset isn't supported by EGL, 254 EGL_BAD_ACCESS is generated. 255 256 257Issues 258 259 1. Should this be a KHR or EXT extension? 260 261 ANSWER: EXT. Khronos EGL working group not keen on this extension as it is 262 seen as contradicting the EGLStream direction the specification is going in. 263 The working group recommends creating additional specs to allow an EGLStream 264 producer/consumer connected to v4l2/DRM or any other Linux interface. 265 266 2. Should this be a generic any platform extension, or a Linux-only 267 extension which explicitly states the handles are dma_buf fds? 268 269 ANSWER: There's currently no intention to port this extension to any OS not 270 based on the Linux kernel. Consequently, this spec can be explicitly written 271 against Linux and the dma_buf API. 272 273 3. Does ownership of the file descriptor pass to the EGL library? 274 275 ANSWER: No, EGL does not take ownership of the file descriptors. It is the 276 responsibility of the application to close the file descriptors on success 277 and failure. 278 279 4. How are the different YUV color spaces handled (BT.709/BT.601)? 280 281 ANSWER: The pixel formats defined in drm_fourcc.h only specify how the data 282 is laid out in memory. It does not define how that data should be 283 interpreted. Added a new EGL_YUV_COLOR_SPACE_HINT_EXT attribute to allow the 284 application to specify which color space the data is in to allow the GL to 285 choose an appropriate set of co-efficients if it needs to convert that data 286 to RGB for example. 287 288 5. What chroma-siting is used for sub-sampled YUV formats? 289 290 ANSWER: The chroma siting is not specified by either the v4l2 or DRM APIs. 291 This is similar to the color-space issue (4) in that the chroma siting 292 doesn't affect how the data is stored in memory. However, the GL will need 293 to know the siting in order to filter the image correctly. While the visual 294 impact of getting the siting wrong is minor, provision should be made to 295 allow an application to specify the siting if desired. Added additional 296 EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT & 297 EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT attributes to allow the siting to 298 be specified using a set of pre-defined values (0 or 0.5). 299 300 6. How can an application query which formats the EGL implementation 301 supports? 302 303 PROPOSAL: Don't provide a query mechanism but instead add an error condition 304 that EGL_BAD_MATCH is raised if the EGL implementation doesn't support that 305 particular format. 306 307 7. Which image formats should be supported and how is format specified? 308 309 Seem to be two options 1) specify a new enum in this specification and 310 enumerate all possible formats. 2) Use an existing enum already in Linux, 311 either v4l2_mbus_pixelcode and/or those formats listed in drm_fourcc.h? 312 313 ANSWER: Go for option 2) and just use values defined in drm_fourcc.h. 314 315 8. How can AYUV images be handled? 316 317 ANSWER: At least on fourcc.org and in drm_fourcc.h, there only seems to be 318 a single AYUV format and that is a packed format, so everything, including 319 the alpha component would be in the first plane. 320 321 9. How can you import interlaced images? 322 323 ANSWER: Interlaced frames are usually stored with the top & bottom fields 324 interleaved in a single buffer. As the fields would need to be displayed as 325 at different times, the application would create two EGLImages from the same 326 buffer, one for the top field and another for the bottom. Both EGLImages 327 would set the pitch to 2x the buffer width and the second EGLImage would use 328 a suitable offset to indicate it started on the second line of the buffer. 329 This should work regardless of whether the data is packed in a single plane, 330 semi-planar or multi-planar. 331 332 If each interlaced field is stored in a separate buffer then it should be 333 trivial to create two EGLImages, one for each field's buffer. 334 335 10. How are semi-planar/planar formats handled that have a different 336 width/height for Y' and CbCr such as YUV420? 337 338 ANSWER: The spec says EGL_WIDTH & EGL_HEIGHT specify the *logical* width and 339 height of the buffer in pixels. For pixel formats with sub-sampled Chroma 340 values, it should be trivial for the EGL implementation to calculate the 341 width/height of the Chroma sample buffers using the logical width & height 342 and by inspecting the pixel format passed as the EGL_LINUX_DRM_FOURCC_EXT 343 attribute. I.e. If the pixel format says it's YUV420, the Chroma buffer's 344 width = EGL_WIDTH/2 & height =EGL_HEIGHT/2. 345 346 11. How are Bayer formats handled? 347 348 ANSWER: As of Linux 2.6.34, drm_fourcc.h does not include any Bayer formats. 349 However, future kernel versions may add such formats in which case they 350 would be handled in the same way as any other format. 351 352 12. Should the spec support buffers which have samples in a "narrow range"? 353 354 Content sampled from older analogue sources typically don't use the full 355 (0-256) range of the data type storing the sample and instead use a narrow 356 (16-235) range to allow some headroom & toeroom in the signals to avoid 357 clipping signals which overshoot slightly during processing. This is 358 sometimes known as signals using "studio swing". 359 360 ANSWER: Add a new attribute to define if the samples use a narrow 16-235 361 range or the full 0-256 range. 362 363 13. Specifying the color space and range seems cumbersome, why not just 364 allow the application to specify the full YUV->RGB color conversion matrix? 365 366 ANSWER: Some hardware may not be able to use an arbitrary conversion matrix 367 and needs to select an appropriate pre-defined matrix based on the color 368 space and the sample range. 369 370 14. How do you handle EGL implementations which have restrictions on pitch 371 and/or offset? 372 373 ANSWER: Buffers being imported using dma_buf pretty much have to be 374 allocated by a kernel-space driver. As such, it is expected that a system 375 integrator would make sure all devices which allocate buffers suitable for 376 exporting make sure they use a pitch supported by all possible importers. 377 However, it is still possible eglCreateImageKHR can fail due to an 378 unsupported pitch. Added a new error to the list indicating this. 379 380 15. Should this specification also describe how to export an existing 381 EGLImage as a dma_buf file descriptor? 382 383 ANSWER: No. Importing and exporting buffers are two separate operations and 384 importing an existing dma_buf fd into an EGLImage is useful functionality in 385 itself. Agree that exporting an EGLImage as a dma_buf fd is useful, E.g. it 386 could be used by an OpenMAX IL implementation's OMX_UseEGLImage function to 387 give access to the buffer backing an EGLImage to video hardware. However, 388 exporting can be split into a separate extension specification. 389 390 391Revision History 392 393#7 (Kristian H. Kristensen, December 13, 2017) 394 - Clarify plane ordering to match Linux FOURCC conventions (Bug 16017). 395 396#6 (David Garbett, December 05, 2013) 397 - Application now retains ownership of dma_buf file descriptors. 398 399#5 (Tom Cooksey, February 19, 2013) 400 - Assigned enum values 401 - Moved out of drafts 402 403#4 (Tom Cooksey, October 04, 2012) 404 - Fixed issue numbering! 405 - Added issues 8 - 15. 406 - Promoted proposal for Issue 3 to be the answer. 407 - Added an additional attribute to allow an application to specify the color 408 space as a hint which should address issue 4. 409 - Added an additional attribute to allow an application to specify the chroma 410 siting as a hint which should address issue 5. 411 - Added an additional attribute to allow an application to specify the sample 412 range as a hint which should address the new issue 12. 413 - Added language to end of error section clarifying who owns the fd passed 414 to eglCreateImageKHR if an error is generated. 415 416#3 (Tom Cooksey, August 16, 2012) 417 - Changed name from EGL_EXT_image_external and re-written language to 418 explicitly state this for use with Linux & dma_buf. 419 - Added a list of issues, including some still open ones. 420 421#2 (Jesse Barker, May 30, 2012) 422 - Revision to split eglCreateImageKHR functionality from export 423 Functionality. 424 - Update definition of EGLNativeBufferType to be a struct containing a list 425 of handles to support multi-buffer/multi-planar formats. 426 427#1 (Jesse Barker, March 20, 2012) 428 - Initial draft. 429