1Name 2 3 NV_vdpau_interop 4 5Name Strings 6 7 GL_NV_vdpau_interop 8 9Contributors 10 11 Stephen Warren, NVIDIA 12 James Jones, NVIDIA 13 14Contact 15 16 Stephen Warren, NVIDIA (swarren 'at' nvidia.com) 17 18Status 19 20 Complete. 21 22Version 23 24 11 (2 Jan 2014) 25 26Number 27 28 396 29 30Dependencies 31 32 This extension is written against the OpenGL 3.2 Specification 33 but can apply to OpenGL 1.1 and up. 34 35 OpenGL 1.1 is required. 36 GL_EXT_framebuffer_object affects the definition of this extension. 37 GL_ARB_texture_rectangle affects the definition of this extension. 38 GL_ARB_texture_non_power_of_two affects the definition of this 39 extension. 40 41Overview 42 43 This extension allows VDPAU video and output surfaces to be used 44 for texturing and rendering. 45 46 This allows the GL to process and display the content of video 47 streams decoded using VDPAU. 48 49 Alternatively, the GL may modify VDPAU surfaces in-place, and VDPAU 50 may then process and/or display those surfaces itself. 51 52 This allows the GL to be used to combine application user-interface 53 elements with decoded video, implement custom video-processing 54 algorithms, etc. 55 56IP Status 57 58 There are no known IP issues. 59 60New Types 61 62 typedef GLintptr GLvdpauSurfaceNV; 63 64New Procedures and Functions 65 66 void VDPAUInitNV (const void *vdpDevice, 67 const void *getProcAddress); 68 69 void VDPAUFiniNV (void); 70 71 vdpauSurfaceNV VDPAURegisterVideoSurfaceNV (const void *vdpSurface, 72 enum target, 73 sizei numTextureNames, 74 const uint *textureNames); 75 76 vdpauSurfaceNV VDPAURegisterOutputSurfaceNV (const void *vdpSurface, 77 enum target, 78 sizei numTextureNames, 79 const uint *textureNames); 80 81 boolean VDPAUIsSurfaceNV (vdpauSurfaceNV surface); 82 83 void VDPAUUnregisterSurfaceNV (vdpauSurfaceNV surface); 84 85 void VDPAUGetSurfaceivNV (vdpauSurfaceNV surface, 86 enum pname, 87 sizei bufSize, 88 sizei *length, 89 int *values); 90 91 void VDPAUSurfaceAccessNV (vdpauSurfaceNV surface, 92 enum access); 93 94 void VDPAUMapSurfacesNV (sizei numSurfaces, 95 const vdpauSurfaceNV *surfaces); 96 97 void VDPAUUnmapSurfacesNV (sizei numSurface, 98 const vdpauSurfaceNV *surfaces); 99 100New Tokens 101 102 Accepted as the <pname> parameter of VDPAUGetSurfaceivNV: 103 104 SURFACE_STATE_NV 0x86EB 105 106 Returned in <values> for VDPAUGetSurfaceivNV <pname> 107 SURFACE_STATE_NV: 108 109 SURFACE_REGISTERED_NV 0x86FD 110 SURFACE_MAPPED_NV 0x8700 111 112 Accepted as the <pname> parameter of VDPAUSurfaceAccessNV: 113 114 WRITE_DISCARD_NV 0x88BE 115 116Additions to Chapter 2 of the OpenGL 3.2 (unabridged) Specification 117(OpenGL Operation) 118 119 Add to Table 2.2, GL data types: 120 121 GL Type Minimum Description 122 Bit Width 123 ------- --------- ---------------------------------------------- 124 vdpauSurfaceNV <ptrbits> VDPAU surface object handle (see section 3.8.3) 125 126Additions to Chapter 3 of the OpenGL 3.2 (unabridged) Specification 127(Rasterization) 128 129 Insert a new section following "Alternate Texture Image 130 Specification Commands" (Section 3.8.2). Renumber existing section 131 3.8.3 "Compressed Texture Images" and all following 3.8.* sections. 132 133 3.8.3, Alternate Texture Image Specification Using VDPAU Surfaces 134 ----------------------------------------------------------------- 135 136 Texture images may also be specified using image data taken 137 directly from the framebuffer, in the form of VDPAU surfaces. 138 139 VDPAU surfaces are logically ascribed an interop state. States 140 and transitions are: 141 142 VDPAUUnegisterSurfaceNV/VDPAUFiniNV 143 /----------------------------------------\ 144 | | 145 v VDPAURegister*SurfaceNV | 146 Unknown/unregistered ----------------------> Registered 147 ^ | ^ 148 | VDPAUMapSurfacesNV | | VDPAUUnmapSurfacesNV 149 | v | 150 \------------------------------------- Mapped 151 VDPAUFiniNV 152 153 In order to use such "interop" functionality, the command 154 155 void VDPAUInitNV (const void *vdpDevice, 156 const void *getProcAddress); 157 158 must be used to inform the GL which VDPAU device to interact with. 159 <vdpDevice> must be an extant VdpDevice handle previously created 160 using VDPAU. <getProcAddress> must be the VdpGetProcAddress 161 generated during VdpDevice creation. 162 163 When interop functionality is no longer required, the GL may be 164 informed using the command 165 166 void VDPAUFiniNV (void); 167 168 This operation will succeed even if surfaces exist in the mapped 169 or registered states; such surfaces will internally be unmapped and 170 unregistered before interop state is destroyed. 171 172 The command 173 174 vdpauSurfaceNV VDPAURegisterVideoSurfaceNV (const void *vdpSurface, 175 enum target, 176 sizei numTextureNames, 177 const uint *textureNames); 178 179 defines a set of two-dimensional textures, where the image data may 180 be taken from the VdpVideoSurface <vdpSurface>. <target> must be 181 one of TEXTURE_2D or TEXTURE_RECTANGLE. <numTextureNames> 182 determines how many textures are defined. <textureNames> contains 183 the names of the textures that are defined. The surface is 184 transitioned into the registered state. 185 186 Legal values for <numTextureNames> are derived from the 187 VdpChromaType of <vdpSurface>, as defined in table 3.8.3.1. 188 189 Internal 190 VdpChromaType numTextureNames Index Size Format Content 191 ------------- --------------- ----- ---- -------- ------------------- 192 VDP_CHROMA_TYPE_420 4 0 w x h/2 R8 Top-field luma 193 1 w x h/2 R8 Bottom-field luma 194 2 w/2 x h/4 R8G8 Top-field chroma 195 3 w/2 x h/4 R8G8 Bottom-field chroma 196 VDP_CHROMA_TYPE_422 4 0 w x h/2 R8 Top-field luma 197 1 w x h/2 R8 Bottom-field luma 198 2 w/2 x h/2 R8G8 Top-field chroma 199 3 w/2 x h/2 R8G8 Bottom-field chroma 200 201 Table 3.8.3.1: Supported VdpChromaType values, and derived values 202 of <numTextureNames>, and texture parameters for each texture. 203 204 VDPAURegisterVideoSurfaceNV's return value is a handle used by 205 various other commands detailed below. 206 207 The command 208 209 vdpauSurfaceNV VDPAURegisterOutputSurfaceNV (const void *vdpSurface, 210 enum target, 211 sizei numTextureNames, 212 const uint *textureNames); 213 214 defines a set of two-dimensional textures, where the image data may 215 be taken from the VdpOutputSurface vdpSurface. <target> must be one 216 of TEXTURE_2D or TEXTURE_RECTANGLE. <numTextureNames> determines 217 how many textures are defined. <textureNames> contains the names of 218 the textures that are defined. The surface is transitioned into the 219 registered state. 220 221 Legal values for <numTextureNames> are derived from the 222 VdpRGBAFormat of <vdpSurface>, as defined in table 3.8.3.2. 223 224 Internal 225 VdpRGBAFormat numTextureNames Index Size Format Content 226 ------------- --------------- ----- ---- -------- -------------- 227 VDP_RGBA_FORMAT_B8G8R8A8 1 0 w x h ARGB8 Entire surface 228 VDP_RGBA_FORMAT_R10G10B10A2 1 0 w x h A2BGR10 Entire surface 229 230 Table 3.8.3.2: Supported VdpRGBAFormat values, and derived values 231 of <numTextureNames>, and texture parameters for each texture. 232 233 In all cases, textures defined by VDPAU surfaces will be y- 234 inverted; applications are expected to use y-inverted texture co- 235 ordinates when using such textures. 236 237 VDPAURegisterOutputSurfaceNV's return value is a handle to be used 238 as the surface parameter to various other commands detailed below. 239 240 The command 241 242 void VDPAUUnregisterSurfaceNV (GLvdpauSurfaceNV surface); 243 244 may be used to revert the effect of VDPAURegisterVideoSurfaceNV or 245 VDPAURegisterOutputSurfaceNV; namely, any <textureName>s that were 246 passed into the register function will be returned to their default 247 state. The surface will be transitioned into the unregistered state. 248 This operation will succeed even if the surface is in the mapped 249 state; such surfaces will internally be unmapped before being 250 unregistered. 251 252 The command 253 254 void VDPAUSurfaceAccessNV (GLvdpauSurfaceNV surface, 255 enum access); 256 257 informs the GL whether the application expects only to read from the 258 VDPAU surface using the GL (<access> set to READ_ONLY), only to 259 write to the VDPAU surface using the GL (<access> set to 260 WRITE_DISCARD_NV), or both (<access> set to READ_WRITE). In 261 implementations that need to copy image data during 262 VDPAUMapSurfacesNV or VDPAUUnmapSurfacesNV, this flag may allow the 263 GL to skip some of those copy operations to improve performance. 264 265 An <access> value of READ_ONLY ensures that any writes to the 266 surface made by VDPAU prior to VDPAUMapSurfacesNV will be seen by 267 the GL when reading the surface after VDPAUMapSurfacesNV. If the 268 GL writes to the surface, results are undefined, possibly including 269 program termination. 270 271 An <access> value of READ_WRITE ensures that any writes to the 272 surface made by VDPAU prior to VDPAUMapSurfacesNV will be seen by 273 the GL when reading the surface after VDPAUMapSurfacesNV. Equally, 274 any writes to the surface made by the GL prior to 275 VDPAUUnmapSurfacesNV will be seen by VDPAU after 276 VDPAUUnmapSurfacesNV. 277 278 An <access> value of WRITE_DISCARD_NV ensures that any writes to the 279 surface made by the GL prior to VDPAUUnmapSurfacesNV will be seen 280 by VDPAU after VDPAUUnmapSurfacesNV. If the GL reads from the 281 surface prior to writing those particular pixels itself, the 282 graphical results are undefined, but should not include program 283 termination. 284 285 VDPAUSurfaceAccessNV sets the access flag for the VDPAU surface. 286 the GL reads the access flag only during VDPAUMapSurfacesNV. 287 VDPAUUnmapSurfacesNV uses the value of the access flag that was 288 set when the previous VDPAUMapSurfacesNV was called. 289 VDPAUSurfaceAccessNV itself has no effect on the image data of any 290 surface, and in particular does not initiate any copy operations. 291 292 Calling VDPAUSurfaceAccessNV when the surface is in a mapped state 293 is illegal, and will generate an error. 294 295 The command 296 297 void VDPAUMapSurfacesNV (sizei numSurfaces, 298 vdpauSurfaceNV *surfaces); 299 300 moves a set of VDPAU surfaces from the registered state into the 301 mapped state. 302 303 The command 304 305 void VDPAUUnmapSurfacesNV (sizei numSurface, 306 vdpauSurfaceNV* surfaces); 307 308 moves a set of VDPAU surfaces from the mapped state into the 309 registered state. 310 311 Using the GL to texture from, or write to, a surface in any state 312 other than mapped will yield undefined results, potentially 313 including program termination. 314 315 Using any other API to access a VDPAU surface while that surface is 316 in the mapped state in the GL will yield undefined results, 317 potentially including program termination. 318 319 Texture image data for VDPAU surfaces in the mapped state is 320 defined as being identical to the VDPAU surface content. 321 322 While a VDPAU surface is in the mapped or registered states, the 323 textures associated with that surface are immutable. In other 324 words, operations that affect texture state, or attempt to replace 325 the source of the texture image data (e.g. TexImage2D) will return 326 an error. Rendering to the texture is allowed. 327 328Additions to Chapter 4 of the OpenGL 3.2 (unabridged) Specification 329(Per-Fragment Operations and the Frame Buffer) 330 331 None 332 333Additions to Chapter 5 of the OpenGL 3.2 (unabridged) Specification 334(Special Functions) 335 336 None 337 338Additions to Chapter 6 of the OpenGL 3.2 (unabridged) Specification 339(State and State Requests) 340 341 Add a new subsection to "Querying GL state" (section 342 6.1) describing VDPAU surface object queries. 343 344 "6.1.X VDPAU Surface Object Queries 345 346 Properties of VDPAU surface objects may be queried using the command 347 348 void VDPAUGetSurfaceivNV (vdpauSurfaceNV surface, 349 enum pname, 350 sizei bufSize, 351 sizei *length, 352 int *values); 353 354 The value or values being queried are returned in the parameters 355 <length> and <values>. 356 357 On success, VDPAUGetSurfaceivNV replaces up to <bufSize> integers 358 in <values> with the corresponding property values of the object 359 being queried. The actual number of integers replaced is returned 360 in *<length>. If <length> is NULL, no length is returned. 361 362 If <pname> is SURFACE_STATE_NV, a single value representing the state 363 of the VDPAU surface object (SURFACE_REGISTERED_NV, 364 SURFACE_MAPPED_NV) is placed in <values>. 365 366 If <surface> is not the name of a VDPAU surface object, an 367 INVALID_VALUE error is generated. If <pname> is not one of the 368 values described above, an INVALID_ENUM error is generated. If an 369 error occurs, nothing will be written to <values> or <length>. 370 371 The command 372 373 boolean VDPAUIsSurfaceNV (vdpauSurfaceNV surface); 374 375 returns TRUE if <surface> is the name of a VDPAU surface object. If 376 <surface> is not the name of a VDPAU surface object, or if an error 377 condition occurs, IsSurface returns FALSE (note that zero is not 378 the name of a VDPAU surface object). 379 380Additions to the AGL/GLX/WGL Specifications 381 382 None 383 384Additions to the OpenGL Shading Language 385 386 None 387 388GLX Protocol 389 390 VDPAU implementations currently only support direct-rendering. 391 Consequently, no GLX protocol is currently defined for this 392 extension. 393 394Dependencies on GL_ARB_texture_rectangle 395 396 If GL_ARB_texture_rectangle is not supported, TEXTURE_RECTANGLE may 397 not be used as target for VDPAURegisterVideoSurfaceNV. 398 399Dependencies on GL_ARB_texture_non_power_of_two 400 401 If GL_ARB_texture_non_power_of_two is not supported, only VDPAU 402 surfaces with power-of-two size may be used with target TEXTURE_2D. 403 404Errors 405 406 INVALID_VALUE is generated if the <vdpDevice> or 407 <getProcAddressOpaque> parameters to VDPAUInitNV is NULL. 408 409 INVALID_OPERATION is generated if VDPAUInitNV is called on a given 410 GL context, where VDPAUInitNV has been called, and VDPAUFiniNV has 411 not been called since. 412 413 INVALID_OPERATION is generated by VDPAUInitNV, 414 VDPAURegisterVideoSurfaceNV, or VDPAURegisterOutputSurfaceNV if the 415 VDPAU driver refuses the request for some reason. 416 417 INVALID_OPERATION is generated if VDPAUFiniNV, 418 VDPAURegisterVideoSurfaceNV, VDPAURegisterOutputSurfaceNV, 419 VDPAUIsSurfaceNV, VDPAUGetSurfaceivNV, VDPAUSurfaceAccessNV, 420 VDPAUMapSurfacesNV, or VDPAUUnmapSurfacesNV are called on a given 421 GL context, where VDPAUInitNV has not been called, or VDPAUFiniNV 422 has been called since. 423 424 INVALID_ENUM is generated if the <target> parameter of 425 VDPAURegisterVideoSurfaceNV or VDPAURegisterOutputSurfaceNV is not 426 TEXTURE_2D or TEXTURE_RECTANGLE. 427 428 INVALID_VALUE is generated if the <numTextureNames> parameter of 429 VDPAURegisterVideoSurfaceNV or VDPAURegisterOutputSurfaceNV does 430 not match the required value; see tables 3.8.3.1 and 3.8.3.2. 431 432 INVALID_OPERATION is generated if any texture named by an entry 433 within the <textureNames> parameter of VDPAURegisterVideoSurfaceNV 434 or VDPAURegisterOutputSurfaceNV is marked as immutable. 435 436 INVALID_VALUE is generated if the VDPAU surface named by the 437 <vdpSurface> parameter of VDPAURegisterVideoSurfaceNV or 438 VDPAUUnregisterOutputSurfaceNV does not have a supported format; 439 see tables 3.8.3.1 and 3.8.3.2. 440 441 INVALID_VALUE is generated if the <surface> parameter of 442 VDPAUGetSurfaceivNV, VDPAUSurfaceAccessNV, VDPAUMapSurfacesNV, 443 VDPAUUnmapSurfacesNV is not the name of a VDPAU surface object. 444 445 INVALID_VALUE is generated if the <surface> parameter of 446 VDPAUUnregisterSurfaceNV is neither zero nor the name of a VDPAU 447 surface object. 448 449 INVALID_VALUE is generated if the <bufSize> parameter of 450 VDPAUGetSurfaceivNV is too small to return the results via the 451 <values> paramter. 452 453 INVALID_ENUM is generated if the <pname> parameter of 454 VDPAUGetSurfaceivNV is not SURFACE_STATE_NV. 455 456 INVALID_VALUE is generated if the <access> parameter of 457 VDPAUSurfaceAccessNV is none of READ_ONLY, WRITE_ONLY, nor 458 READ_WRITE. 459 460 INVALID_OPERATION is generated if the state of the surface(s) named 461 by the <surface> parameter of VDPAUSurfaceAccessNV, or named by any 462 entry within the <surfaces> parameter of VDPAUMapSurfacesNV is 463 SURFACE_MAPPED_NV. 464 465 INVALID_OPERATION is generated if the state of any of the surfaces 466 named by entries within the <surfaces> parameter of 467 VDPAUUnmapSurfacesNV is not SURFACE_MAPPED_NV. 468 469New State 470 471Table 6.X. VDPAU Surface Objects. 472 473Get value Type Get command Initial value Description Section 474------------------ ---- ----------- ---------------------------- --------------- ------- 475SURFACE_STATE_NV Z VDPAUGetSurfaceivNV GL_SURFACE_REGISTERED_NV Surface state 3.8.3 476 477New Implementation State 478 479 None 480 481Issues 482 483 1. Should YUV surfaces be exposed as a frame Y and frame UV 484 texture, or two field Y and two field UV textures? 485 486 RESOLVED: YUV surfaces will be exposed as separate fields 487 488 Exposing the surface as separate fields allows applications to 489 directly implement de-interlacing algorithms other than weave 490 without having to undo the weaving of fields together. Put 491 another way: 492 493 Frames: 494 495 Pros: 496 497 * Simple applications can directly access the entire frame, 498 without having to manually implement weave de-interlacing. 499 This also applies for progressive content. 500 501 However, simple applications probably won't want to interop 502 on a VdpVideoSurface, but will rather rely on VDPAU's 503 VdpVideoMixer for post-processing, and instead interop on a 504 VdpOutputSurface. 505 506 Cons: 507 508 * In order to implement bob de-interlacing, vertical 509 interpolation between lines of a single field is required. 510 The GL's texturing does not allow interpolation to skip 511 lines in a texture. Consequently, the two fields must be 512 manually separated by the application using a shader and 513 some texture co-ordinate calculation prior to bobbing. This 514 forces a copy operation, which would reduce performance. 515 516 * In NVIDIA's implementation, VDPAU surfaces are stored as 517 separate fields. Consequently, VDPAUMapSurfacesNV would have 518 to perform a copy operation, which would reduce performance. 519 520 Fields: 521 522 Pros: 523 524 * Advanced de-interlacing and post-processing algorithms get 525 direct access to individual fields. 526 527 * In NVIDIA's implementation, no copying is required during 528 VDPAUMapSurfacesNV. 529 530 Cons: 531 532 * Simple applications that simply want to weave, or for 533 progressive content, must manually combine the fields. 534 However, this operation is relatively simple to code, and 535 sample code will be provided. 536 537 2. Should Map/Unmap functions be removed, such that the GL 538 implements those operations transparently at appropriate times? 539 540 RESOLVED: No, map/unmap functions will be kept. 541 542 The map/unmap functions provide a convenient place to perform 543 synchronization between VDPAU and the GL command streams, and 544 any format-converting copies that may be required by the GL or 545 VDPAU implementation. Without explicit application-controlled 546 APIs, it would potentially be difficult to trigger those 547 operations at the appropriate time. It would also be less 548 obvious to software developers when such operations were 549 occurring, which may prevent successful investigation of 550 interop performance. 551 552 3. Should SurfaceAccess function be removed, and instead an 553 "access" parameter added to the Map function? 554 555 RESOLVED: No, a separate function will be kept. 556 557 The value of this flag typically will not change on a per-frame 558 basis, so specifying it for each map call was deemed redundant. 559 For an advanced application that needs to change the flag often, 560 the overhead of using a separate function should not be too 561 significant. 562 563Revision History 564 565 1. 23 Dec 2009 - Stephen W 566 Initial version 567 568 2. 5 Jan 2010 - Stephen W 569 -Renamed VDPAUSetSurfaceAccessNV to VDPAUSurfaceAccessNV thus 570 avoiding the word Set. 571 -Fixed/wrote "Dependencies", "Dependencies on ..." sections. 572 -Initial version of "Additions to Chapter 3". 573 -Fill in answers for issues 1-3. 574 575 3. 7 Jan 2010 - Stephen W 576 -Dropped the GL prefix from type names. 577 -Refer to "the GL" not "GL". 578 -Removed XXX re: PROXY_TEXTURE_*; those targets don't seem 579 useful for this extension. 580 -Use <> around function parameter names in text. 581 -Reword description of legal values for <numTextureNames>. 582 Number, title, and reformat related tables. 583 Rename "format" column to "internal format". 584 -Describe VDPAUSurfaceAccessNV before Map/Ummap for more 585 logical ordering. 586 -Significant text rewrite to more explicitly describe states, 587 state transitions, legal access to surfaces by various APIs 588 during specific states, etc. 589 -Explicitly state that registered/mapped textures are 590 immutable. 591 -Re-used VBO's READ/WRITE/READ_WRITE enums instead of inventing 592 new enums. 593 -Added explicit type GLvdpauSurfaceNV instead of using GLvoid*. 594 -Added VDPAUIsSurface, VDPAUGetSurfaceiv, and associated 595 tokens. 596 -Documented state, state queries, and errors. 597 598 4. 13 Jan 2010 - Stephen W 599 - Added token WRITE_DISCARD_NV. Stated semantics of each legal 600 value of VDPAUSurfaceAccessNV's <access> parameter. 601 602 5. 13 Jan 2010 - Stephen W 603 - Removed some XXXs. Marked spec as complete. 604 605 6. 18 Jan 2010 - Stephen W 606 - Added missing NV suffix to VDPAUIsSurfaceNV, 607 VDPAUGetSurfaceivNV 608 609 7. 18 Jan 2010 610 - Made various function parameters const. 611 612 8. 24 Feb 2010 613 - Fixed typo in table 3.8.3.1; the heights of luma surfaces 614 were incorrectly specified. 615 616 9. 29 May 2010 617 - Fix minor typos and formatting issues. 618 619 10. 16 Jun 2010 620 - Added "New Types" section 621 622 11. 2 Jan 2014 623 - Fixed return type of VDPAUIsSurfaceNV and remove it from the 624 list of commands that generate INVALID_VALUE. 625 626