1Name 2 3 NV_stream_remote 4 NV_stream_cross_object 5 NV_stream_cross_display 6 NV_stream_cross_process 7 NV_stream_cross_partition 8 NV_stream_cross_system 9 10Name Strings 11 12 EGL_NV_stream_remote 13 14Contributors 15 16 Daniel Kartch 17 Bogdan Naodovic 18 James Jones 19 Zander Clucas 20 21Contacts 22 23 Daniel Kartch, NVIDIA (dkartch 'at' nvidia.com) 24 25Status 26 27 Draft 28 29Version 30 31 Version 4 - June 01, 2016 32 33Number 34 35 EGL Extension #114 36 37Extension Type 38 39 EGL display extension 40 41Dependencies 42 43 Requires EGL_KHR_stream 44 45 Interacts with EGL_KHR_stream_cross_process_fd 46 47 All EGL_NV_stream_cross_* extensions require EGL_NV_stream_remote, 48 but are independent of each other. 49 50Overview 51 52 This extension provides a generic framework for establishing stream 53 connections when the producer and consumer endpoints are not 54 associated with the same EGLStream object. The cross-process streams 55 allowed by the EGL_KHR_stream_cross_process_fd extension are one 56 example of such a connection. Other potential examples include 57 streams between objects associated with different EGLDisplays in a 58 single process, streams between processes residing in separate 59 virtual machine partitions on a single system, or streams between 60 entirely separate systems connected via the internet. 61 62 The approach used by EGL_KHR_stream_cross_process_fd has several 63 drawbacks: 64 - It creates a new pair of stream creation and query functions 65 which are specific to both the type of stream and the method 66 used for communicating the stream's identity. Extending this 67 methodology would require new pairs of functions for every new 68 type of stream connection. 69 - It allows creation of the connected stream objects without any 70 indication of which side will be the consumer and which will be 71 the producer. It even allows, although this was probably not 72 intended, a cross-process stream to be established, but have 73 both producer and consumer exist in the same process, with the 74 other process acting as a voyeur that can observe state and 75 perhaps modify attributes, but not operate on frames. 76 - The use of file descriptors generated by EGL in one process 77 and accessed by EGL in another to establish communication has 78 potential security flaws, and may not be available at all on 79 some real-time operating systems. 80 81 Rather than implement new connection types using this model, we 82 propose a more generic approach that does not require a 83 proliferation of new interfaces and avoids any ambiguities in how 84 the stream will be used. These extensions can coexist with 85 EGL_KHR_stream_cross_process_fd, but allow for more control of 86 stream communication by the applications. 87 88 These extensions provide the framework with which arbitrary stream 89 object pairs can be established, but by themselves are insufficient 90 to create such pairs. In addition, a separate extension must be 91 used to determine the protocol by which the two objects will 92 communicate. See for example the EGL_NV_stream_socket extension. 93 94 Several optional types of separate stream objects are defined, 95 with successive levels of remoteness. It should be assumed that 96 a more remote type will be less efficient in transfering images, 97 and therefore applications should choose the least remote type 98 available that is compatible with their use cases. 99 100New Types 101 102 None 103 104New Functions 105 106 None 107 108New Tokens 109 110 Returned by eglQueryStreamKHR and eglQueryStreamAttribKHR when 111 attribute is EGL_STREAM_STATE_KHR: 112 113 EGL_STREAM_STATE_INITIALIZING_NV 0x3240 114 115 Accepted as attribute names in eglCreateStreamKHR, 116 eglCreateStreamAttribKHR, eglQueryStreamKHR, and 117 eglQueryStreamAttribKHR: 118 119 EGL_STREAM_TYPE_NV 0x3241 120 EGL_STREAM_PROTOCOL_NV 0x3242 121 EGL_STREAM_ENDPOINT_NV 0x3243 122 123 Accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR, and 124 returned by eglQueryStreamKHR and eglQueryStreamAttribKHR, as value 125 when attribute is EGL_STREAM_TYPE_NV, EGL_STREAM_PROTOCOL_NV or 126 EGL_STREAM_ENDPOINT_NV: 127 128 EGL_STREAM_LOCAL_NV 0x3244 129 130 When EGL_NV_stream_cross_object is present, accepted by 131 eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by 132 eglQueryStreamKHR and eglQueryStreamAttribKHR as value for 133 attribute EGL_STREAM_TYPE_NV: 134 135 EGL_STREAM_CROSS_OBJECT_NV 0x334D 136 137 When EGL_NV_stream_cross_display is present, accepted by 138 eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by 139 eglQueryStreamKHR and eglQueryStreamAttribKHR as value for 140 attribute EGL_STREAM_TYPE_NV: 141 142 EGL_STREAM_CROSS_DISPLAY_NV 0x334E 143 144 When EGL_NV_stream_cross_process is present, accepted as 145 value for attribute EGL_STREAM_TYPE_NV by eglCreateStreamKHR 146 and eglCreateStreamAttribKHR. When either 147 EGL_NV_stream_cross_process or EGL_KHR_stream_cross_process_fd 148 is present, returned by eglQueryStreamKHR and 149 eglQueryStreamAttribKHR for attribute EGL_STREAM_TYPE_NV: 150 151 EGL_STREAM_CROSS_PROCESS_NV 0x3245 152 153 When EGL_NV_stream_cross_partition is present, accepted by 154 eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by 155 eglQueryStreamKHR and eglQueryStreamAttribKHR as value for 156 attribute EGL_STREAM_TYPE_NV: 157 158 EGL_STREAM_CROSS_PARTITION_NV 0x323F 159 160 When EGL_NV_stream_cross_system is present, accepted by 161 eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by 162 eglQueryStreamKHR and eglQueryStreamAttribKHR as value for 163 attribute EGL_STREAM_TYPE_NV: 164 165 EGL_STREAM_CROSS_SYSTEM_NV 0x334F 166 167 If EGL_KHR_stream_cross_process_fd is present, returned by 168 eglQueryStreamKHR and eglQueryStreamAttribKHR when attribute is 169 EGL_STREAM_PROTOCOL_NV: 170 171 EGL_STREAM_PROTOCOL_FD_NV 0x3246 172 173 Accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR, and 174 returned by eglQueryStreamKHR and eglQueryStreamAttribKHR when 175 attribute is EGL_STREAM_ENDPOINT_NV: 176 177 EGL_STREAM_PRODUCER_NV 0x3247 178 EGL_STREAM_CONSUMER_NV 0x3248 179 180Add to "Glossary" of EGL_KHR_stream: 181 182 Local stream 183 A stream implemented with both consumer and producer attached to a 184 single EGLStream object. 185 186 Remote stream 187 A stream implemented with two EGLStream objects that communicate 188 with each other, with the consumer attached to one object and the 189 producer attached to the other. 190 191Add to section "3.10.1 Creating an EGLStream" in EGL_KHR_stream: 192 193 If a remote EGLStream is created, as described in section "3.10.5.x 194 Remote streams", and is paired with an EGLStream object which has 195 conflicting values for some attributes, creation will succeed, but 196 the stream will enter state EGL_STREAM_STATE_DISCONNECTED_KHR once 197 communication is established. 198 199Add to list of failures in section "3.10.1 Creating an EGLStream" in 200EGL_KHR stream: 201 202 - EGL_BAD_MATCH is generated if the value of any one of the 203 EGL_STREAM_TYPE_NV, EGL_STREAM_PROTOCOL_NV, or 204 EGL_STREAM_ENDPOINT_NV attributes is EGL_STREAM_LOCAL_NV and 205 any other of these attributes is neither EGL_STREAM_LOCAL_NV 206 nor EGL_DONT_CARE. 207 208Add to section "3.10.2 Connecting an EGLStream to a consumer" in 209EGL_KHR_stream: 210 211 Any function which connects a consumer to an EGLStream will fail and 212 generate an EGL_BAD_ACCESS error if the value of the EGLStream's 213 EGL_STREAM_ENDPOINT_NV attribute is EGL_STREAM_PRODUCER_NV. 214 215Add to section "3.10.3 Connecting an EGLStream to a producer" in 216EGL_KHR_stream: 217 218 Any function which connects a producer to an EGLStream will fail and 219 generate an EGL_BAD_ACCESS error if the value of the EGLStream's 220 EGL_STREAM_ENDPOINT_NV attribute is EGL_STREAM_CONSUMER_NV. 221 222Add to "Table 3.10.4.4 EGLStream Attributes" in EGL_KHR_stream: 223 224 Attribute Read/Write Type Section 225 -------------------------- ---------- ------ ---------- 226 EGL_STREAM_TYPE_NV io EGLint 3.10.4.x 227 EGL_STREAM_PROTOCOL_NV io EGLint 3.10.4.x+1 228 EGL_STREAM_ENDPOINT_NV io EGLint 3.10.4.x+2 229 230Add to beginning of state list in section "3.10.4.3 EGL_STREAM_STATE_KHR 231Attribute" in EGL_KHR_stream: 232 233 - EGL_STREAM_STATE_INITIALIZING_NV - The EGLStream object 234 represents one endpoint of a remote stream and has not yet 235 established communication with the opposite endpoint. 236 237Replace the first entry in the state transition table in section 238"3.10.4.3 EGL_STREAM_STATE_KHR Attribute" in EGL_KHR_stream: 239 240 -> EGL_STREAM_STATE_INITIALIZING_NV 241 A new remote EGLStream is created in this state. 242 243 EGL_STREAM_STATE_INITIALIZING_NV -> 244 EGL_STREAM_STATE_CREATED_KHR 245 Occurs when a remote EGLStream object establishes communication with 246 the opposite endpoint. 247 248 -> EGL_STREAM_STATE_CREATED_KHR 249 A new local EGLStream or EGLStream of undetermined type is created 250 in this state. 251 252Add new subsections to the end of section "3.10.4 EGLStream Attributes" 253in EGL_KHR_stream: 254 255 3.10.4.x EGL_STREAM_TYPE_NV Attribute 256 257 The EGL_STREAM_TYPE_NV attribute may be set when the stream 258 is created, and indicates the type of connection between the 259 EGLStream objects to which the consumer and producer are attached. 260 Legal values are EGL_DONT_CARE and EGL_STREAM_LOCAL_NV. The 261 default value is EGL_DONT_CARE. 262 263 A value of EGL_STREAM_LOCAL_NV indicates that this is a local 264 stream with both consumer and producer connected to a single 265 EGLStream object. 266 267 If EGL_DONT_CARE is initially specified, the value of the attribute 268 will automatically be changed to the appropriate value once both 269 consumer and producer are attached, depending on the functions used. 270 271 Other possible values for EGL_STREAM_TYPE_NV may be defined by 272 additional extensions to indicate a remote stream. Where used, 273 the EGL_STREAM_PROTOCOL_NV attribute must also be set to a 274 communication protocol compatible with the stream type. 275 276 3.10.4.x+1 EGL_STREAM_PROTOCOL_NV Attribute 277 278 The EGL_STREAM_PROTOCOL_NV attribute may be set when the stream is 279 created, and indicates the manner in which communication is 280 established between the EGLStream objects to which the consumer 281 and producer are attached. Legal values are EGL_DONT_CARE and 282 EGL_STREAM_LOCAL_NV. The default value is EGL_DONT_CARE. 283 284 A value of EGL_STREAM_LOCAL_NV indicates that this is a local 285 stream with both consumer and producer connected to a single 286 EGLStream object, so no communication protocol is required. 287 288 If EGL_DONT_CARE is initially specified, the value of the attribute 289 will automatically be changed to the appropriate value once both 290 consumer and producer are attached, depending on the functions used. 291 292 Other possible values for EGL_STREAM_PROTOCOL_NV may be defined by 293 additional extensions to indicate the communication protocol to be 294 used for a remote stream. Not all communication protocols are 295 compatible with all stream types, and vice versa. If incompatible 296 types and protocols are specified, an EGL_BAD_MATCH error will be 297 generated. 298 299 3.10.4.x+2 EGL_STREAM_ENDPOINT_NV Attribute 300 301 The EGL_STREAM_ENDPOINT_NV attribute may be set when the stream is 302 created, and indicates the endpoints which will be attached to the 303 EGLStream object. Legal values are EGL_DONT_CARE, 304 EGL_STREAM_LOCAL_NV, EGL_STREAM_CONSUMER_NV, and 305 EGL_STREAM_PRODUCER_NV. The default value is EGL_DONT_CARE. 306 307 A value of EGL_STREAM_LOCAL_NV indicates that this is a local 308 stream with both consumer and producer connected to a single 309 EGLStream object. 310 311 A value of EGL_STREAM_CONSUMER_NV indicates that the EGLStream 312 object represents the consumer side of a remote stream. 313 314 A value of EGL_STREAM_PRODUCER_NV indicates that the EGLStream 315 object represents the producer side of a remote stream. 316 317 If EGL_DONT_CARE is initially specified, the value of the attribute 318 will automatically be changed to the appropriate value once both 319 consumer and producer are attached, depending on the functions used. 320 321If EGL_NV_stream_cross_object is present, in section 3.10.4.x above, 322add EGL_STREAM_CROSS_OBJECT_NV to the list of legal values, and insert 323 324 A value of EGL_STREAM_CROSS_OBJECT_NV indicates that the stream 325 object represents one endpoint of a remote stream whose other 326 endpoint is obtained from the same EGLDisplay. 327 328If EGL_NV_stream_cross_display is present, in section 3.10.4.x above, 329add EGL_STREAM_CROSS_DISPLAY_NV to the list of legal values, and insert 330 331 A value of EGL_STREAM_CROSS_DISPLAY_NV indicates that the stream 332 object represents one endpoint of a remote stream whose other 333 endpoint may be obtained from a different EGLDisplay in the same 334 process. 335 336If EGL_NV_stream_cross_process or EGL_NV_stream_cross_process_fd is 337present, in section "3.10.4.x" above, add EGL_STREAM_CROSS_PROCESS_NV 338to the list of legal values, and insert 339 340 A value of EGL_STREAM_CROSS_PROCESS_NV indicates that the stream 341 object represents one endpoint of a remote stream whose other 342 endpoint may reside in a separate process. 343 344If EGL_NV_stream_cross_partition is present, in section 3.10.4.x 345above, add EGL_STREAM_CROSS_PARTITION_NV to the list of legal values, 346and insert 347 348 A value of EGL_STREAM_CROSS_PARTITION_NV indicates that the stream 349 object represents one endpoint of a remote stream whose other 350 endpoint may reside in a separate virtual machine partition on 351 the same system. The partitions are not required to be using the 352 same operating systems, but must support compatible communication 353 protocols. 354 355If EGL_NV_stream_cross_system is present, in section 3.10.4.x above, 356add EGL_STREAM_CROSS_SYSTEM_NV to the list of legal values, and insert 357 358 A value of EGL_STREAM_CROSS_SYSTEM_NV indicates that the stream 359 object represents one endpoint of a remote stream whose other 360 endpoint may reside on an independent hardware system with no 361 directly sharable memory resources. 362 363If EGL_KHR_stream_cross_process_fd is present, in section 3.10.4.x+1 364above, add EGL_STREAM_PROTOCOL_FD_NV to the list of legal values, and 365insert 366 367 A value of EGL_STREAM_PROTOCOL_FD_NV indicates that the stream is 368 a remote stream whose communication is established using a file 369 descriptor. The details of what this file descriptor represents 370 are implementation dependent. If the EGL_STREAM_PROTOCOL_NV 371 attribute is initialized with this value, the EGL_STREAM_TYPE_NV 372 attribute must specify a value of EGL_STREAM_CROSS_PROCESS_NV or 373 an EGL_BAD_MATCH failure is generated. 374 375 If an initial value of EGL_DONT_CARE is specified and a file 376 descriptor is subsequently obtained with 377 eglGetStreamFileDescriptorKHR, the value will be automatically 378 changed to EGL_STREAM_PROTOCOL_FD_NV. 379 380Add a new subsection to the end of "3.10.5 EGLStream operation": 381 382 3.10.5.x Remote streams 383 384 An EGLStream object may be created as the endpoint of a remote 385 stream by specifying EGL_STREAM_PRODUCER_NV or 386 EGL_STREAM_CONSUMER_NV as the value for its EGL_STREAM_ENDPOINT_NV 387 attribute. Valid and compatible EGL_STREAM_TYPE_NV and 388 EGL_STREAM_PROTOCOL_NV values other than EGL_DONT_CARE or 389 EGL_STREAM_LOCAL_NV must also be specified. 390 391 If a value for EGL_STREAM_ENDPOINT_NV is not specified, the stream 392 is assumed to be local, but other extensions (see for example 393 EGL_KHR_stream_cross_process_fd) may allow it to be converted to a 394 remote stream before the producer has been attached. 395 396 When an EGLStream object is created as remote, any unspecified 397 attribute will be initially set to a value of EGL_DONT_CARE. Pairs 398 of EGLStream objects representing opposite endpoints of a stream are 399 not required to specify the same attribute lists, but their 400 attributes must be compatible. When communication is established 401 between the endpoints, they will exchange attribute settings: 402 - If both endpoints have a value of EGL_DONT_CARE for an 403 attribute, the default value will be assigned. 404 - If one endpoint has a value of EGL_DONT_CARE for an attribute, 405 it will be set to the other endpoint's value. 406 - If neither endpoint has a value of EGL_DONT_CARE for an 407 attribute, their values must agree. In most cases, this means 408 the values must be identical, but this may not be true for all 409 attributes. In particular, one endpoint must specify an 410 EGL_STREAM_ENDPOINT_NV value of EGL_STREAM_CONSUMER_NV, and 411 the other must specify a value of EGL_STREAM_PRODUCER_NV. 412 If the values for any attribute are not compatible, the stream will 413 transition to the EGL_STREAM_STATE_DISCONNECTED_KHR state. 414 415 Additionally, if the two EGLStream objects representing a remote 416 stream are created for EGLDisplays which cannot share resources, 417 the stream will transition to the EGL_STREAM_STATE_DISCONNECTED_KHR 418 state. 419 420 When using remote streams, there may be latency in communicating 421 state changes between the EGLStream objects representing the two 422 endpoints. For instance, when a new frame is inserted into the 423 stream by the producer, the consumer endpoint may not immediately 424 transition to EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR. Similarly, 425 when a frame is acquired by the consumer, the producer endpoint may 426 not immediately transition to 427 EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR. Applications should rely 428 only on the state of the endpoint on which they are operating to 429 determine the expected results of that operation. 430 431In section "3.10.1.1 Duplicating an EGLStream from a file descriptor" of 432the EGL_KHR_stream_cross_process_fd extension, add to the failure list 433for eglGetStreamFileDescriptor 434 435 - EGL_BAD_ACCESS is generated if the EGL_STREAM_TYPE_NV 436 attribute of <stream> is anything other than EGL_DONT_CARE or 437 EGL_STREAM_CROSS_PROCESS_NV. 438 439 - EGL_BAD_ACCESS is generated if the EGL_STREAM_PROTOCOL_NV 440 attribute of <stream> is anything other than EGL_DONT_CARE or 441 EGL_STREAM_PROTOCOL_FD_NV. 442 443 - EGL_BAD_ACCESS is generated if the EGL_STREAM_ENDPOINT_NV 444 attribute of <stream> is EGL_STREAM_LOCAL_NV. 445 446Issues 447 448 None 449 450Revision History 451 452 #4 (June 01, 2016) 453 - Removed sync object definition. It will be provided by a 454 separate extension. 455 - Folded in cross-partition extension. 456 - Added types for cross-object, cross-display, and 457 cross-system streams. 458 - General cleanup in preparation for publication. 459 460 #3 (September 16, 2015) Zander Clucas 461 - Removed dependence of the CROSS_PROCESS type on the 462 EGL_NV_cross_process_fd extension 463 - Added CROSS_PROCESS to the list of STREAM_TYPE legal values 464 - Added CROSS_PROCESS requirement for cross_process_fd protocol 465 466 #2 (December 11, 2014) Daniel Kartch 467 - Rewrote as NV draft for earlier release. 468 - Added section for creation of sync object. 469 - Reserved enum values. 470 471 #1 (October 10, 2014) Daniel Kartch 472 - Initial EXT draft 473