1// Copyright 2018-2024 The Khronos Group Inc. 2// 3// SPDX-License-Identifier: CC-BY-4.0 4 5[[video-encode-operations]] 6== Video Encode Operations 7 8[[encode-input-picture]] 9Video encode operations consume an _encode input picture_ and zero or more 10reference pictures, and produce compressed video data to a video bitstream 11buffer and an optional <<reconstructed-picture,reconstructed picture>>. 12 13[NOTE] 14.Note 15==== 16Such encode input pictures can be used 17ifdef::VK_KHR_video_decode_queue[] 18as the <<decode-output-picture,output>> of video decode operations, 19endif::VK_KHR_video_decode_queue[] 20with graphics or compute operations, 21ifdef::VK_KHR_surface[] 22or with <<wsi,Window System Integration>> APIs, 23endif::VK_KHR_surface[] 24depending on the capabilities of the implementation. 25==== 26 27Video encode operations may: access the following resources in the 28ename:VK_PIPELINE_STAGE_2_VIDEO_ENCODE_BIT_KHR stage: 29 30 * The image subregions corresponding to the source 31 <<encode-input-picture-info,encode input picture>> and 32 <<encode-active-reference-picture-info,active reference pictures>> with 33 access ename:VK_ACCESS_2_VIDEO_ENCODE_READ_BIT_KHR. 34 * The destination video bitstream buffer range and the optional 35 <<encode-reconstructed-picture-info,reconstructed picture>> with access 36 ename:VK_ACCESS_2_VIDEO_ENCODE_WRITE_BIT_KHR. 37 38The image subresource of each <<video-picture-resources,video picture 39resource>> accessed by the video encode operation is specified using a 40corresponding slink:VkVideoPictureResourceInfoKHR structure. 41Each such image subresource must: be in the appropriate image layout as 42follows: 43 44 * If the image subresource is used in the video encode operation as an 45 <<encode-input-picture,encode input picture>>, then it must: be in the 46 ename:VK_IMAGE_LAYOUT_VIDEO_ENCODE_SRC_KHR layout. 47 * If the image subresource is used in the video encode operation as a 48 <<reconstructed-picture,reconstructed picture>> or <<reference-picture, 49 reference picture>>, then it must: be in the 50 ename:VK_IMAGE_LAYOUT_VIDEO_ENCODE_DPB_KHR layout. 51 52[[encode-unsuccessful]] 53A video encode operation may: complete unsuccessfully. 54In this case the target video bitstream buffer will have undefined: 55contents. 56Similarly, if <<encode-ref-pic-setup,reference picture setup>> is requested, 57the <<reconstructed-picture,reconstructed-picture>> will also have 58undefined: contents, and the activated DPB slot will have an 59<<dpb-slot-states,invalid picture reference>>. 60 61If a video encode operation completes successfully and the codec-specific 62parameters provided by the application adhere to the syntactic and semantic 63requirements defined in the corresponding video compression standard, then 64the target video bitstream buffer will contain compressed video data after 65the execution of the video encode operation according to the respective 66<<encode-codec-specific-semantics,codec-specific semantics>>. 67 68 69[[encode-codec-specific-semantics]] 70=== Codec-Specific Semantics 71 72The following aspects of video encode operations are codec-specific: 73 74 * The compressed video data written to the target video bitstream buffer 75 range. 76 * The construction and interpretation of the list of 77 <<encode-active-reference-picture-info,active reference pictures>> and 78 the interpretation of the picture data referred to by the corresponding 79 image subregions. 80 * The construction and interpretation of information related to the 81 <<encode-input-picture-info,encode input picture>> and the 82 interpretation of the picture data referred to by the corresponding 83 image subregion. 84 * The decision on <<encode-ref-pic-setup,reference picture setup>>. 85 * The construction and interpretation of information related to the 86 optional <<encode-reconstructed-picture-info,reconstructed picture>> and 87 the generation of picture data to the corresponding image subregion. 88 * Certain aspects of <<encode-rate-control,rate control>>. 89 90These codec-specific behaviors are defined for each video codec operation 91separately. 92 93ifdef::VK_KHR_video_encode_h264[] 94 * If the used video codec operation is 95 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then the 96 codec-specific aspects of the video encoding process are performed as 97 defined in the <<encode-h264,H.264 Encode Operations>> section. 98endif::VK_KHR_video_encode_h264[] 99ifdef::VK_KHR_video_encode_h265[] 100 * If the used video codec operation is 101 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then the 102 codec-specific aspects of the video encoding process are performed as 103 defined in the <<encode-h265,H.265 Encode Operations>> section. 104endif::VK_KHR_video_encode_h265[] 105 106 107[[encode-overrides]] 108==== Video Encode Parameter Overrides 109 110Implementations supporting video encode operations for any particular video 111codec operation often support only a subset of the available encoding tools 112defined by the corresponding video compression standards. 113Accordingly, certain implementation-dependent limitations may: apply to 114codec-specific parameters provided through the structures defined in the 115Video Std headers corresponding to the used video codec operation. 116 117Exposing all of these restrictions on particular codec-specific parameter 118values or combinations thereof in the form of application-queryable 119capabilities is impractical, hence this specification allows implementations 120to _override_ the value of any of the codec-specific parameters, unless 121otherwise specified, as long as all of the following conditions are met: 122 123 * If the application-provided codec-specific parameters adhere to the 124 syntactic and semantic requirements and rules defined by the used video 125 compression standard, and thus would be usable to produce a video 126 bitstream compliant with that standard, then the codec-specific 127 parameters resulting from the process of implementation overrides must: 128 also adhere to the same requirements and rules, and any video bitstream 129 produced using the overridden parameters must: also be compliant. 130 * The overridden codec-specific parameter values must: not have an impact 131 on the codec-independent behaviors defined for video encode operations. 132 * The implementation must: not override any codec-specific parameters 133 specified to a command that may: cause application-provided 134 codec-specific parameters specified to subsequent commands to no longer 135 adhere to the semantic requirements and rules defined by the used video 136 compression standard, unless the implementation also overrides those 137 parameters to adhere to any such requirements and rules. 138 * The overridden codec-specific parameter values must: not have an impact 139 on the codec-specific picture data access semantics. 140 * The overridden codec-specific parameter values may: change the contents 141 of the codec-specific bitstream elements produced by video encode 142 operations or otherwise retrieved by the application (e.g. using the 143 flink:vkGetEncodedVideoSessionParametersKHR command) but must: still 144 adhere to the codec-specific semantics defined for that video codec 145 operation, including, but not limited to, the number, type, and order of 146 the encoded codec-specific bitstream elements. 147 148Besides codec-specific parameter overrides performed for 149implementation-dependent reasons, applications can: enable the 150implementation to apply additional <<encode-optimizing-overrides,optimizing 151overrides>> that may: improve the efficiency or performance of video 152encoding operations. 153However, implementations must: meet the conditions listed above even in case 154of such optimizing overrides. 155 156[NOTE] 157.Note 158==== 159Unless the application opts in for optimizing overrides, implementations are 160not expected to override any of the codec-specific parameters, except when 161such overrides are necessary for the correct operation of video encoder 162implementation due to limitations to the available encoding tools on that 163implementation. 164==== 165 166 167[[encode-operation-steps]] 168=== Video Encode Operation Steps 169 170Each video encode operation performs the following steps in the 171ename:VK_PIPELINE_STAGE_2_VIDEO_ENCODE_BIT_KHR stage: 172 173 1. Reads the input picture data from the <<encode-input-picture,encode 174 input picture>>; 175 2. Determine derived encoding quality parameters according to the 176 <<encode-codec-specific-semantics,codec-specific semantics>> and the 177 current <<encode-rate-control,rate control>> state; 178 3. Compresses the input picture data according to the 179 <<encode-codec-specific-semantics,codec-specific semantics>>, applying 180 any prediction data read from the <<active-reference-pictures,active 181 reference pictures>> and rate control restrictions in the process; 182 4. Writes the encoded bitstream data to the destination video bitstream 183 buffer range; 184 5. Performs picture reconstruction of the encoded video data according to 185 the <<encode-codec-specific-semantics,codec-specific semantics>>, 186 applying any prediction data read from the <<active-reference-pictures, 187 active reference pictures>> in the process, if a 188 <<reconstructed-picture,reconstructed picture>> is specified and 189 <<encode-ref-pic-setup,reference picture setup>> is requested; 190 6. If <<encode-ref-pic-setup,reference picture setup>> is requested, the 191 <<dpb-slot,DPB slot index>> specified in the 192 <<encode-reconstructed-picture-info,reconstructed picture information>> 193 is <<dpb-slot-states,activated>> with the 194 <<reconstructed-picture,reconstructed picture>>; 195 7. Writes the reconstructed picture data to the <<reconstructed-picture, 196 reconstructed picture>>, if one is specified, according to the 197 <<encode-codec-specific-semantics,codec-specific semantics>>. 198 199When <<encode-reconstructed-picture-info,reconstructed picture information>> 200is provided, the specified <<dpb-slot,DPB slot>> index is associated with 201the corresponding <<bound-reference-picture-resources,bound reference 202picture resource>>, indifferent of whether <<encode-ref-pic-setup,reference 203picture setup>> is requested. 204 205=== Capabilities 206 207[open,refpage='VkVideoEncodeCapabilitiesKHR',desc='Structure describing general video encode capabilities for a video profile',type='structs'] 208-- 209When calling flink:vkGetPhysicalDeviceVideoCapabilitiesKHR with 210pname:pVideoProfile->videoCodecOperation specifying an encode operation, the 211slink:VkVideoEncodeCapabilitiesKHR structure must: be included in the 212pname:pNext chain of the slink:VkVideoCapabilitiesKHR structure to retrieve 213capabilities specific to video encoding. 214 215The sname:VkVideoEncodeCapabilitiesKHR structure is defined as: 216 217include::{generated}/api/structs/VkVideoEncodeCapabilitiesKHR.adoc[] 218 219 * pname:sType is a elink:VkStructureType value identifying this structure. 220 * pname:pNext is `NULL` or a pointer to a structure extending this 221 structure. 222 * pname:flags is a bitmask of elink:VkVideoEncodeCapabilityFlagBitsKHR 223 describing supported encoding features. 224 * pname:rateControlModes is a bitmask of 225 elink:VkVideoEncodeRateControlModeFlagBitsKHR indicating supported 226 <<encode-rate-control-modes,rate control modes>>. 227 * pname:maxRateControlLayers indicates the maximum number of 228 <<encode-rate-control-layers,rate control layers>> supported. 229 * pname:maxBitrate indicates the maximum supported bitrate. 230 * pname:maxQualityLevels indicates the number of discrete 231 <<encode-quality-level,video encode quality levels>> supported. 232 Implementations must: support at least one quality level. 233 * pname:encodeInputPictureGranularity indicates the granularity at which 234 <<encode-input-picture,encode input picture>> data is encoded and may: 235 indicate a texel granularity up to the size of the codec-specific coding 236 block size. 237 This capability does not impose any valid usage constraints on the 238 application, however, depending on the contents of the encode input 239 picture, it may: have effects on the encoded bitstream, as described in 240 more detail below. 241 * pname:supportedEncodeFeedbackFlags is a bitmask of 242 elink:VkVideoEncodeFeedbackFlagBitsKHR values specifying the supported 243 flags for <<queries-video-encode-feedback,video encode feedback 244 queries>>. 245 246Implementations must: include support for at least 247ename:VK_VIDEO_ENCODE_FEEDBACK_BITSTREAM_BUFFER_OFFSET_BIT_KHR and 248ename:VK_VIDEO_ENCODE_FEEDBACK_BITSTREAM_BYTES_WRITTEN_BIT_KHR in 249pname:supportedEncodeFeedbackFlags. 250 251pname:encodeInputPictureGranularity provides information about the way 252<<encode-input-picture,encode input picture>> data is used as input to video 253encode operations. 254In particular, some implementations may: not be able to limit the set of 255texels used to encode the output video bitstream to the image subregion 256specified in the slink:VkVideoPictureResourceInfoKHR structure corresponding 257to the encode input picture (i.e. to the resolution of the image data to 258encode specified in its pname:codedExtent member). 259 260[NOTE] 261.Note 262==== 263For example, the application requests the coded extent to be 1920x1080, but 264the implementation is only able to source the encode input picture data at 265the granularity of the codec-specific coding block size which is 16x16 266pixels (or as otherwise indicated in pname:encodeInputPictureGranularity). 267In this example, the content is horizontally aligned with the coding block 268size, but not vertically aligned with it. 269Thus encoding of the last row of coding blocks will be impacted by the 270contents of the input image at texel rows 1080 to 1087 (the latter being the 271next row which is vertically aligned with the coding block size, assuming a 272zero-based texel row index). 273==== 274 275If pname:codedExtent rounded up to the next integer multiple of 276pname:encodeInputPictureGranularity is greater than the extent of the image 277subresource specified for the <<encode-input-picture,encode input picture>>, 278then the texel values corresponding to texel coordinates outside of the 279bounds of the image subresource may: be undefined:. 280However, implementations should: use well-defined default values for such 281texels in order to maximize the encoding efficiency for the last coding 282block row/column, and/or to ensure consistent encoding results across 283repeated encoding of the same input content. 284Nonetheless, the values used for such texels must: not have an effect on 285whether the video encode operation produces a compliant bitstream, and must: 286not have any other effects on the encoded picture data beyond what may: 287otherwise result from using these texel values as input to any compression 288algorithm, as defined in the used video compression standard. 289 290[NOTE] 291.Note 292==== 293While not required, it is generally a good practice for applications to make 294sure that the image subresource used for the encode input picture has an 295extent that is an integer multiple of the codec-specific coding block size 296(or at least pname:encodeInputPictureGranularity) and that this padding area 297is filled with known values in order to improve encoding efficiency, 298portability, and reproducibility. 299==== 300 301include::{generated}/validity/structs/VkVideoEncodeCapabilitiesKHR.adoc[] 302-- 303 304[open,refpage='VkVideoEncodeCapabilityFlagBitsKHR',desc='Video encode capability flags',type='enums'] 305-- 306Bits which may: be set in slink:VkVideoEncodeCapabilitiesKHR::pname:flags, 307indicating the encoding tools supported, are: 308 309include::{generated}/api/enums/VkVideoEncodeCapabilityFlagBitsKHR.adoc[] 310 311 * ename:VK_VIDEO_ENCODE_CAPABILITY_PRECEDING_EXTERNALLY_ENCODED_BYTES_BIT_KHR 312 indicates that the implementation supports the use of 313 slink:VkVideoEncodeInfoKHR::pname:precedingExternallyEncodedBytes. 314 * ename:VK_VIDEO_ENCODE_CAPABILITY_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_DETECTION_BIT_KHR 315 indicates that the implementation is able to detect and report when the 316 destination video bitstream buffer range provided by the application is 317 not sufficiently large to fit the encoded bitstream data produced by a 318 video encode operation by reporting the 319 ename:VK_QUERY_RESULT_STATUS_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_KHR 320 <<query-result-status-codes,query result status code>>. 321+ 322[NOTE] 323.Note 324==== 325Some implementations may: not be able to reliably detect insufficient 326bitstream buffer range conditions in all situations. 327Such implementations will not report support for the 328ename:VK_VIDEO_ENCODE_CAPABILITY_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_DETECTION_BIT_KHR 329encode capability flag for the video profile, but may: still report the 330ename:VK_QUERY_RESULT_STATUS_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_KHR query 331result status code in certain cases. 332Applications should: always check for the specific query result status code 333ename:VK_QUERY_RESULT_STATUS_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_KHR even 334when this encode capability flag is not supported by the implementation for 335the video profile in question. 336However, applications must: not assume that a different negative query 337result status code indicating an unsuccessful completion of a video encode 338operation is not the result of an insufficient bitstream buffer condition 339unless this encode capability flag is supported. 340==== 341-- 342 343[open,refpage='VkVideoEncodeCapabilityFlagsKHR',desc='Bitmask of VkVideoEncodeCapabilityFlagBitsKHR',type='flags'] 344-- 345include::{generated}/api/flags/VkVideoEncodeCapabilityFlagsKHR.adoc[] 346 347tname:VkVideoEncodeCapabilityFlagsKHR is a bitmask type for setting a mask 348of zero or more elink:VkVideoEncodeCapabilityFlagBitsKHR. 349-- 350 351 352[[encode-quality-level]] 353=== Video Encode Quality Levels 354 355Implementations can: support more than one video encode quality levels for a 356video encode profile, which control the number and type of 357implementation-specific encoding tools and algorithms utilized in the 358encoding process. 359 360[NOTE] 361.Note 362==== 363Generally, using higher video encode quality levels may: produce higher 364quality video streams at the cost of additional processing time. 365However, as the final quality of an encoded picture depends on the contents 366of the <<encode-input-picture,encode input picture>>, the contents of the 367<<active-reference-pictures,active reference pictures>>, the codec-specific 368encode parameters, and the particular implementation-specific tools used 369corresponding to the individual video encode quality levels, there are no 370guarantees that using a higher video encode quality level will always 371produce a higher quality encoded picture for any given set of inputs. 372==== 373 374[open,refpage='vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR',desc='Query video encode quality level properties',type='protos'] 375-- 376To query properties for a specific video encode quality level supported by a 377video encode profile, call: 378 379include::{generated}/api/protos/vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR.adoc[] 380 381 * pname:physicalDevice is the physical device to query the video encode 382 quality level properties for. 383 * pname:pQualityLevelInfo is a pointer to a 384 slink:VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR structure 385 specifying the video encode profile and quality level to query 386 properties for. 387 * pname:pQualityLevelProperties is a pointer to a 388 slink:VkVideoEncodeQualityLevelPropertiesKHR structure in which the 389 properties are returned. 390 391.Valid Usage 392**** 393ifdef::VK_KHR_video_encode_h264[] 394 * [[VUID-vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR-pQualityLevelInfo-08257]] 395 If pname:pQualityLevelInfo->pVideoProfile->videoCodecOperation is 396 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then the pname:pNext 397 chain of pname:pQualityLevelProperties must: include a 398 slink:VkVideoEncodeH264QualityLevelPropertiesKHR structure 399endif::VK_KHR_video_encode_h264[] 400ifdef::VK_KHR_video_encode_h265[] 401 * [[VUID-vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR-pQualityLevelInfo-08258]] 402 If pname:pQualityLevelInfo->pVideoProfile->videoCodecOperation is 403 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then the pname:pNext 404 chain of pname:pQualityLevelProperties must: include a 405 slink:VkVideoEncodeH265QualityLevelPropertiesKHR structure 406endif::VK_KHR_video_encode_h265[] 407**** 408 409include::{generated}/validity/protos/vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR.adoc[] 410-- 411 412 413[open,refpage='VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR',desc='Structure describing the video encode profile and quality level to query properties for',type='structs'] 414-- 415The sname:VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR structure is 416defined as: 417 418include::{generated}/api/structs/VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR.adoc[] 419 420 * pname:sType is a elink:VkStructureType value identifying this structure. 421 * pname:pNext is `NULL` or a pointer to a structure extending this 422 structure. 423 * pname:pVideoProfile is a pointer to a slink:VkVideoProfileInfoKHR 424 structure specifying the video profile to query the video encode quality 425 level properties for. 426 * pname:qualityLevel is the video encode quality level to query properties 427 for. 428 429.Valid Usage 430**** 431 * [[VUID-VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR-pVideoProfile-08259]] 432 pname:pVideoProfile must: be a <<video-profile-support, supported video 433 profile>> 434 * [[VUID-VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR-pVideoProfile-08260]] 435 pname:pVideoProfile->videoCodecOperation must: specify an encode 436 operation 437 * [[VUID-VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR-qualityLevel-08261]] 438 pname:qualityLevel must: be less than 439 slink:VkVideoEncodeCapabilitiesKHR::pname:maxQualityLevels, as returned 440 by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video profile 441 specified in pname:pVideoProfile 442**** 443 444include::{generated}/validity/structs/VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR.adoc[] 445-- 446 447 448[open,refpage='VkVideoEncodeQualityLevelPropertiesKHR',desc='Structure describing the video encode quality level properties',type='structs'] 449-- 450The sname:VkVideoEncodeQualityLevelPropertiesKHR structure is defined as: 451 452include::{generated}/api/structs/VkVideoEncodeQualityLevelPropertiesKHR.adoc[] 453 454 * pname:sType is a elink:VkStructureType value identifying this structure. 455 * pname:pNext is `NULL` or a pointer to a structure extending this 456 structure. 457 * pname:preferredRateControlMode is a 458 elink:VkVideoEncodeRateControlModeFlagBitsKHR value indicating the 459 preferred <<encode-rate-control-modes,rate control mode>> to use with 460 the video encode quality level. 461 * pname:preferredRateControlLayerCount indicates the preferred number of 462 <<encode-rate-control-layers,rate control layers>> to use with the video 463 encode quality level. 464 465include::{generated}/validity/structs/VkVideoEncodeQualityLevelPropertiesKHR.adoc[] 466-- 467 468 469[open,refpage='VkVideoEncodeQualityLevelInfoKHR',desc='Structure specifying used video encode quality level',type='structs'] 470-- 471The sname:VkVideoEncodeQualityLevelInfoKHR structure is defined as: 472 473include::{generated}/api/structs/VkVideoEncodeQualityLevelInfoKHR.adoc[] 474 475 * pname:sType is a elink:VkStructureType value identifying this structure. 476 * pname:pNext is `NULL` or a pointer to a structure extending this 477 structure. 478 * pname:qualityLevel is the used video encode quality level. 479 480This structure can: be specified in the following places: 481 482 * In the pname:pNext chain of slink:VkVideoSessionParametersCreateInfoKHR 483 to specify the video encode quality level to use for a video session 484 parameters object created for a video encode session. 485 If no instance of this structure is included in the pname:pNext chain of 486 slink:VkVideoSessionParametersCreateInfoKHR, then the video session 487 parameters object is created with a video encode quality level of zero. 488 * In the pname:pNext chain of slink:VkVideoCodingControlInfoKHR to change 489 the video encode quality level state of the bound video session. 490 491.Valid Usage 492**** 493 * [[VUID-VkVideoEncodeQualityLevelInfoKHR-qualityLevel-08311]] 494 pname:qualityLevel must: be less than 495 slink:VkVideoEncodeCapabilitiesKHR::pname:maxQualityLevels, as returned 496 by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video 497 profile 498**** 499 500include::{generated}/validity/structs/VkVideoEncodeQualityLevelInfoKHR.adoc[] 501-- 502 503 504=== Retrieving Encoded Session Parameters 505 506Any codec-specific parameters stored in video session parameters objects 507may: need to be separately encoded and included in the final video bitstream 508data, depending on the used video compression standard. 509In such cases the application must: call the 510flink:vkGetEncodedVideoSessionParametersKHR command to retrieve the encoded 511parameter data from the used video session parameters object in order to be 512able to produce a compliant video bitstream. 513 514[NOTE] 515.Note 516==== 517This is needed because implementations may: have changed some of the 518codec-specific parameters stored in the video session parameters object, as 519defined in the <<encode-overrides,Video Encode Parameter Overrides>> 520section. 521In addition, the flink:vkGetEncodedVideoSessionParametersKHR command enables 522the application to retrieve the encoded parameter data without having to 523encode these codec-specific parameters manually. 524==== 525 526[open,refpage='vkGetEncodedVideoSessionParametersKHR',desc='Get encoded parameter sets from a video session parameters object',type='protos'] 527-- 528Encoded parameter data can: be retrieved from a video session parameters 529object created with a video encode operation using the command: 530 531include::{generated}/api/protos/vkGetEncodedVideoSessionParametersKHR.adoc[] 532 533 * pname:device is the logical device that owns the video session 534 parameters object. 535 * pname:pVideoSessionParametersInfo is a pointer to a 536 slink:VkVideoEncodeSessionParametersGetInfoKHR structure specifying the 537 parameters of the encoded parameter data to retrieve. 538 * pname:pFeedbackInfo is either `NULL` or a pointer to a 539 slink:VkVideoEncodeSessionParametersFeedbackInfoKHR structure in which 540 feedback about the requested parameter data is returned. 541 * pname:pDataSize is a pointer to a code:size_t value related to the 542 amount of encode parameter data returned, as described below. 543 * pname:pData is either `NULL` or a pointer to a buffer to write the 544 encoded parameter data to. 545 546If pname:pData is `NULL`, then the size of the encoded parameter data, in 547bytes, that can: be retrieved is returned in pname:pDataSize. 548Otherwise, pname:pDataSize must: point to a variable set by the application 549to the size of the buffer, in bytes, pointed to by pname:pData, and on 550return the variable is overwritten with the number of bytes actually written 551to pname:pData. 552If pname:pDataSize is less than the size of the encoded parameter data that 553can: be retrieved, then no data will be written to pname:pData, zero will be 554written to pname:pDataSize, and ename:VK_INCOMPLETE will be returned instead 555of ename:VK_SUCCESS, to indicate that no encoded parameter data was 556returned. 557 558If pname:pFeedbackInfo is not `NULL` then the members of the 559slink:VkVideoEncodeSessionParametersFeedbackInfoKHR structure and any 560additional structures included in its pname:pNext chain that are applicable 561to the video session parameters object specified in 562pname:pVideoSessionParametersInfo->videoSessionParameters will be filled 563with feedback about the requested parameter data on all successful calls to 564this command. 565 566[NOTE] 567.Note 568==== 569This includes the cases when pname:pData is `NULL` or when 570ename:VK_INCOMPLETE is returned by the command, and enables the application 571to determine whether the implementation <<encode-overrides,overrode>> any of 572the requested video session parameters without actually needing to retrieve 573the encoded parameter data itself. 574==== 575 576.Valid Usage 577**** 578 * [[VUID-vkGetEncodedVideoSessionParametersKHR-pVideoSessionParametersInfo-08359]] 579 pname:pVideoSessionParametersInfo->videoSessionParameters must: have 580 been created with an encode operation 581ifdef::VK_KHR_video_encode_h264[] 582 * [[VUID-vkGetEncodedVideoSessionParametersKHR-pVideoSessionParametersInfo-08262]] 583 If pname:pVideoSessionParametersInfo->videoSessionParameters was created 584 with the video codec operation 585 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then the pname:pNext 586 chain of pname:pVideoSessionParametersInfo must: include a 587 slink:VkVideoEncodeH264SessionParametersGetInfoKHR structure 588 * [[VUID-vkGetEncodedVideoSessionParametersKHR-pVideoSessionParametersInfo-08263]] 589 If pname:pVideoSessionParametersInfo->videoSessionParameters was created 590 with the video codec operation 591 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then for the 592 slink:VkVideoEncodeH264SessionParametersGetInfoKHR structure included in 593 the pname:pNext chain of pname:pVideoSessionParametersInfo, if its 594 pname:writeStdSPS member is ename:VK_TRUE, then 595 pname:pVideoSessionParametersInfo->videoSessionParameters must: contain 596 a code:StdVideoH264SequenceParameterSet entry with 597 pname:seq_parameter_set_id matching 598 slink:VkVideoEncodeH264SessionParametersGetInfoKHR::pname:stdSPSId 599 * [[VUID-vkGetEncodedVideoSessionParametersKHR-pVideoSessionParametersInfo-08264]] 600 If pname:pVideoSessionParametersInfo->videoSessionParameters was created 601 with the video codec operation 602 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then for the 603 slink:VkVideoEncodeH264SessionParametersGetInfoKHR structure included in 604 the pname:pNext chain of pname:pVideoSessionParametersInfo, if its 605 pname:writeStdPPS member is ename:VK_TRUE, then 606 pname:pVideoSessionParametersInfo->videoSessionParameters must: contain 607 a code:StdVideoH264PictureParameterSet entry with 608 pname:seq_parameter_set_id and pname:pic_parameter_set_id matching 609 slink:VkVideoEncodeH264SessionParametersGetInfoKHR::pname:stdSPSId and 610 slink:VkVideoEncodeH264SessionParametersGetInfoKHR::pname:stdPPSId, 611 respectively 612endif::VK_KHR_video_encode_h264[] 613ifdef::VK_KHR_video_encode_h265[] 614 * [[VUID-vkGetEncodedVideoSessionParametersKHR-pVideoSessionParametersInfo-08265]] 615 If pname:pVideoSessionParametersInfo->videoSessionParameters was created 616 with the video codec operation 617 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then the pname:pNext 618 chain of pname:pVideoSessionParametersInfo must: include a 619 slink:VkVideoEncodeH265SessionParametersGetInfoKHR structure 620 * [[VUID-vkGetEncodedVideoSessionParametersKHR-pVideoSessionParametersInfo-08266]] 621 If pname:pVideoSessionParametersInfo->videoSessionParameters was created 622 with the video codec operation 623 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then for the 624 slink:VkVideoEncodeH265SessionParametersGetInfoKHR structure included in 625 the pname:pNext chain of pname:pVideoSessionParametersInfo, if its 626 pname:writeStdVPS member is ename:VK_TRUE, then 627 pname:pVideoSessionParametersInfo->videoSessionParameters must: contain 628 a code:StdVideoH265VideoParameterSet entry with 629 pname:vps_video_parameter_set_id matching 630 slink:VkVideoEncodeH265SessionParametersGetInfoKHR::pname:stdVPSId 631 * [[VUID-vkGetEncodedVideoSessionParametersKHR-pVideoSessionParametersInfo-08267]] 632 If pname:pVideoSessionParametersInfo->videoSessionParameters was created 633 with the video codec operation 634 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then for the 635 slink:VkVideoEncodeH265SessionParametersGetInfoKHR structure included in 636 the pname:pNext chain of pname:pVideoSessionParametersInfo, if its 637 pname:writeStdSPS member is ename:VK_TRUE, then 638 pname:pVideoSessionParametersInfo->videoSessionParameters must: contain 639 a code:StdVideoH265SequenceParameterSet entry with 640 pname:sps_video_parameter_set_id and pname:sps_seq_parameter_set_id 641 matching 642 slink:VkVideoEncodeH265SessionParametersGetInfoKHR::pname:stdVPSId and 643 slink:VkVideoEncodeH265SessionParametersGetInfoKHR::pname:stdSPSId, 644 respectively 645 * [[VUID-vkGetEncodedVideoSessionParametersKHR-pVideoSessionParametersInfo-08268]] 646 If pname:pVideoSessionParametersInfo->videoSessionParameters was created 647 with the video codec operation 648 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then for the 649 slink:VkVideoEncodeH265SessionParametersGetInfoKHR structure included in 650 the pname:pNext chain of pname:pVideoSessionParametersInfo, if its 651 pname:writeStdPPS member is ename:VK_TRUE, then 652 pname:pVideoSessionParametersInfo->videoSessionParameters must: contain 653 a code:StdVideoH265PictureParameterSet entry with 654 pname:sps_video_parameter_set_id, pname:pps_seq_parameter_set_id, and 655 pname:pps_pic_parameter_set_id matching 656 slink:VkVideoEncodeH265SessionParametersGetInfoKHR::pname:stdVPSId, 657 slink:VkVideoEncodeH265SessionParametersGetInfoKHR::pname:stdSPSId, and 658 slink:VkVideoEncodeH265SessionParametersGetInfoKHR::pname:stdPPSId, 659 respectively 660endif::VK_KHR_video_encode_h265[] 661**** 662 663include::{generated}/validity/protos/vkGetEncodedVideoSessionParametersKHR.adoc[] 664-- 665 666[open,refpage='VkVideoEncodeSessionParametersGetInfoKHR',desc='Structure specifying parameters for retrieving encoded video session parameter data',type='structs'] 667-- 668The sname:VkVideoEncodeSessionParametersGetInfoKHR structure is defined as: 669 670include::{generated}/api/structs/VkVideoEncodeSessionParametersGetInfoKHR.adoc[] 671 672 * pname:sType is a elink:VkStructureType value identifying this structure. 673 * pname:pNext is `NULL` or a pointer to a structure extending this 674 structure. 675 * pname:videoSessionParameters is the slink:VkVideoSessionParametersKHR 676 object to retrieve encoded parameter data from. 677 678Depending on the used video encode operation, additional codec-specific 679structures may: need to be included in the pname:pNext chain of this 680structure to identify the specific video session parameters to retrieve 681encoded parameter data for, as described in the corresponding sections. 682 683include::{generated}/validity/structs/VkVideoEncodeSessionParametersGetInfoKHR.adoc[] 684-- 685 686[open,refpage='VkVideoEncodeSessionParametersFeedbackInfoKHR',desc='Structure providing feedback about the requested video session parameters',type='structs'] 687-- 688The sname:VkVideoEncodeSessionParametersFeedbackInfoKHR structure is defined 689as: 690 691include::{generated}/api/structs/VkVideoEncodeSessionParametersFeedbackInfoKHR.adoc[] 692 693 * pname:sType is a elink:VkStructureType value identifying this structure. 694 * pname:pNext is `NULL` or a pointer to a structure extending this 695 structure. 696 * pname:hasOverrides indicates whether any of the requested parameter data 697 were <<encode-overrides,overridden>> by the implementation. 698 699Depending on the used video encode operation, additional codec-specific 700structures can: be be included in the pname:pNext chain of this structure to 701capture codec-specific feedback information about the requested parameter 702data, as described in the corresponding sections. 703 704include::{generated}/validity/structs/VkVideoEncodeSessionParametersFeedbackInfoKHR.adoc[] 705-- 706 707 708[[video-encode-commands]] 709=== Video Encode Commands 710 711[open,refpage='vkCmdEncodeVideoKHR',desc='Launch video encode operations',type='protos'] 712-- 713To launch video encode operations, call: 714 715include::{generated}/api/protos/vkCmdEncodeVideoKHR.adoc[] 716 717 * pname:commandBuffer is the command buffer in which to record the 718 command. 719 * pname:pEncodeInfo is a pointer to a slink:VkVideoEncodeInfoKHR structure 720 specifying the parameters of the video encode operations. 721 722Each call issues one or more video encode operations. 723The implicit parameter pname:opCount corresponds to the number of video 724encode operations issued by the command. 725After calling this command, the 726<<queries-operation-active-query-index,active query index>> of each 727<<queries-operation-active,active>> query is incremented by pname:opCount. 728 729Currently each call to this command results in the issue of a single video 730encode operation. 731 732ifdef::VK_KHR_video_maintenance1[] 733If the bound video session was created with 734ename:VK_VIDEO_SESSION_CREATE_INLINE_QUERIES_BIT_KHR and the pname:pNext 735chain of pname:pEncodeInfo includes a slink:VkVideoInlineQueryInfoKHR 736structure with its pname:queryPool member specifying a valid 737sname:VkQueryPool handle, then this command will execute a query for each 738video encode operation issued by it. 739endif::VK_KHR_video_maintenance1[] 740 741[[encode-active-reference-picture-info]] 742Active Reference Picture Information:: 743 744The list of <<active-reference-pictures,active reference pictures>> used by 745a video encode operation is a list of image subregions used as the source of 746<<reference-picture,reference picture>> data and related parameters, and is 747derived from the slink:VkVideoReferenceSlotInfoKHR structures provided as 748the elements of the pname:pEncodeInfo->pReferenceSlots array. 749For each element of pname:pEncodeInfo->pReferenceSlots, one or more elements 750are added to the active reference picture list, as defined by the 751<<encode-codec-specific-semantics,codec-specific semantics>>. 752Each element of this list contains the following information: 753 754 * The image subregion within the image subresource 755 <<video-image-subresource-reference,referred>> to by the 756 <<video-picture-resources,video picture resource>> used as the reference 757 picture. 758 * The <<dpb-slot,DPB slot>> index the reference picture is associated 759 with. 760 * The codec-specific reference information related to the reference 761 picture. 762 763[[encode-reconstructed-picture-info]] 764Reconstructed Picture Information:: 765 766Information related to the optional <<reconstructed-picture,reconstructed 767picture>> used by a video encode operation is derived from the 768slink:VkVideoReferenceSlotInfoKHR structure pointed to by 769pname:pEncodeInfo->pSetupReferenceSlot, if not `NULL`, as defined by the 770<<encode-codec-specific-semantics,codec-specific semantics>>, and consists 771of the following: 772 773 * The image subregion within the image subresource 774 <<video-image-subresource-reference,referred>> to by the 775 <<video-picture-resources,video picture resource>> used as the 776 reconstructed picture. 777 * The <<dpb-slot,DPB slot>> index to use for picture reconstruction. 778 * The codec-specific reference information related to the reconstructed 779 picture. 780 781[[encode-ref-pic-setup]] 782Specifying a valid slink:VkVideoReferenceSlotInfoKHR structure in 783pname:pEncodeInfo->pSetupReferenceSlot is always required, unless the video 784session was created with slink:VkVideoSessionCreateInfoKHR::pname:maxDpbSlot 785equal to zero. 786However, the DPB slot identified by 787pname:pEncodeInfo->pSetupReferenceSlot->slotIndex is only 788<<dpb-slot-states,activated>> with the <<reconstructed-picture,reconstructed 789picture>> specified in 790pname:pEncodeInfo->pSetupReferenceSlot->pPictureResource if reference 791picture setup is requested according to the 792<<encode-codec-specific-semantics,codec-specific semantics>>. 793 794If reconstructed picture information is specified, but reference picture 795setup is not requested, according to the codec-specific semantics, the 796contents of the <<video-picture-resources,video picture resource>> 797corresponding to the reconstructed picture will be undefined: after the 798video encode operation. 799 800[NOTE] 801.Note 802==== 803Some implementations may always output the reconstructed picture or use it 804as temporary storage during the video encode operation even when the 805reconstructed picture is not marked for future reference. 806==== 807 808[[encode-input-picture-info]] 809Encode Input Picture Information:: 810 811Information related to the <<encode-input-picture,encode input picture>> 812used by a video encode operation is derived from 813pname:pEncodeInfo->srcPictureResource and any codec-specific parameters 814provided in the pname:pEncodeInfo->pNext chain, as defined by the 815<<encode-codec-specific-semantics,codec-specific semantics>>, and consists 816of the following: 817 818 * The image subregion within the image subresource 819 <<video-image-subresource-reference,referred>> to by the 820 <<video-picture-resources,video picture resource>> used as the encode 821 input picture. 822 * The codec-specific picture information related to the encoded picture. 823 824Several limiting values are defined below that are referenced by the 825relevant valid usage statements of this command. 826 827 * Let `uint32_t activeReferencePictureCount` be the size of the list of 828 active reference pictures used by the video encode operation. 829 Unless otherwise defined, pname:activeReferencePictureCount is set to 830 the value of pname:pEncodeInfo->referenceSlotCount. 831 * Let `VkOffset2D codedOffsetGranularity` be the minimum alignment 832 requirement for the coded offset of video picture resources. 833 Unless otherwise defined, the value of the pname:x and pname:y members 834 of pname:codedOffsetGranularity are `0`. 835 * Let `uint32_t dpbFrameUseCount[]` be an array of size pname:maxDpbSlots, 836 where pname:maxDpbSlots is the 837 slink:VkVideoSessionCreateInfoKHR::pname:maxDpbSlots the bound video 838 session was created with, with each element indicating the number of 839 times a frame associated with the corresponding DPB slot index is 840 referred to by the video coding operation. 841 Let the initial value of each element of the array be `0`. 842 ** If pname:pEncodeInfo->pSetupReferenceSlot is not `NULL`, then 843 `dpbFrameUseCount[i]` is incremented by one, where pname:i equals 844 pname:pEncodeInfo->pSetupReferenceSlot->slotIndex. 845 ** For each element of pname:pEncodeInfo->pReferenceSlots, 846 `dpbFrameUseCount[i]` is incremented by one, where pname:i equals the 847 pname:slotIndex member of the corresponding element. 848 * Let `VkExtent2D maxCodingBlockSize` be the maximum codec-specific coding 849 block size that may: be used by the video encode operation. 850ifdef::VK_KHR_video_encode_h264[] 851 ** If the bound video session object was created with an 852 <<encode-h264-profile,H.264 encode profile>>, then let 853 pname:maxCodingBlockSize be equal to the size of an H.264 macroblock, 854 i.e. `{16,16}`. 855endif::VK_KHR_video_encode_h264[] 856ifdef::VK_KHR_video_encode_h265[] 857 ** If the bound video session object was created with an 858 <<encode-h265-profile,H.265 encode profile>>, then let 859 pname:maxCodingBlockSize be equal to the maximum H.265 coding block 860 size that may: be used by the video encode operation derived as the 861 maximum of the CTB sizes corresponding to the 862 elink:VkVideoEncodeH265CtbSizeFlagBitsKHR bits set in 863 slink:VkVideoEncodeH265CapabilitiesKHR::pname:ctbSizes, as returned by 864 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video profile the 865 bound video session was created with. 866endif::VK_KHR_video_encode_h265[] 867 ** Otherwise, pname:maxCodingBlockSize is undefined:. 868 * If pname:maxCodingBlockSize is defined, then let `VkExtent2D 869 minCodingBlockExtent` be the coded extent of the 870 <<encode-input-picture,encode input picture>> expressed in terms of 871 codec-specific coding blocks, assuming the maximum size of such coding 872 blocks, as defined by pname:maxCodingBlockSize, calculated from the 873 value of the pname:codedExtent member of 874 pname:pEncodeInfo->srcPictureResource as follows: 875 ** [eq]#pname:minCodingBlockExtent.width = (pname:codedExtent.width + 876 pname:maxCodingBlockSize.width - 1) / pname:maxCodingBlockSize.width# 877 ** [eq]#pname:minCodingBlockExtent.height = (pname:codedExtent.height + 878 pname:maxCodingBlockSize.height - 1) / pname:maxCodingBlockSize.height# 879ifdef::VK_KHR_video_encode_h264[] 880 * If the bound video session object was created with an 881 <<encode-h264-profile,H.264 encode profile>>, then: 882 ** Let `StdVideoH264PictureType h264PictureType` be the picture type of 883 the encoded picture set to the value of 884 pname:pStdPictureInfo->primary_pic_type specified in the 885 slink:VkVideoEncodeH264PictureInfoKHR structure included in the 886 pname:pEncodeInfo->pNext chain. 887 ** Let `StdVideoH264PictureType h264L0PictureTypes[]` and 888 `StdVideoH264PictureType h264L1PictureTypes[]` be the picture types of 889 the reference pictures in the L0 and L1 reference lists, respectively. 890 If pname:pStdPictureInfo->pRefLists specified in the 891 slink:VkVideoEncodeH264PictureInfoKHR structure included in the 892 pname:pEncodeInfo->pNext chain is not `NULL`, then for each reference 893 index specified in the elements of the 894 pname:pStdPictureInfo->pRefLists->RefPicList0 and 895 pname:pStdPictureInfo->pRefLists->RefPicList1 arrays, if the reference 896 index is not code:STD_VIDEO_H264_NO_REFERENCE_PICTURE, 897 pname:pStdReferenceInfo->primary_pic_type is added to 898 pname:h264L0PictureTypes or pname:h264L1PictureTypes, respectively, 899 where pname:pStdReferenceInfo is the member of the 900 slink:VkVideoEncodeH264DpbSlotInfoKHR structure included in the 901 pname:pNext chain of the element of pname:pEncodeInfo->pReferenceSlots 902 for which pname:slotIndex equals the reference index in question. 903endif::VK_KHR_video_encode_h264[] 904ifdef::VK_KHR_video_encode_h265[] 905 * If the bound video session object was created with an 906 <<encode-h264-profile,H.265 encode profile>>, then: 907 ** Let `StdVideoH265PictureType h265PictureType` be the picture type of 908 the encoded picture set to the value of pname:pStdPictureInfo->pic_type 909 specified in the slink:VkVideoEncodeH265PictureInfoKHR structure 910 included in the pname:pEncodeInfo->pNext chain. 911 ** Let `StdVideoH265PictureType h265L0PictureTypes[]` and 912 `StdVideoH265PictureType h265L1PictureTypes[]` be the picture types of 913 the reference pictures in the L0 and L1 reference lists, respectively. 914 If pname:pStdPictureInfo->pRefLists specified in the 915 slink:VkVideoEncodeH265PictureInfoKHR structure included in the 916 pname:pEncodeInfo->pNext chain is not `NULL`, then for each reference 917 index specified in the elements of the 918 pname:pStdPictureInfo->pRefLists->RefPicList0 and 919 pname:pStdPictureInfo->pRefLists->RefPicList1 arrays, if the reference 920 index is not code:STD_VIDEO_H265_NO_REFERENCE_PICTURE, 921 pname:pStdReferenceInfo->pic_type is added to pname:h265L0PictureTypes 922 or pname:h265L1PictureTypes, respectively, where 923 pname:pStdReferenceInfo is the member of the 924 slink:VkVideoEncodeH265DpbSlotInfoKHR structure included in the 925 pname:pNext chain of the element of pname:pEncodeInfo->pReferenceSlots 926 for which pname:slotIndex equals the reference index in question. 927endif::VK_KHR_video_encode_h265[] 928 929.Valid Usage 930**** 931 * [[VUID-vkCmdEncodeVideoKHR-None-08250]] 932 The bound video session must: have been created with an encode operation 933 * [[VUID-vkCmdEncodeVideoKHR-None-07012]] 934 The bound video session must: not be in <<video-session-uninitialized, 935 uninitialized>> state at the time the command is executed on the device 936 * [[VUID-vkCmdEncodeVideoKHR-None-08318]] 937 The bound video session parameters object must: have been created with 938 the currently set <<encode-quality-level,video encode quality level>> 939 for the bound video session at the time the command is executed on the 940 device 941 * [[VUID-vkCmdEncodeVideoKHR-opCount-07174]] 942 For each <<queries-operation-active,active>> query, the 943 <<queries-operation-active-query-index,active query index>> 944 corresponding to the query type of that query plus pname:opCount must: 945 be less than or equal to the 946 <<queries-operation-last-activatable-query-index,last activatable query 947 index>> corresponding to the query type of that query plus one 948ifdef::VK_KHR_video_maintenance1[] 949 * [[VUID-vkCmdEncodeVideoKHR-pNext-08360]] 950 If the bound video session was created with 951 ename:VK_VIDEO_SESSION_CREATE_INLINE_QUERIES_BIT_KHR, and the 952 pname:pNext chain of pname:pEncodeInfo includes a 953 slink:VkVideoInlineQueryInfoKHR structure with its pname:queryPool 954 member specifying a valid sname:VkQueryPool handle, then 955 slink:VkVideoInlineQueryInfoKHR::queryCount must: equal ename:opCount 956 * [[VUID-vkCmdEncodeVideoKHR-pNext-08361]] 957 If the bound video session was created with 958 ename:VK_VIDEO_SESSION_CREATE_INLINE_QUERIES_BIT_KHR, and the 959 pname:pNext chain of pname:pEncodeInfo includes a 960 slink:VkVideoInlineQueryInfoKHR structure with its pname:queryPool 961 member specifying a valid sname:VkQueryPool handle, then all the queries 962 used by the command, as specified by the slink:VkVideoInlineQueryInfoKHR 963 structure, must: be _unavailable_ 964 * [[VUID-vkCmdEncodeVideoKHR-queryType-08362]] 965 If the bound video session was created with 966 ename:VK_VIDEO_SESSION_CREATE_INLINE_QUERIES_BIT_KHR, then the 967 pname:queryType used to create the pname:queryPool specified in the 968 slink:VkVideoInlineQueryInfoKHR structure included in the pname:pNext 969 chain of pname:pEncodeInfo must: be 970 ename:VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR or 971 ename:VK_QUERY_TYPE_VIDEO_ENCODE_FEEDBACK_KHR 972 * [[VUID-vkCmdEncodeVideoKHR-queryPool-08363]] 973 If the bound video session was created with 974 ename:VK_VIDEO_SESSION_CREATE_INLINE_QUERIES_BIT_KHR, then the 975 pname:queryPool specified in the slink:VkVideoInlineQueryInfoKHR 976 structure included in the pname:pNext chain of pname:pEncodeInfo must: 977 have been created with a slink:VkVideoProfileInfoKHR structure included 978 in the pname:pNext chain of slink:VkQueryPoolCreateInfo identical to the 979 one specified in slink:VkVideoSessionCreateInfoKHR::pname:pVideoProfile 980 the bound video session was created with 981 * [[VUID-vkCmdEncodeVideoKHR-queryType-08364]] 982 If the bound video session was created with 983 ename:VK_VIDEO_SESSION_CREATE_INLINE_QUERIES_BIT_KHR, and the 984 pname:queryType used to create the pname:queryPool specified in the 985 slink:VkVideoInlineQueryInfoKHR structure included in the pname:pNext 986 chain of pname:pEncodeInfo is 987 ename:VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR, then the sname:VkCommandPool 988 that pname:commandBuffer was allocated from must: have been created with 989 a queue family index that supports <<queries-result-status-only,result 990 status queries>>, as indicated by 991 slink:VkQueueFamilyQueryResultStatusPropertiesKHR::pname:queryResultStatusSupport 992endif::VK_KHR_video_maintenance1[] 993 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08201]] 994 pname:pEncodeInfo->dstBuffer must: be <<video-profile-compatibility, 995 compatible>> with the video profile the bound video session was created 996 with 997 * [[VUID-vkCmdEncodeVideoKHR-commandBuffer-08202]] 998 If pname:commandBuffer is an unprotected command buffer and 999 <<limits-protectedNoFault, pname:protectedNoFault>> is not supported, 1000 then pname:pEncodeInfo->dstBuffer must: not be a protected buffer 1001 * [[VUID-vkCmdEncodeVideoKHR-commandBuffer-08203]] 1002 If pname:commandBuffer is a protected command buffer and 1003 <<limits-protectedNoFault, pname:protectedNoFault>> is not supported, 1004 then pname:pEncodeInfo->dstBuffer must: be a protected buffer 1005 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08204]] 1006 pname:pEncodeInfo->dstBufferOffset must: be an integer multiple of 1007 slink:VkVideoCapabilitiesKHR::pname:minBitstreamBufferOffsetAlignment, 1008 as returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the 1009 video profile the bound video session was created with 1010 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08205]] 1011 pname:pEncodeInfo->dstBufferRange must: be an integer multiple of 1012 slink:VkVideoCapabilitiesKHR::pname:minBitstreamBufferSizeAlignment, as 1013 returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video 1014 profile the bound video session was created with 1015 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08206]] 1016 pname:pEncodeInfo->srcPictureResource.imageViewBinding must: be 1017 <<video-profile-compatibility,compatible>> with the video profile the 1018 bound video session was created with 1019 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207]] 1020 The format of pname:pEncodeInfo->srcPictureResource.imageViewBinding 1021 must: match the slink:VkVideoSessionCreateInfoKHR::pname:pictureFormat 1022 the bound video session was created with 1023 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08208]] 1024 pname:pEncodeInfo->srcPictureResource.codedOffset must: be an integer 1025 multiple of pname:codedOffsetGranularity 1026 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08209]] 1027 pname:pEncodeInfo->srcPictureResource.codedExtent must: be between 1028 pname:minCodedExtent and pname:maxCodedExtent, inclusive, the bound 1029 video session was created with 1030 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08210]] 1031 pname:pEncodeInfo->srcPictureResource.imageViewBinding must: have been 1032 created with ename:VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR 1033 * [[VUID-vkCmdEncodeVideoKHR-commandBuffer-08211]] 1034 If pname:commandBuffer is an unprotected command buffer and 1035 <<limits-protectedNoFault, pname:protectedNoFault>> is not supported, 1036 then pname:pEncodeInfo->srcPictureResource.imageViewBinding must: not 1037 have been created from a protected image 1038 * [[VUID-vkCmdEncodeVideoKHR-commandBuffer-08212]] 1039 If pname:commandBuffer is a protected command buffer and 1040 <<limits-protectedNoFault, pname:protectedNoFault>> is not supported, 1041 then pname:pEncodeInfo->srcPictureResource.imageViewBinding must: have 1042 been created from a protected image 1043 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08377]] 1044 pname:pEncodeInfo->pSetupReferenceSlot must: not be `NULL` unless the 1045 bound video session was created with 1046 slink:VkVideoSessionCreateInfoKHR::pname:maxDpbSlots equal to zero 1047 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08213]] 1048 If pname:pEncodeInfo->pSetupReferenceSlot is not `NULL`, then 1049 pname:pEncodeInfo->pSetupReferenceSlot->slotIndex must: be less than the 1050 slink:VkVideoSessionCreateInfoKHR::pname:maxDpbSlots specified when the 1051 bound video session was created 1052 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08214]] 1053 If pname:pEncodeInfo->pSetupReferenceSlot is not `NULL`, then 1054 pname:pEncodeInfo->pSetupReferenceSlot->pPictureResource->codedOffset 1055 must: be an integer multiple of pname:codedOffsetGranularity 1056 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08215]] 1057 If pname:pEncodeInfo->pSetupReferenceSlot is not `NULL`, then 1058 pname:pEncodeInfo->pSetupReferenceSlot->pPictureResource must: 1059 <<video-picture-resource-matching,match>> one of the 1060 <<bound-reference-picture-resources,bound reference picture resource>> 1061 * [[VUID-vkCmdEncodeVideoKHR-activeReferencePictureCount-08216]] 1062 pname:activeReferencePictureCount must: be less than or equal to the 1063 slink:VkVideoSessionCreateInfoKHR::pname:maxActiveReferencePictures 1064 specified when the bound video session was created 1065 * [[VUID-vkCmdEncodeVideoKHR-slotIndex-08217]] 1066 The pname:slotIndex member of each element of 1067 pname:pEncodeInfo->pReferenceSlots must: be less than the 1068 slink:VkVideoSessionCreateInfoKHR::pname:maxDpbSlots specified when the 1069 bound video session was created 1070 * [[VUID-vkCmdEncodeVideoKHR-codedOffset-08218]] 1071 The pname:codedOffset member of the slink:VkVideoPictureResourceInfoKHR 1072 structure pointed to by the pname:pPictureResource member of each 1073 element of pname:pEncodeInfo->pReferenceSlots must: be an integer 1074 multiple of pname:codedOffsetGranularity 1075 * [[VUID-vkCmdEncodeVideoKHR-pPictureResource-08219]] 1076 The pname:pPictureResource member of each element of 1077 pname:pEncodeInfo->pReferenceSlots must: 1078 <<video-picture-resource-matching,match>> one of the 1079 <<bound-reference-picture-resources,bound reference picture resource>> 1080 associated with the DPB slot index specified in the pname:slotIndex 1081 member of that element 1082 * [[VUID-vkCmdEncodeVideoKHR-pPictureResource-08220]] 1083 Each video picture resource corresponding to the pname:pPictureResource 1084 member specified in the elements of pname:pEncodeInfo->pReferenceSlots 1085 must: be <<video-picture-resource-uniqueness,unique>> within 1086 pname:pEncodeInfo->pReferenceSlots 1087 * [[VUID-vkCmdEncodeVideoKHR-dpbFrameUseCount-08221]] 1088 All elements of pname:dpbFrameUseCount must: be less than or equal to 1089 `1` 1090 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08222]] 1091 The image subresource <<video-image-subresource-reference,referred>> to 1092 by pname:pEncodeInfo->srcPictureResource must: be in the 1093 ename:VK_IMAGE_LAYOUT_VIDEO_ENCODE_SRC_KHR layout at the time the video 1094 encode operation is executed on the device 1095 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08223]] 1096 If pname:pEncodeInfo->pSetupReferenceSlot is not `NULL`, then the image 1097 subresource <<video-image-subresource-reference,referred>> to by 1098 pname:pEncodeInfo->pSetupReferenceSlot->pPictureResource must: be in the 1099 ename:VK_IMAGE_LAYOUT_VIDEO_ENCODE_DPB_KHR layout at the time the video 1100 encode operation is executed on the device 1101 * [[VUID-vkCmdEncodeVideoKHR-pPictureResource-08224]] 1102 The image subresource <<video-image-subresource-reference,referred>> to 1103 by the pname:pPictureResource member of each element of 1104 pname:pEncodeInfo->pReferenceSlots must: be in the 1105 ename:VK_IMAGE_LAYOUT_VIDEO_ENCODE_DPB_KHR layout at the time the video 1106 encode operation is executed on the device 1107ifdef::VK_KHR_video_encode_h264[] 1108 * [[VUID-vkCmdEncodeVideoKHR-pNext-08225]] 1109 If the bound video session was created with the video codec operation 1110 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then the pname:pNext 1111 chain of pname:pEncodeInfo must: include a 1112 slink:VkVideoEncodeH264PictureInfoKHR structure 1113 * [[VUID-vkCmdEncodeVideoKHR-StdVideoH264SequenceParameterSet-08226]] 1114 If the bound video session was created with the video codec operation 1115 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then the bound video 1116 session parameters object must: contain a 1117 code:StdVideoH264SequenceParameterSet entry with 1118 pname:seq_parameter_set_id matching 1119 code:StdVideoEncodeH264PictureInfo::pname:seq_parameter_set_id that is 1120 provided in the pname:pStdPictureInfo member of the 1121 slink:VkVideoEncodeH264PictureInfoKHR structure included in the 1122 pname:pNext chain of pname:pEncodeInfo 1123 * [[VUID-vkCmdEncodeVideoKHR-StdVideoH264PictureParameterSet-08227]] 1124 If the bound video session was created with the video codec operation 1125 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then the bound video 1126 session parameters object must: contain a 1127 code:StdVideoH264PictureParameterSet entry with 1128 pname:seq_parameter_set_id and pname:pic_parameter_set_id matching 1129 code:StdVideoEncodeH264PictureInfo::pname:seq_parameter_set_id and 1130 code:StdVideoEncodeH264PictureInfo::pname:pic_parameter_set_id, 1131 respectively, that are provided in the pname:pStdPictureInfo member of 1132 the slink:VkVideoEncodeH264PictureInfoKHR structure included in the 1133 pname:pNext chain of pname:pEncodeInfo 1134 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08228]] 1135 If the bound video session was created with the video codec operation 1136 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and 1137 pname:pEncodeInfo->pSetupReferenceSlot is not `NULL`, then the 1138 pname:pNext chain of pname:pEncodeInfo->pSetupReferenceSlot must: 1139 include a slink:VkVideoEncodeH264DpbSlotInfoKHR structure 1140 * [[VUID-vkCmdEncodeVideoKHR-pNext-08229]] 1141 If the bound video session was created with the video codec operation 1142 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then the pname:pNext 1143 chain of each element of pname:pEncodeInfo->pReferenceSlots must: 1144 include a slink:VkVideoEncodeH264DpbSlotInfoKHR structure 1145 * [[VUID-vkCmdEncodeVideoKHR-constantQp-08269]] 1146 If the bound video session was created with the video codec operation 1147 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and the current 1148 <<encode-rate-control-modes,rate control mode>> is not 1149 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR, then 1150 slink:VkVideoEncodeH264NaluSliceInfoKHR::pname:constantQp must: be zero 1151 for each element of the pname:pNaluSliceEntries member of the 1152 slink:VkVideoEncodeH264PictureInfoKHR structure included in the 1153 pname:pNext chain of pname:pEncodeInfo 1154 * [[VUID-vkCmdEncodeVideoKHR-constantQp-08270]] 1155 If the bound video session was created with the video codec operation 1156 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and the current 1157 <<encode-rate-control-modes,rate control mode>> is 1158 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR, then 1159 slink:VkVideoEncodeH264NaluSliceInfoKHR::pname:constantQp must: be 1160 between slink:VkVideoEncodeH264CapabilitiesKHR::pname:minQp and 1161 slink:VkVideoEncodeH264CapabilitiesKHR::pname:maxQp, as returned by 1162 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video profile the 1163 bound video session was created with, for each element of the 1164 pname:pNaluSliceEntries member of the 1165 slink:VkVideoEncodeH264PictureInfoKHR structure included in the 1166 pname:pNext chain of pname:pEncodeInfo 1167 * [[VUID-vkCmdEncodeVideoKHR-constantQp-08271]] 1168 If the bound video session was created with the video codec operation 1169 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and 1170 slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags does not include 1171 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PER_SLICE_CONSTANT_QP_BIT_KHR, as 1172 returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video 1173 profile the bound video session was created with, then 1174 slink:VkVideoEncodeH264NaluSliceInfoKHR::pname:constantQp must: have the 1175 same value for each element of the pname:pNaluSliceEntries member of the 1176 slink:VkVideoEncodeH264PictureInfoKHR structure included in the 1177 pname:pNext chain of pname:pEncodeInfo 1178 * [[VUID-vkCmdEncodeVideoKHR-naluSliceEntryCount-08302]] 1179 If the bound video session was created with the video codec operation 1180 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then the 1181 pname:naluSliceEntryCount member of the 1182 slink:VkVideoEncodeH264PictureInfoKHR structure included in the 1183 pname:pNext chain of pname:pEncodeInfo must: be less than or equal to 1184 pname:minCodingBlockExtent.width multiplied by 1185 pname:minCodingBlockExtent.height 1186 * [[VUID-vkCmdEncodeVideoKHR-naluSliceEntryCount-08312]] 1187 If the bound video session was created with the video codec operation 1188 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and 1189 slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags does not include 1190 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_ROW_UNALIGNED_SLICE_BIT_KHR, as 1191 returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video 1192 profile the bound video session was created with, then the 1193 pname:naluSliceEntryCount member of the 1194 slink:VkVideoEncodeH264PictureInfoKHR structure included in the 1195 pname:pNext chain of pname:pEncodeInfo must: be less than or equal to 1196 pname:minCodingBlockExtent.height 1197 * [[VUID-vkCmdEncodeVideoKHR-pNext-08352]] 1198 If the bound video session was created with the video codec operation 1199 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, the pname:pNext 1200 chain of pname:pEncodeInfo includes a 1201 slink:VkVideoEncodeH264PictureInfoKHR structure, and 1202 pname:pEncodeInfo->referenceSlotCount is greater than zero, then 1203 slink:VkVideoEncodeH264PictureInfoKHR::pname:pStdPictureInfo->pRefLists 1204 must: not be `NULL` 1205 * [[VUID-vkCmdEncodeVideoKHR-pNext-08339]] 1206 If the bound video session was created with the video codec operation 1207 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, the pname:pNext 1208 chain of pname:pEncodeInfo includes a 1209 slink:VkVideoEncodeH264PictureInfoKHR structure, and 1210 slink:VkVideoEncodeH264PictureInfoKHR::pname:pStdPictureInfo->pRefLists 1211 is not `NULL`, then each element of the code:RefPicList0 and 1212 code:RefPicList1 array members of the 1213 code:StdVideoEncodeH264ReferenceListsInfo structure pointed to by 1214 slink:VkVideoEncodeH264PictureInfoKHR::pname:pStdPictureInfo->pRefLists 1215 must: either be code:STD_VIDEO_H264_NO_REFERENCE_PICTURE or must: equal 1216 the pname:slotIndex member of one of the elements of 1217 pname:pEncodeInfo->pReferenceSlots 1218 * [[VUID-vkCmdEncodeVideoKHR-pNext-08353]] 1219 If the bound video session was created with the video codec operation 1220 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, the pname:pNext 1221 chain of pname:pEncodeInfo includes a 1222 slink:VkVideoEncodeH264PictureInfoKHR structure, and 1223 pname:pEncodeInfo->referenceSlotCount is greater than zero, then the 1224 pname:slotIndex member of each element of 1225 pname:pEncodeInfo->pReferenceSlots must: equal one of the elements of 1226 the code:RefPicList0 or code:RefPicList1 array members of the 1227 code:StdVideoEncodeH264ReferenceListsInfo structure pointed to by 1228 slink:VkVideoEncodeH264PictureInfoKHR::pname:pStdPictureInfo->pRefLists 1229 * [[VUID-vkCmdEncodeVideoKHR-maxPPictureL0ReferenceCount-08340]] 1230 If the bound video session was created with the video codec operation 1231 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and 1232 slink:VkVideoEncodeH264CapabilitiesKHR::pname:maxPPictureL0ReferenceCount 1233 is zero, as returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR 1234 for the video profile the bound video session was created with, then 1235 pname:h264PictureType and each element of pname:h264L0PictureTypes and 1236 pname:h264L1PictureTypes must: not be code:STD_VIDEO_H264_PICTURE_TYPE_P 1237 * [[VUID-vkCmdEncodeVideoKHR-maxBPictureL0ReferenceCount-08341]] 1238 If the bound video session was created with the video codec operation 1239 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and 1240 slink:VkVideoEncodeH264CapabilitiesKHR::pname:maxBPictureL0ReferenceCount 1241 and slink:VkVideoEncodeH264CapabilitiesKHR::pname:maxL1ReferenceCount 1242 are both zero, as returned by 1243 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video profile the 1244 bound video session was created with, then pname:h264PictureType and 1245 each element of pname:h264L0PictureTypes and pname:h264L1PictureTypes 1246 must: not be code:STD_VIDEO_H264_PICTURE_TYPE_B 1247 * [[VUID-vkCmdEncodeVideoKHR-flags-08342]] 1248 If the bound video session was created with the video codec operation 1249 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and 1250 slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags does not include 1251 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_B_FRAME_IN_L0_LIST_BIT_KHR, as 1252 returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video 1253 profile the bound video session was created with, then each element of 1254 pname:h264L0PictureTypes must: not be code:STD_VIDEO_H264_PICTURE_TYPE_B 1255 * [[VUID-vkCmdEncodeVideoKHR-flags-08343]] 1256 If the bound video session was created with the video codec operation 1257 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and 1258 slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags does not include 1259 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_B_FRAME_IN_L1_LIST_BIT_KHR, as 1260 returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video 1261 profile the bound video session was created with, then each element of 1262 pname:h264L1PictureTypes must: not be code:STD_VIDEO_H264_PICTURE_TYPE_B 1263endif::VK_KHR_video_encode_h264[] 1264ifdef::VK_KHR_video_encode_h265[] 1265 * [[VUID-vkCmdEncodeVideoKHR-pNext-08230]] 1266 If the bound video session was created with the video codec operation 1267 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then the pname:pNext 1268 chain of pname:pEncodeInfo must: include a 1269 slink:VkVideoEncodeH265PictureInfoKHR structure 1270 * [[VUID-vkCmdEncodeVideoKHR-StdVideoH265VideoParameterSet-08231]] 1271 If the bound video session was created with the video codec operation 1272 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then the bound video 1273 session parameters object must: contain a 1274 code:StdVideoH265VideoParameterSet entry with 1275 pname:vps_video_parameter_set_id matching 1276 code:StdVideoEncodeH265PictureInfo::pname:sps_video_parameter_set_id 1277 that is provided in the pname:pStdPictureInfo member of the 1278 slink:VkVideoEncodeH265PictureInfoKHR structure included in the 1279 pname:pNext chain of pname:pEncodeInfo 1280 * [[VUID-vkCmdEncodeVideoKHR-StdVideoH265SequenceParameterSet-08232]] 1281 If the bound video session was created with the video codec operation 1282 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then the bound video 1283 session parameters object must: contain a 1284 code:StdVideoH265SequenceParameterSet entry with 1285 pname:sps_video_parameter_set_id and pname:sps_seq_parameter_set_id 1286 matching 1287 code:StdVideoEncodeH265PictureInfo::pname:sps_video_parameter_set_id and 1288 code:StdVideoEncodeH265PictureInfo::pname:pps_seq_parameter_set_id, 1289 respectively, that are provided in the pname:pStdPictureInfo member of 1290 the slink:VkVideoEncodeH265PictureInfoKHR structure included in the 1291 pname:pNext chain of pname:pEncodeInfo 1292 * [[VUID-vkCmdEncodeVideoKHR-StdVideoH265PictureParameterSet-08233]] 1293 If the bound video session was created with the video codec operation 1294 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then the bound video 1295 session parameters object must: contain a 1296 code:StdVideoH265PictureParameterSet entry with 1297 pname:sps_video_parameter_set_id, pname:pps_seq_parameter_set_id, and 1298 pname:pps_pic_parameter_set_id matching 1299 code:StdVideoEncodeH265PictureInfo::pname:sps_video_parameter_set_id, 1300 code:StdVideoEncodeH265PictureInfo::pname:pps_seq_parameter_set_id, and 1301 code:StdVideoEncodeH265PictureInfo::pname:pps_pic_parameter_set_id, 1302 respectively, that are provided in the pname:pStdPictureInfo member of 1303 the slink:VkVideoEncodeH265PictureInfoKHR structure included in the 1304 pname:pNext chain of pname:pEncodeInfo 1305 * [[VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08234]] 1306 If the bound video session was created with the video codec operation 1307 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR and 1308 pname:pEncodeInfo->pSetupReferenceSlot is not `NULL`, then the 1309 pname:pNext chain of pname:pEncodeInfo->pSetupReferenceSlot must: 1310 include a slink:VkVideoEncodeH265DpbSlotInfoKHR structure 1311 * [[VUID-vkCmdEncodeVideoKHR-pNext-08235]] 1312 If the bound video session was created with the video codec operation 1313 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then the pname:pNext 1314 chain of each element of pname:pEncodeInfo->pReferenceSlots must: 1315 include a slink:VkVideoEncodeH265DpbSlotInfoKHR structure 1316 * [[VUID-vkCmdEncodeVideoKHR-constantQp-08272]] 1317 If the bound video session was created with the video codec operation 1318 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR and the current 1319 <<encode-rate-control-modes,rate control mode>> is not 1320 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR, then 1321 slink:VkVideoEncodeH265NaluSliceSegmentInfoKHR::pname:constantQp must: 1322 be zero for each element of the pname:pNaluSliceSegmentEntries member of 1323 the slink:VkVideoEncodeH265PictureInfoKHR structure included in the 1324 pname:pNext chain of pname:pEncodeInfo 1325 * [[VUID-vkCmdEncodeVideoKHR-constantQp-08273]] 1326 If the bound video session was created with the video codec operation 1327 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR and the current 1328 <<encode-rate-control-modes,rate control mode>> is 1329 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR, then 1330 slink:VkVideoEncodeH265NaluSliceSegmentInfoKHR::pname:constantQp must: 1331 be between slink:VkVideoEncodeH265CapabilitiesKHR::pname:minQp and 1332 slink:VkVideoEncodeH265CapabilitiesKHR::pname:maxQp, as returned by 1333 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video profile the 1334 bound video session was created with, for each element of the 1335 pname:pNaluSliceSegmentEntries member of the 1336 slink:VkVideoEncodeH265PictureInfoKHR structure included in the 1337 pname:pNext chain of pname:pEncodeInfo 1338 * [[VUID-vkCmdEncodeVideoKHR-constantQp-08274]] 1339 If the bound video session was created with the video codec operation 1340 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR and 1341 slink:VkVideoEncodeH265CapabilitiesKHR::pname:flags does not include 1342 ename:VK_VIDEO_ENCODE_H265_CAPABILITY_PER_SLICE_SEGMENT_CONSTANT_QP_BIT_KHR, 1343 as returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the 1344 video profile the bound video session was created with, then 1345 slink:VkVideoEncodeH265NaluSliceSegmentInfoKHR::pname:constantQp must: 1346 have the same value for each element of the 1347 pname:pNaluSliceSegmentEntries member of the 1348 slink:VkVideoEncodeH264PictureInfoKHR structure included in the 1349 pname:pNext chain of pname:pEncodeInfo 1350 * [[VUID-vkCmdEncodeVideoKHR-naluSliceSegmentEntryCount-08307]] 1351 If the bound video session was created with the video codec operation 1352 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then the 1353 pname:naluSliceSegmentEntryCount member of the 1354 slink:VkVideoEncodeH265PictureInfoKHR structure included in the 1355 pname:pNext chain of pname:pEncodeInfo must: be less than or equal to 1356 pname:minCodingBlockExtent.width multiplied by 1357 pname:minCodingBlockExtent.height 1358 * [[VUID-vkCmdEncodeVideoKHR-naluSliceSegmentEntryCount-08313]] 1359 If the bound video session was created with the video codec operation 1360 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR and 1361 slink:VkVideoEncodeH265CapabilitiesKHR::pname:flags does not include 1362 ename:VK_VIDEO_ENCODE_H265_CAPABILITY_ROW_UNALIGNED_SLICE_SEGMENT_BIT_KHR, 1363 as returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the 1364 video profile the bound video session was created with, then the 1365 pname:naluSliceSegmentEntryCount member of the 1366 slink:VkVideoEncodeH265PictureInfoKHR structure included in the 1367 pname:pNext chain of pname:pEncodeInfo must: be less than or equal to 1368 pname:minCodingBlockExtent.height 1369 * [[VUID-vkCmdEncodeVideoKHR-pNext-08354]] 1370 If the bound video session was created with the video codec operation 1371 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, the pname:pNext 1372 chain of pname:pEncodeInfo includes a 1373 slink:VkVideoEncodeH265PictureInfoKHR structure, and 1374 pname:pEncodeInfo->referenceSlotCount is greater than zero, then 1375 slink:VkVideoEncodeH265PictureInfoKHR::pname:pStdPictureInfo->pRefLists 1376 must: not be `NULL` 1377 * [[VUID-vkCmdEncodeVideoKHR-pNext-08344]] 1378 If the bound video session was created with the video codec operation 1379 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, the pname:pNext 1380 chain of pname:pEncodeInfo includes a 1381 slink:VkVideoEncodeH265PictureInfoKHR structure, and 1382 slink:VkVideoEncodeH265PictureInfoKHR::pname:pStdPictureInfo->pRefLists 1383 is not `NULL`, then each element of the code:RefPicList0 and 1384 code:RefPicList1 array members of the 1385 code:StdVideoEncodeH265ReferenceListsInfo structure pointed to by 1386 slink:VkVideoEncodeH265PictureInfoKHR::pname:pStdPictureInfo->pRefLists 1387 must: either be code:STD_VIDEO_H265_NO_REFERENCE_PICTURE or must: equal 1388 the pname:slotIndex member of one of the elements of 1389 pname:pEncodeInfo->pReferenceSlots 1390 * [[VUID-vkCmdEncodeVideoKHR-pNext-08355]] 1391 If the bound video session was created with the video codec operation 1392 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, the pname:pNext 1393 chain of pname:pEncodeInfo includes a 1394 slink:VkVideoEncodeH265PictureInfoKHR structure, and 1395 pname:pEncodeInfo->referenceSlotCount is greater than zero, then the 1396 pname:slotIndex member of each element of 1397 pname:pEncodeInfo->pReferenceSlots must: equal one of the elements of 1398 the code:RefPicList0 or code:RefPicList1 array members of the 1399 code:StdVideoEncodeH265ReferenceListsInfo structure pointed to by 1400 slink:VkVideoEncodeH265PictureInfoKHR::pname:pStdPictureInfo->pRefLists 1401 * [[VUID-vkCmdEncodeVideoKHR-maxPPictureL0ReferenceCount-08345]] 1402 If the bound video session was created with the video codec operation 1403 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR and 1404 slink:VkVideoEncodeH265CapabilitiesKHR::pname:maxPPictureL0ReferenceCount 1405 is zero, as returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR 1406 for the video profile the bound video session was created with, then 1407 pname:h265PictureType and each element of pname:h265L0PictureTypes and 1408 pname:h265L1PictureTypes must: not be code:STD_VIDEO_H265_PICTURE_TYPE_P 1409 * [[VUID-vkCmdEncodeVideoKHR-maxBPictureL0ReferenceCount-08346]] 1410 If the bound video session was created with the video codec operation 1411 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR and 1412 slink:VkVideoEncodeH265CapabilitiesKHR::pname:maxBPictureL0ReferenceCount 1413 and slink:VkVideoEncodeH265CapabilitiesKHR::pname:maxL1ReferenceCount 1414 are both zero, as returned by 1415 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video profile the 1416 bound video session was created with, then pname:h265PictureType and 1417 each element of pname:h265L0PictureTypes and pname:h265L1PictureTypes 1418 must: not be code:STD_VIDEO_H265_PICTURE_TYPE_B 1419 * [[VUID-vkCmdEncodeVideoKHR-flags-08347]] 1420 If the bound video session was created with the video codec operation 1421 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR and 1422 slink:VkVideoEncodeH265CapabilitiesKHR::pname:flags does not include 1423 ename:VK_VIDEO_ENCODE_H265_CAPABILITY_B_FRAME_IN_L0_LIST_BIT_KHR, as 1424 returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video 1425 profile the bound video session was created with, then each element of 1426 pname:h265L0PictureTypes must: not be code:STD_VIDEO_H264_PICTURE_TYPE_B 1427 * [[VUID-vkCmdEncodeVideoKHR-flags-08348]] 1428 If the bound video session was created with the video codec operation 1429 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR and 1430 slink:VkVideoEncodeH265CapabilitiesKHR::pname:flags does not include 1431 ename:VK_VIDEO_ENCODE_H265_CAPABILITY_B_FRAME_IN_L1_LIST_BIT_KHR, as 1432 returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video 1433 profile the bound video session was created with, then each element of 1434 pname:h265L1PictureTypes must: not be code:STD_VIDEO_H265_PICTURE_TYPE_B 1435endif::VK_KHR_video_encode_h265[] 1436**** 1437 1438include::{generated}/validity/protos/vkCmdEncodeVideoKHR.adoc[] 1439-- 1440 1441[open,refpage='VkVideoEncodeInfoKHR',desc='Structure specifying video encode parameters',type='structs'] 1442-- 1443The sname:VkVideoEncodeInfoKHR structure is defined as: 1444 1445include::{generated}/api/structs/VkVideoEncodeInfoKHR.adoc[] 1446 1447 * pname:sType is a elink:VkStructureType value identifying this structure. 1448 * pname:pNext is a pointer to a structure extending this structure. 1449 * pname:flags is reserved for future use. 1450 * pname:dstBuffer is the destination video bitstream buffer to write the 1451 encoded bitstream to. 1452 * pname:dstBufferOffset is the starting offset in bytes from the start of 1453 pname:dstBuffer to write the encoded bitstream to. 1454 * pname:dstBufferRange is the maximum bitstream size in bytes that can: be 1455 written to pname:dstBuffer, starting from pname:dstBufferOffset. 1456 * pname:srcPictureResource is the video picture resource to use as the 1457 <<encode-input-picture,encode input picture>>. 1458 * pname:pSetupReferenceSlot is `NULL` or a pointer to a 1459 slink:VkVideoReferenceSlotInfoKHR structure specifying the 1460 <<encode-reconstructed-picture-info,reconstructed picture information>>. 1461 * pname:referenceSlotCount is the number of elements in the 1462 pname:pReferenceSlots array. 1463 * pname:pReferenceSlots is `NULL` or a pointer to an array of 1464 slink:VkVideoReferenceSlotInfoKHR structures describing the DPB slots 1465 and corresponding <<reference-picture,reference picture>> resources to 1466 use in this video encode operation (the set of 1467 <<active-reference-pictures, active reference pictures>>). 1468 * pname:precedingExternallyEncodedBytes is the number of bytes externally 1469 encoded by the application to the video bitstream and is used to update 1470 the internal state of the implementation's <<encode-rate-control,rate 1471 control>> algorithm to account for the bitrate budget consumed by these 1472 externally encoded bytes. 1473 1474.Valid Usage 1475**** 1476 * [[VUID-VkVideoEncodeInfoKHR-dstBuffer-08236]] 1477 pname:dstBuffer must: have been created with 1478 ename:VK_BUFFER_USAGE_VIDEO_ENCODE_DST_BIT_KHR set 1479 * [[VUID-VkVideoEncodeInfoKHR-dstBufferOffset-08237]] 1480 pname:dstBufferOffset must: be less than the size of pname:dstBuffer 1481 * [[VUID-VkVideoEncodeInfoKHR-dstBufferRange-08238]] 1482 pname:dstBufferRange must: be less than or equal to the size of 1483 pname:dstBuffer minus pname:dstBufferOffset 1484 * [[VUID-VkVideoEncodeInfoKHR-pSetupReferenceSlot-08239]] 1485 If pname:pSetupReferenceSlot is not `NULL`, then its pname:slotIndex 1486 member must: not be negative 1487 * [[VUID-VkVideoEncodeInfoKHR-pSetupReferenceSlot-08240]] 1488 If pname:pSetupReferenceSlot is not `NULL`, then its 1489 pname:pPictureResource must: not be `NULL` 1490 * [[VUID-VkVideoEncodeInfoKHR-slotIndex-08241]] 1491 The pname:slotIndex member of each element of pname:pReferenceSlots 1492 must: not be negative 1493 * [[VUID-VkVideoEncodeInfoKHR-pPictureResource-08242]] 1494 The pname:pPictureResource member of each element of 1495 pname:pReferenceSlots must: not be `NULL` 1496**** 1497 1498include::{generated}/validity/structs/VkVideoEncodeInfoKHR.adoc[] 1499-- 1500 1501[open,refpage='VkVideoEncodeFlagsKHR',desc='Reserved for future use',type='flags'] 1502-- 1503include::{generated}/api/flags/VkVideoEncodeFlagsKHR.adoc[] 1504 1505tlink:VkVideoEncodeFlagsKHR is a bitmask type for setting a mask, but is 1506currently reserved for future use. 1507-- 1508 1509 1510[[encode-rate-control]] 1511== Video Encode Rate Control 1512 1513The size of the encoded bitstream data produced by video encode operations 1514is a function of the following set of constraints: 1515 1516 * The capabilities of the compression algorithms defined and employed by 1517 the used video compression standard; 1518 * Restrictions imposed by the selected <<video-profiles,video profile>> 1519 according to the rules defined by the used video compression standard; 1520 * Further restrictions imposed by the <<video-coding-capabilities, 1521 capabilities>> supported by the implementation for the selected 1522 <<video-profiles,video profile>>; 1523 * The image data in the <<encode-input-picture,encode input picture>> and 1524 the set of <<active-reference-pictures,active reference pictures>> (as 1525 these affect the effectiveness of the compression algorithms employed by 1526 the video encode operations); 1527 * The set of codec-specific and codec-independent encoding parameters 1528 provided by the application. 1529 1530These also inherently define the set of decoder capabilities required for 1531reconstructing and processing the picture data in the encoded bitstream. 1532 1533[[encode-bitrate]] 1534Video coding uses _bitrate_ as the quantitative metric associated with 1535encoded bitstream data size which expresses the rate at which video 1536bitstream data can: be transferred or processed, measured in number of bits 1537per second. 1538This bitrate is both a function of the encoded bitstream data size of the 1539encoded pictures as well as the _frame rate_ used by the video sequence. 1540 1541Rate control algorithms are used by video encode operations to enable 1542adjusting encoding parameters to achieve a target bitrate, or otherwise 1543directly or indirectly control the bitrate of the generated video bitstream 1544data. 1545These algorithms are usually not defined by the used video compression 1546standard, although some video compression standards do provide non-normative 1547guidelines for implementations. 1548 1549Accordingly, this specification does not mandate implementations to produce 1550identical encoded bitstream data outputs in response to video encode 1551operations, however, it does define a set of codec-independent and 1552codec-specific parameters that enable the application to control the 1553behavior of the rate control algorithms supported by the implementation. 1554Some of these parameters guarantee certain implementation behavior while 1555others provide guidance for implementations to apply various rate control 1556heuristics. 1557 1558[NOTE] 1559.Note 1560==== 1561Applications need to make sure that they configure rate control parameters 1562appropriately and that they follow the promises made to the implementation 1563through parameters providing guidance for the implementation's rate control 1564algorithms and heuristics in order to be able to get the desired rate 1565control behavior and to be able to hit the set bitrate targets. 1566In addition, the behavior of rate control may also differ across 1567implementations even if the capabilities of the used video profile match 1568between those implementations. 1569This may happen due to implementations applying different rate control 1570algorithms or heuristics internally, and thus even the same set of guidance 1571parameter values may have different effects on the rate control behavior 1572across implementations. 1573==== 1574 1575 1576[[encode-rate-control-modes]] 1577=== Rate Control Modes 1578 1579After a video session is reset to the <<video-session-uninitialized,initial 1580state>>, the default behavior and parameters of video encode rate control 1581are entirely implementation-dependent and the application cannot: affect the 1582bitrate or quality parameters of the encoded bitstream data produced by 1583video encode operations unless the application changes the rate control 1584configuration of the video session, as described in the 1585<<video-coding-control,Video Coding Control>> section. 1586 1587For each supported <<video-profiles,video profile>>, the implementation may: 1588expose a set of _rate control modes_ that are available for use by the 1589application when encoding bitstreams targeting that video profile. 1590These modes allow using different rate control algorithms that fall into one 1591of the following two categories: 1592 1593 1. Per-operation rate control 1594 2. Stream-level rate control 1595 1596In case of _per-operation rate control_, the bitrate of the generated video 1597bitstream data is indirectly controlled by quality, size, or other encoding 1598parameters specified by the application for each individual video encode 1599operation. 1600 1601In case of _stream-level rate control_, the application can: directly 1602specify target bitrates besides other encoding parameters to control the 1603behavior of the rate control algorithm used by the implementation across 1604multiple video encode operations. 1605 1606[open,refpage='VkVideoEncodeRateControlModeFlagBitsKHR',desc='Video encode rate control modes',type='enums'] 1607-- 1608The rate control modes are defined with the following enums: 1609 1610include::{generated}/api/enums/VkVideoEncodeRateControlModeFlagBitsKHR.adoc[] 1611 1612 * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DEFAULT_KHR specifies the use of 1613 implementation-specific rate control. 1614 * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR specifies that 1615 rate control is disabled and the application will specify per-operation 1616 rate control parameters controlling the encoding quality. 1617 In this mode implementations will encode pictures independently of the 1618 output bitrate of prior video encode operations. 1619ifdef::VK_KHR_video_encode_h264[] 1620 ** When using an <<encode-h264-profile,H.264 encode profile>>, 1621 implementations will use the QP value specified in 1622 slink:VkVideoEncodeH264NaluSliceInfoKHR::pname:constantQp to control 1623 the quality of the encoded picture. 1624endif::VK_KHR_video_encode_h264[] 1625ifdef::VK_KHR_video_encode_h265[] 1626 ** When using an <<encode-h265-profile,H.265 encode profile>>, 1627 implementations will use the QP value specified in 1628 slink:VkVideoEncodeH265NaluSliceSegmentInfoKHR::pname:constantQp to 1629 control the quality of the encoded picture. 1630endif::VK_KHR_video_encode_h265[] 1631 * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR specifies the use of 1632 constant bitrate (CBR) rate control mode. 1633 In this mode the implementation will attempt to produce the encoded 1634 bitstream at a constant bitrate while conforming to the constraints of 1635 other rate control parameters. 1636 * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR specifies the use of 1637 variable bitrate (VBR) rate control mode. 1638 In this mode the implementation will produce the encoded bitstream at a 1639 variable bitrate according to the constraints of other rate control 1640 parameters. 1641-- 1642 1643[open,refpage='VkVideoEncodeRateControlModeFlagsKHR',desc='Bitmask of VkVideoEncodeRateControlModeFlagBitsKHR', type='flags'] 1644-- 1645include::{generated}/api/flags/VkVideoEncodeRateControlModeFlagsKHR.adoc[] 1646 1647tname:VkVideoEncodeRateControlModeFlagsKHR is a bitmask type for setting a 1648mask of zero or more elink:VkVideoEncodeRateControlModeFlagBitsKHR. 1649-- 1650 1651 1652[[encode-leaky-bucket-model]] 1653=== Leaky Bucket Model 1654 1655Video encoding implementations use the _leaky bucket model_ for stream-level 1656rate control. 1657The leaky bucket is a concept referring to the interface between the video 1658encoder and the consumer (for example, a network connection), where the 1659video encoder produces encoded bitstream data corresponding to the encoded 1660pictures and adds them in the leaky bucket while its content are drained by 1661the consumer. 1662 1663Analogously, a similar leaky bucket is considered to exist at the input 1664interface of a video decoder, into which encoded bitstream data is 1665continuously added and is subsequently consumed by the video decoder. 1666It is desirable to avoid overflowing or underflowing this leaky bucked 1667because: 1668 1669 * In case of an underflow, the video decoder will be unable to consume 1670 encoded bitstream data in order to decode pictures (and optionally 1671 display them). 1672 * In case of an overflow, the leaky bucket will be unable to accommodate 1673 more encoded bitstream data and such data may: need to be thrown away, 1674 leading to the loss of the corresponding encoded pictures. 1675 1676These requirements can: be satisfied by imposing various constraints on the 1677encoder-side leaky bucket to avoid its overflow or underflow, depending on 1678the used rate control algorithm and codec parameters. 1679However, enumerating these constraints is outside the scope of this 1680specification. 1681 1682The term _virtual buffer_ is often used as an alternative to refer to the 1683leaky bucket. 1684 1685This virtual buffer model is defined by the following parameters: 1686 1687 * The bitrate (`R`) at which the encoded bitstream is expected to be 1688 processed. 1689 * The size (`B`) of the virtual buffer. 1690 * The initial occupancy (`F`) of the virtual buffer. 1691 1692In this model the virtual buffer is used to smooth out fluctuations in the 1693bitrate of the encoded bitstream over time without experiencing buffer 1694overflow or underflow, as long as the bitrate of the encoded stream does not 1695diverge from the target bitrate for extended periods of time. 1696 1697This buffering may: inherently impose a processing delay, as the goal of the 1698model is to enable decoders maintain a consistent processing rate of an 1699encoded bitstream with varying data rate. 1700 1701The initial or start-up delay (`D`) is computed as: 1702 1703 {empty}:: [eq]#`D` = `F` / `R`# 1704 1705[NOTE] 1706.Note 1707==== 1708Applications need to configure the virtual buffer with sufficient size to 1709avoid or minimize buffer overflows and underflows while also keeping it 1710small enough to meet their latency goals. 1711==== 1712 1713 1714[[encode-rate-control-layers]] 1715=== Rate Control Layers 1716 1717Some video compression standards and <<video-profiles,video profiles>> allow 1718associating encoded pictures with specific _video coding layers_. 1719The name, identification, and semantics associated with such video coding 1720layers are defined by the corresponding video compression standards. 1721 1722Analogously, stream-level rate control can: be configured to use one or more 1723_rate control layers_: 1724 1725 * When a single rate control layer is configured, it is applied to all 1726 encoded pictures, regardless of the picture's video coding layer. 1727 In this case the distribution of the available bitrate budget across 1728 video coding layers is implementation-dependent. 1729 * When multiple rate control layers are configured, each rate control 1730 layer is applied to the corresponding video coding layer, i.e. only 1731 across encoded pictures pertaining to the corresponding video coding 1732 layer. 1733 1734Individual rate control layers are identified using _layer indices_ between 1735zero and `N-1`, where `N` is the number of active rate control layers. 1736 1737Rate control layers are only applicable when using 1738<<encode-rate-control-modes,stream-level rate control modes>>. 1739 1740 1741[[encode-rate-control-state]] 1742=== Rate Control State 1743 1744Rate control state is maintained by the implementation in the 1745<<video-session, video session>> objects and its parameters are specified 1746using an instance of the sname:VkVideoEncodeRateControlInfoKHR structure. 1747The complete rate control state of a video session is defined by the 1748following set of parameters: 1749 1750 * The values of the members of the slink:VkVideoEncodeRateControlInfoKHR 1751 structure used to configure the rate control state. 1752 * The values of the members of any 1753 slink:VkVideoEncodeRateControlLayerInfoKHR structures specified in 1754 slink:VkVideoEncodeRateControlInfoKHR::pname:pLayers used to configure 1755 the state of individual <<encode-rate-control-layers,rate control 1756 layers>>. 1757ifdef::VK_KHR_video_encode_h264[] 1758 * If the video session was created with an <<encode-h264-profile,H.264 1759 encode profile>>: 1760 ** The values of the members of the 1761 slink:VkVideoEncodeH264RateControlInfoKHR structure, if one is 1762 specified in the pname:pNext chain of the 1763 slink:VkVideoEncodeRateControlInfoKHR used to configure the rate 1764 control state. 1765 ** The values of the members of any 1766 slink:VkVideoEncodeH264RateControlLayerInfoKHR structures included in 1767 the pname:pNext chain of a slink:VkVideoEncodeRateControlLayerInfoKHR 1768 structure used to configure the state of a rate control layer. 1769endif::VK_KHR_video_encode_h264[] 1770ifdef::VK_KHR_video_encode_h265[] 1771 * If the video session was created with an <<encode-h265-profile,H.265 1772 encode profile>>: 1773 ** The values of the members of the 1774 slink:VkVideoEncodeH265RateControlInfoKHR structure, if one is 1775 specified in the pname:pNext chain of the 1776 slink:VkVideoEncodeRateControlInfoKHR used to configure the rate 1777 control state. 1778 ** The values of the members of any 1779 slink:VkVideoEncodeH265RateControlLayerInfoKHR structures included in 1780 the pname:pNext chain of a slink:VkVideoEncodeRateControlLayerInfoKHR 1781 structure used to configure the state of a rate control layer. 1782endif::VK_KHR_video_encode_h265[] 1783 1784[[encode-rate-control-state-matching]] 1785Two rate control states match if all the parameters listed above match 1786between them. 1787 1788[open,refpage='VkVideoEncodeRateControlInfoKHR',desc='Structure to set encode stream rate control parameters',type='structs'] 1789-- 1790The sname:VkVideoEncodeRateControlInfoKHR structure is defined as: 1791 1792include::{generated}/api/structs/VkVideoEncodeRateControlInfoKHR.adoc[] 1793 1794 * pname:sType is a elink:VkStructureType value identifying this structure. 1795 * pname:pNext is `NULL` or a pointer to a structure extending this 1796 structure. 1797 * pname:flags is reserved for future use. 1798 * pname:rateControlMode is a elink:VkVideoEncodeRateControlModeFlagBitsKHR 1799 value specifying the <<encode-rate-control-modes,rate control mode>>. 1800 * pname:layerCount specifies the number of <<encode-rate-control-layers, 1801 rate control layers>> to use. 1802 * pname:pLayers is a pointer to an array of pname:layerCount 1803 slink:VkVideoEncodeRateControlLayerInfoKHR structures, each specifying 1804 the rate control configuration of the corresponding rate control layer. 1805 * pname:virtualBufferSizeInMs is the size in milliseconds of the virtual 1806 buffer used by the implementation's rate control algorithm for the 1807 <<encode-leaky-bucket-model,leaky bucket model>>, with respect to the 1808 average bitrate of the stream calculated by summing the values of the 1809 pname:averageBitrate members of the elements of the pname:pLayers array. 1810 * pname:initialVirtualBufferSizeInMs is the initial occupancy in 1811 milliseconds of the virtual buffer used by the implementation's rate 1812 control algorithm for the <<encode-leaky-bucket-model,leaky bucket 1813 model>>. 1814 1815If pname:layerCount is zero then the values of pname:virtualBufferSizeInMs 1816and pname:initialVirtualBufferSizeInMs are ignored. 1817 1818This structure can: be specified in the following places: 1819 1820 * In the pname:pNext chain of slink:VkVideoBeginCodingInfoKHR to specify 1821 the current rate control state expected to be configured when beginning 1822 a <<video-coding-scope,video coding scope>>. 1823 * In the pname:pNext chain of slink:VkVideoCodingControlInfoKHR to change 1824 the rate control configuration of the bound video session. 1825 1826Including this structure in the pname:pNext chain of 1827slink:VkVideoCodingControlInfoKHR and including 1828ename:VK_VIDEO_CODING_CONTROL_ENCODE_RATE_CONTROL_BIT_KHR in 1829slink:VkVideoCodingControlInfoKHR::pname:flags enables updating the rate 1830control configuration of the bound video session. 1831This replaces the entire rate control configuration of the bound video 1832session and may: reset the state of all enabled rate control layers to an 1833initial state according to the codec-specific rate control semantics defined 1834in the corresponding sections listed below. 1835 1836When pname:layerCount is greater than one, multiple 1837<<encode-rate-control-layers,rate control layers>> are configured, and each 1838rate control layer is applied to the corresponding video coding layer 1839identified by the index of the corresponding element of pname:pLayer. 1840 1841ifdef::VK_KHR_video_encode_h264[] 1842 * If the video session was created with the video codec operation 1843 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then this index 1844 specifies the H.264 temporal layer ID of the video coding layer the rate 1845 control layer is applied to. 1846endif::VK_KHR_video_encode_h264[] 1847ifdef::VK_KHR_video_encode_h265[] 1848 * If the video session was created with the video codec operation 1849 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, then this index 1850 specifies the H.265 temporal ID of the video coding layer the rate 1851 control layer is applied to. 1852endif::VK_KHR_video_encode_h265[] 1853 1854Additional structures providing codec-specific rate control parameters can: 1855be included in the pname:pNext chain of sname:VkVideoCodingControlInfoKHR 1856depending on the <<video-profiles,video profile>> the bound video session 1857was created. 1858For further details see: 1859 1860 * <<video-coding-control,Video Coding Control>> 1861ifdef::VK_KHR_video_encode_h264[] 1862 * <<encode-h264-rate-control,H.264 Encode Rate Control>> 1863endif::VK_KHR_video_encode_h264[] 1864ifdef::VK_KHR_video_encode_h265[] 1865 * <<encode-h265-rate-control,H.265 Encode Rate Control>> 1866endif::VK_KHR_video_encode_h265[] 1867 1868The new rate control configuration takes effect when the corresponding 1869flink:vkCmdControlVideoCodingKHR is executed on the device, and only impacts 1870video encode operations that follow in execution order. 1871 1872.Valid Usage 1873**** 1874 * [[VUID-VkVideoEncodeRateControlInfoKHR-rateControlMode-08248]] 1875 If pname:rateControlMode is 1876 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DEFAULT_KHR or 1877 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR, then 1878 pname:layerCount must: be `0` 1879 * [[VUID-VkVideoEncodeRateControlInfoKHR-rateControlMode-08275]] 1880 If pname:rateControlMode is 1881 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR or 1882 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR, then 1883 pname:layerCount must: be greater than `0` 1884 * [[VUID-VkVideoEncodeRateControlInfoKHR-rateControlMode-08244]] 1885 If pname:rateControlMode is not 1886 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DEFAULT_KHR, then it must: 1887 specify one of the bits included in 1888 slink:VkVideoEncodeCapabilitiesKHR::pname:rateControlModes, as returned 1889 by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video 1890 profile 1891 * [[VUID-VkVideoEncodeRateControlInfoKHR-layerCount-08245]] 1892 pname:layerCount member must: be less than or equal to 1893 slink:VkVideoEncodeCapabilitiesKHR::pname:maxRateControlLayers, as 1894 returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used 1895 video profile 1896 * [[VUID-VkVideoEncodeRateControlInfoKHR-pLayers-08276]] 1897 For each element of pname:pLayers, its pname:averageBitrate member must: 1898 be between `1` and slink:VkVideoEncodeCapabilitiesKHR::pname:maxBitrate, 1899 as returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the 1900 used video profile 1901 * [[VUID-VkVideoEncodeRateControlInfoKHR-pLayers-08277]] 1902 For each element of pname:pLayers, its pname:maxBitrate member must: be 1903 between `1` and slink:VkVideoEncodeCapabilitiesKHR::pname:maxBitrate, as 1904 returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used 1905 video profile 1906 * [[VUID-VkVideoEncodeRateControlInfoKHR-rateControlMode-08356]] 1907 If pname:rateControlMode is 1908 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR, then for each 1909 element of pname:pLayers, its pname:averageBitrate member must: equal 1910 its pname:maxBitrate member 1911 * [[VUID-VkVideoEncodeRateControlInfoKHR-rateControlMode-08278]] 1912 If pname:rateControlMode is 1913 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR, then for each 1914 element of pname:pLayers, its pname:averageBitrate member must: be less 1915 than or equal to its pname:maxBitrate member 1916 * [[VUID-VkVideoEncodeRateControlInfoKHR-layerCount-08357]] 1917 If pname:layerCount is not zero, then pname:virtualBufferSizeInMs must: 1918 be greater than zero 1919 * [[VUID-VkVideoEncodeRateControlInfoKHR-layerCount-08358]] 1920 If pname:layerCount is not zero, then pname:initialVirtualBufferSizeInMs 1921 must: be less than pname:virtualBufferSizeInMs 1922ifdef::VK_KHR_video_encode_h264[] 1923 * [[VUID-VkVideoEncodeRateControlInfoKHR-videoCodecOperation-07022]] 1924 If the pname:videoCodecOperation of the used video profile is 1925 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, the pname:pNext 1926 chain this structure is included in also includes an instance of the 1927 slink:VkVideoEncodeH264RateControlInfoKHR structure, and 1928 pname:layerCount is greater than `1`, then pname:layerCount must: equal 1929 slink:VkVideoEncodeH264RateControlInfoKHR::pname:temporalLayerCount 1930endif::VK_KHR_video_encode_h264[] 1931ifdef::VK_KHR_video_encode_h265[] 1932 * [[VUID-VkVideoEncodeRateControlInfoKHR-videoCodecOperation-07025]] 1933 If the pname:videoCodecOperation of the used video profile is 1934 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, the pname:pNext 1935 chain this structure is included in also includes an instance of the 1936 slink:VkVideoEncodeH265RateControlInfoKHR structure, and 1937 pname:layerCount is greater than `1`, then pname:layerCount must: equal 1938 slink:VkVideoEncodeH265RateControlInfoKHR::pname:subLayerCount 1939endif::VK_KHR_video_encode_h265[] 1940**** 1941 1942include::{generated}/validity/structs/VkVideoEncodeRateControlInfoKHR.adoc[] 1943-- 1944 1945[open,refpage='VkVideoEncodeRateControlFlagsKHR',desc='Reserved for future use',type='flags'] 1946-- 1947include::{generated}/api/flags/VkVideoEncodeRateControlFlagsKHR.adoc[] 1948 1949tname:VkVideoEncodeRateControlFlagsKHR is a bitmask type for setting a mask, 1950but currently reserved for future use. 1951-- 1952 1953 1954==== Rate Control Layer State 1955 1956The configuration of individual rate control layers is specified using an 1957instance of the sname:VkVideoEncodeRateControlLayerInfoKHR structure. 1958 1959[open,refpage='VkVideoEncodeRateControlLayerInfoKHR',desc='Structure to set encode per-layer rate control parameters',type='structs'] 1960-- 1961The sname:VkVideoEncodeRateControlLayerInfoKHR structure is defined as: 1962 1963include::{generated}/api/structs/VkVideoEncodeRateControlLayerInfoKHR.adoc[] 1964 1965 * pname:sType is a elink:VkStructureType value identifying this structure. 1966 * pname:pNext is a pointer to a structure extending this structure. 1967 * pname:averageBitrate is the average <<encode-bitrate,bitrate>> to be 1968 targeted by the implementation's rate control algorithm. 1969 * pname:maxBitrate is the peak <<encode-bitrate,bitrate>> to be targeted 1970 by the implementation's rate control algorithm. 1971 * pname:frameRateNumerator is the numerator of the frame rate assumed by 1972 the implementation's rate control algorithm. 1973 * pname:frameRateDenominator is the denominator of the frame rate assumed 1974 by the implementation's rate control algorithm. 1975 1976[NOTE] 1977.Note 1978==== 1979The ability of the implementation's rate control algorithm to be able to 1980match the requested average and/or peak bitrates may: be limited by the set 1981of other codec-independent and codec-specific rate control parameters 1982specified by the application, the input content, as well as the application 1983conforming to the rate control guidance provided to the implementation, as 1984described <<encode-rate-control,earlier>>. 1985==== 1986 1987Additional structures providing codec-specific rate control parameters can: 1988be included in the pname:pNext chain of 1989sname:VkVideoEncodeRateControlLayerInfoKHR depending on the 1990<<video-profiles,video profile>> the bound video session was created with. 1991For further details see: 1992 1993 * <<video-coding-control,Video Coding Control>> 1994ifdef::VK_KHR_video_encode_h264[] 1995 * <<encode-h264-rate-control,H.264 Encode Rate Control>> 1996endif::VK_KHR_video_encode_h264[] 1997ifdef::VK_KHR_video_encode_h265[] 1998 * <<encode-h265-rate-control,H.265 Encode Rate Control>> 1999endif::VK_KHR_video_encode_h265[] 2000 2001.Valid Usage 2002**** 2003 * [[VUID-VkVideoEncodeRateControlLayerInfoKHR-frameRateNumerator-08350]] 2004 pname:frameRateNumerator must: be greater than zero 2005 * [[VUID-VkVideoEncodeRateControlLayerInfoKHR-frameRateDenominator-08351]] 2006 pname:frameRateDenominator must: be greater than zero 2007**** 2008 2009include::{generated}/validity/structs/VkVideoEncodeRateControlLayerInfoKHR.adoc[] 2010-- 2011