1// Copyright 2018-2023 The Khronos Group Inc. 2// 3// SPDX-License-Identifier: CC-BY-4.0 4 5[[video-encode-operations]] 6== Video Encode Operations 7 8Before the application can start recording Vulkan command buffers for the 9Video Encode Operations, it must: do the following, beforehand: 10 11 . Ensure that the implementation can encode the Video Content by querying 12 the supported codec operations and profiles using 13 flink:vkGetPhysicalDeviceQueueFamilyProperties2. 14 . By using flink:vkGetPhysicalDeviceVideoFormatPropertiesKHR and providing 15 one or more video profiles, choose the Vulkan formats supported by the 16 implementation. 17 The formats for <<encode-input-picture,input>> and 18 <<reference-picture,reference>> pictures must: be queried and chosen 19 separately. 20 Refer to the section on <<video-format-capabilities,Video Format 21 Capabilities>>. 22 . Before creating an image to be used as a video picture resource, obtain 23 the supported image creation parameters by querying with 24 flink:vkGetPhysicalDeviceFormatProperties2 and 25 flink:vkGetPhysicalDeviceImageFormatProperties2 using one of the 26 reported formats and adding slink:VkVideoProfileListInfoKHR to the 27 pname:pNext chain of slink:VkFormatProperties2. 28 When querying the parameters with 29 flink:vkGetPhysicalDeviceImageFormatProperties2 for images targeting 30 <<encode-input-picture,input>> and <<reference-picture,reference (DPB)>> 31 pictures, the slink:VkPhysicalDeviceImageFormatInfo2::pname:usage field 32 should contain ename:VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR and 33 ename:VK_IMAGE_USAGE_VIDEO_ENCODE_DPB_BIT_KHR, respectively. 34 . Create none, some, or all of the required <<VkImage,images>> for the 35 <<encode-input-picture,input>> and <<reference-picture,reference>> 36 pictures. 37 More Video Picture Resources can: be created at some later point if 38 needed while processing the content to be encoded. 39 Also, if the size of the picture to be encoded is expected to change, 40 the images can: be created based on the maximum expected content size. 41 . Create the <<video-session,video session>> to be used for video encode 42 operations. 43 Before creating the Encode Video Session, the encode capabilities 44 should: be queried with flink:vkGetPhysicalDeviceVideoCapabilitiesKHR to 45 obtain the limits of the parameters allowed by the implementation for a 46 particular codec profile. 47 . Bind memory resources with the encode video session by calling 48 flink:vkBindVideoSessionMemoryKHR. 49 The video session cannot: be used until memory resources are allocated 50 and bound to it. 51 In order to determine the required memory sizes and heap types of the 52 device memory allocations, flink:vkGetVideoSessionMemoryRequirementsKHR 53 should: be called. 54 . Create one or more <<video-session-parameters,Video Session Parameter 55 objects>> for use across command buffer recording operations, if 56 required by the codec extension in use. 57 These objects must: be created against a <<video-session,video session>> 58 with the parameters required by the codec. 59 Each <<video-session-parameters,Video Session Parameter object>> created 60 is a child object of the associated <<video-session, Session object>> 61 and cannot: be bound in the command buffer with any other 62 <<video-session,Session Object>>. 63 64 65The recording of Video Encode Commands against a Vulkan Command Buffer 66consists of the following sequence: 67 68 . flink:vkCmdBeginVideoCodingKHR starts the recording of one or more Video 69 Encode operations in the command buffer. 70 For each Video Encode Command operation, a Video Session must: be bound 71 to the command buffer within this command. 72 This command establishes a Vulkan Video Encode Context that consists of 73 the bound Video Session Object, Session Parameters Object, and the 74 required Video Picture Resources. 75 The established Video Encode Context is in effect until the 76 flink:vkCmdEndVideoCodingKHR command is recorded. 77 If more Video Encode operations are to be required after the 78 flink:vkCmdEndVideoCodingKHR command, another Video Encode Context can: 79 be started with the flink:vkCmdBeginVideoCodingKHR command. 80 . flink:vkCmdEncodeVideoKHR specifies one or more frames to be encoded. 81 The slink:VkVideoEncodeInfoKHR parameters, and the codec extension 82 structures chained to this, specify the details of the encode operation. 83 . flink:vkCmdControlVideoCodingKHR records operations against the encoded 84 data, encoding device, or the Video Session state. 85 . flink:vkCmdEndVideoCodingKHR signals the end of the recording of the 86 Vulkan Video Encode Context, as established by 87 flink:vkCmdBeginVideoCodingKHR. 88 89In addition to the above, the following commands can: be recorded between 90flink:vkCmdBeginVideoCodingKHR and flink:vkCmdEndVideoCodingKHR: 91 92 * Query operations 93 * Global Memory Barriers 94 * Buffer Memory Barriers 95 * Image Memory Barriers (these must: be used to transition the Video 96 Picture Resources to the proper 97 ename:VK_IMAGE_LAYOUT_VIDEO_ENCODE_SRC_KHR and 98 ename:VK_IMAGE_LAYOUT_VIDEO_ENCODE_DPB_KHR layouts). 99 * Pipeline Barriers 100 * Events 101 * Timestamps 102 * Device Groups (device mask) 103 104The following Video Encode related commands must: be recorded *outside* the 105Vulkan Video Encode Context established with the 106flink:vkCmdBeginVideoCodingKHR and flink:vkCmdEndVideoCodingKHR commands: 107 108 * Sparse Memory Binding 109 * Copy Commands 110 * Clear Commands 111 112 113[[encode-input-picture]] 114=== Encode Input Picture 115 116The primary source of input pixels for the video encoding process is the 117_Encode Input Picture_, represented by a slink:VkImageView. 118It may: also be a direct target of 119ifdef::VK_KHR_video_decode_queue[] 120video decode, 121endif::VK_KHR_video_decode_queue[] 122graphics, or compute operations 123ifdef::VK_KHR_surface[] 124, or with <<wsi, Window System Integration>> APIs 125endif::VK_KHR_surface[] 126. 127 128 129=== Capabilities 130 131[open,refpage='VkVideoEncodeCapabilitiesKHR',desc='Structure specifying encode capabilities',type='structs'] 132-- 133When calling flink:vkGetPhysicalDeviceVideoCapabilitiesKHR with 134pname:pVideoProfile->videoCodecOperation specified as one of the encode 135operation bits, the slink:VkVideoEncodeCapabilitiesKHR structure must: be 136included in the pname:pNext chain of the slink:VkVideoCapabilitiesKHR 137structure to retrieve capabilities specific to video encoding. 138 139The sname:VkVideoEncodeCapabilitiesKHR structure is defined as: 140 141include::{generated}/api/structs/VkVideoEncodeCapabilitiesKHR.adoc[] 142 143 * pname:sType is a elink:VkStructureType value identifying this structure. 144 * pname:pNext is `NULL` or a pointer to a structure extending this 145 structure. 146 * pname:flags is a bitmask of elink:VkVideoEncodeCapabilityFlagBitsKHR 147 describing supported encoding features. 148 * pname:rateControlModes is a bitmask of 149 elink:VkVideoEncodeRateControlModeFlagBitsKHR indicating supported rate 150 control modes. 151 * pname:maxRateControlLayers indicates the maximum number of rate control 152 layers supported. 153 * pname:maxBitrate indicates the maximum supported bitrate. 154 * pname:maxQualityLevels indicates the number of discrete video encode 155 quality levels supported. 156 Implementations must: report at least 1. 157 * pname:encodeInputPictureGranularity indicates the granularity at which 158 <<encode-input-picture,encode input picture>> data is encoded. 159 * pname:supportedEncodeFeedbackFlags is a bitmask of 160 elink:VkVideoEncodeFeedbackFlagBitsKHR values specifying the supported 161 flags for <<queries-video-encode-feedback,video encode feedback 162 queries>>. 163 164Implementations must: include support for at least 165ename:VK_VIDEO_ENCODE_FEEDBACK_BITSTREAM_BUFFER_OFFSET_BIT_KHR and 166ename:VK_VIDEO_ENCODE_FEEDBACK_BITSTREAM_BYTES_WRITTEN_BIT_KHR in 167pname:supportedEncodeFeedbackFlags. 168 169The input content and encode resolution (specified in 170slink:VkVideoEncodeInfoKHR::pname:codedExtent) may not be aligned with the 171codec-specific coding block size. 172For example, the input content may be 1920x1080 and the coding block size 173may be 16x16 pixel blocks. 174In this example, the content is horizontally aligned with the coding block 175size, but not vertically aligned with the coding block size. 176Encoding of the last row of blocks may be impacted by contents of the input 177image in pixel rows 1081 to 1088 (the next vertical alignment with the 178coding block size). 179In general, to ensure efficient encoding for the last row/column of blocks, 180and/or to ensure consistent encoding results between repeated encoding of 181the same input content, these extra pixel rows/columns should be filled to 182known values up to the coding block size alignment before encoding 183operations are performed. 184Some implementations support performing auto-fill of unaligned pixels beyond 185a specific alignment for the purposes of encoding, which is reported in 186pname:encodeInputPictureGranularity. 187For example, if an implementation reports 1x1 in 188pname:encodeInputPictureGranularity, then the implementation will perform 189auto-fill for any unaligned pixels beyond the encode resolution up to the 190next coding block size. 191For a coding block size of 16x16, if the implementation reports 16x16 in 192pname:encodeInputPictureGranularity, then it is the application's 193responsibility to fill any unaligned pixels, if desired. 194When the application does not fill these unaligned pixels, there may: be an 195impact on the encoding efficiency but there will be no effect on the 196validity of the generated bitstream. 197If the implementation reports 8x8 in pname:encodeInputPictureGranularity, 198then for the 1920x1080 example, since the content is aligned to 8 pixels 199vertically, the implementation will auto-fill pixel rows 1081 to 1088 (up to 200the 16x16 coding block size in the example). 201The auto-fill value(s) are implementation-specific. 202The auto-fill value(s) are not written to the input image memory, but are 203used as part of the encoding operation on the input image. 204 205include::{generated}/validity/structs/VkVideoEncodeCapabilitiesKHR.adoc[] 206-- 207 208[open,refpage='VkVideoEncodeCapabilityFlagsKHR',desc='Bitmask of VkVideoEncodeCapabilityFlagBitsKHR',type='flags'] 209-- 210include::{generated}/api/flags/VkVideoEncodeCapabilityFlagsKHR.adoc[] 211 212tname:VkVideoEncodeCapabilityFlagsKHR is a bitmask type for setting a mask 213of zero or more elink:VkVideoEncodeCapabilityFlagBitsKHR. 214-- 215 216[open,refpage='VkVideoEncodeCapabilityFlagBitsKHR',desc='Video encode capability flags',type='enums'] 217-- 218Bits which may: be set in slink:VkVideoEncodeCapabilitiesKHR::pname:flags, 219indicating the encoding tools supported, are: 220 221include::{generated}/api/enums/VkVideoEncodeCapabilityFlagBitsKHR.adoc[] 222 223 * ename:VK_VIDEO_ENCODE_CAPABILITY_PRECEDING_EXTERNALLY_ENCODED_BYTES_BIT_KHR 224 indicates that the implementation supports the use of 225 slink:VkVideoEncodeInfoKHR::pname:precedingExternallyEncodedBytes. 226 * ename:VK_VIDEO_ENCODE_CAPABILITY_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_DETECTION_BIT_KHR 227 indicates that the implementation is able to detect and report when the 228 destination video bitstream buffer range provided by the application is 229 not sufficiently large to fit the encoded bitstream data produced by a 230 video encode operation by reporting the 231 ename:VK_QUERY_RESULT_STATUS_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_KHR 232 <<query-result-status-codes,query result status code>>. 233+ 234[NOTE] 235.Note 236==== 237Some implementations may: not be able to reliably detect insufficient 238bitstream buffer range conditions in all situations. 239Such implementations will not report support for the 240ename:VK_VIDEO_ENCODE_CAPABILITY_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_DETECTION_BIT_KHR 241encode capability flag for the video profile, but may: still report the 242ename:VK_QUERY_RESULT_STATUS_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_KHR query 243result status code in certain cases. 244Applications should: always check for the specific query result status code 245ename:VK_QUERY_RESULT_STATUS_INSUFFICIENT_BITSTREAM_BUFFER_RANGE_KHR even 246when this encode capability flag is not supported by the implementation for 247the video profile in question. 248However, applications must: not assume that a different negative query 249result status code indicating an unsuccessful completion of a video encode 250operation is not the result of an insufficient bitstream buffer condition 251unless this encode capability flag is supported. 252==== 253-- 254 255 256=== Video Encode Quality Levels 257 258[open,refpage='vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR',desc='Query video encode quality level properties',type='protos'] 259-- 260To query properties for a specific video encode quality level supported by a 261video encode profile, call: 262 263include::{generated}/api/protos/vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR.adoc[] 264 265 * pname:physicalDevice is the physical device to query the video encode 266 quality level properties for. 267 * pname:pQualityLevelInfo is a pointer to a 268 slink:VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR structure 269 specifying the video encode profile and quality level to query 270 properties for. 271 * pname:pQualityLevelProperties is a pointer to a 272 slink:VkVideoEncodeQualityLevelPropertiesKHR structure in which the 273 properties are returned. 274 275include::{generated}/validity/protos/vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR.adoc[] 276-- 277 278 279[open,refpage='VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR',desc='Structure describing the video encode profile and quality level to query properties for',type='structs'] 280-- 281The sname:VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR structure is 282defined as: 283 284include::{generated}/api/structs/VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR.adoc[] 285 286 * pname:sType is a elink:VkStructureType value identifying this structure. 287 * pname:pNext is `NULL` or a pointer to a structure extending this 288 structure. 289 * pname:pVideoProfile is a pointer to a slink:VkVideoProfileInfoKHR 290 structure specifying the video profile to query the video encode quality 291 level properties for. 292 * pname:qualityLevel is the video encode quality level to query properties 293 for. 294 295include::{generated}/validity/structs/VkPhysicalDeviceVideoEncodeQualityLevelInfoKHR.adoc[] 296-- 297 298 299[open,refpage='VkVideoEncodeQualityLevelPropertiesKHR',desc='Structure describing the video encode quality level properties',type='structs'] 300-- 301The sname:VkVideoEncodeQualityLevelPropertiesKHR structure is defined as: 302 303include::{generated}/api/structs/VkVideoEncodeQualityLevelPropertiesKHR.adoc[] 304 305 * pname:sType is a elink:VkStructureType value identifying this structure. 306 * pname:pNext is `NULL` or a pointer to a structure extending this 307 structure. 308 * pname:preferredRateControlMode is a 309 elink:VkVideoEncodeRateControlModeFlagBitsKHR value indicating the 310 preferred rate control mode to use with the video encode quality level. 311 * pname:preferredRateControlLayerCount indicates the preferred number of 312 rate control layers to use with the quality level. 313 314include::{generated}/validity/structs/VkVideoEncodeQualityLevelPropertiesKHR.adoc[] 315-- 316 317 318[open,refpage='VkVideoEncodeQualityLevelInfoKHR',desc='Structure specifying used video encode quality level',type='structs'] 319-- 320The sname:VkVideoEncodeQualityLevelInfoKHR structure is defined as: 321 322include::{generated}/api/structs/VkVideoEncodeQualityLevelInfoKHR.adoc[] 323 324 * pname:sType is a elink:VkStructureType value identifying this structure. 325 * pname:pNext is `NULL` or a pointer to a structure extending this 326 structure. 327 * pname:qualityLevel is the used video encode quality level. 328 329This structure can: be specified in the following places: 330 331 * In the pname:pNext chain of slink:VkVideoSessionParametersCreateInfoKHR 332 to specify the video encode quality level to use for a video session 333 parameters object created for a video encode session. 334 If no instance of this structure is included in the pname:pNext chain of 335 slink:VkVideoSessionParametersCreateInfoKHR, then the video session 336 parameters object is created with a video encode quality level of zero. 337 * In the pname:pNext chain of slink:VkVideoCodingControlInfoKHR to change 338 the video encode quality level state of the bound video session. 339 340include::{generated}/validity/structs/VkVideoEncodeQualityLevelInfoKHR.adoc[] 341-- 342 343 344=== Retrieving Encoded Session Parameters 345 346Any codec-specific parameters stored in video session parameters objects 347may: need to be separately encoded and included in the final video bitstream 348data, depending on the used video compression standard. 349In such cases the application must: call the 350flink:vkGetEncodedVideoSessionParametersKHR command to retrieve the encoded 351parameter data from the used video session parameters object in order to be 352able to produce a compliant video bitstream. 353 354[open,refpage='vkGetEncodedVideoSessionParametersKHR',desc='Get encoded parameter sets from a video session parameters object',type='protos'] 355-- 356Encoded parameter data can: be retrieved from a video session parameters 357object created with a video encode operation using the command: 358 359include::{generated}/api/protos/vkGetEncodedVideoSessionParametersKHR.adoc[] 360 361 * pname:device is the logical device that owns the video session 362 parameters object. 363 * pname:pVideoSessionParametersInfo is a pointer to a 364 slink:VkVideoEncodeSessionParametersGetInfoKHR structure specifying the 365 parameters of the encoded parameter data to retrieve. 366 * pname:pFeedbackInfo is either `NULL` or a pointer to a 367 slink:VkVideoEncodeSessionParametersFeedbackInfoKHR structure in which 368 feedback about the requested parameter data is returned. 369 * pname:pDataSize is a pointer to a code:size_t value related to the 370 amount of encode parameter data returned, as described below. 371 * pname:pData is either `NULL` or a pointer to a buffer to write the 372 encoded parameter data to. 373 374If pname:pData is `NULL`, then the size of the encoded parameter data, in 375bytes, that can: be retrieved is returned in pname:pDataSize. 376Otherwise, pname:pDataSize must: point to a variable set by the application 377to the size of the buffer, in bytes, pointed to by pname:pData, and on 378return the variable is overwritten with the number of bytes actually written 379to pname:pData. 380If pname:pDataSize is less than the size of the encoded parameter data that 381can: be retrieved, then no data will be written to pname:pData, zero will be 382written to pname:pDataSize, and ename:VK_INCOMPLETE will be returned instead 383of ename:VK_SUCCESS, to indicate that no encoded parameter data was 384returned. 385 386If pname:pFeedbackInfo is not `NULL` then the members of the 387slink:VkVideoEncodeSessionParametersFeedbackInfoKHR structure and any 388additional structures included in its pname:pNext chain that are applicable 389to the video session parameters object specified in 390pname:pVideoSessionParametersInfo::pname:videoSessionParameters will be 391filled with feedback about the requested parameter data on all successful 392calls to this command. 393 394[NOTE] 395.Note: 396==== 397This includes the cases when pname:pData is `NULL` or when 398ename:VK_INCOMPLETE is returned by the command, and enables the application 399to determine whether the implementation overrode any of the requested video 400session parameters without actually needing to retrieve the encoded 401parameter data itself. 402==== 403 404include::{generated}/validity/protos/vkGetEncodedVideoSessionParametersKHR.adoc[] 405-- 406 407[open,refpage='VkVideoEncodeSessionParametersGetInfoKHR',desc='Structure specifying parameters for retrieving encoded video session parameter data',type='structs'] 408-- 409The sname:VkVideoEncodeSessionParametersGetInfoKHR structure is defined as: 410 411include::{generated}/api/structs/VkVideoEncodeSessionParametersGetInfoKHR.adoc[] 412 413 * pname:sType is a elink:VkStructureType value identifying this structure. 414 * pname:pNext is `NULL` or a pointer to a structure extending this 415 structure. 416 * pname:videoSessionParameters is the slink:VkVideoSessionParametersKHR 417 object to retrieve encoded parameter data from. 418 419Depending on the used video encode operation, additional codec-specific 420structures may: need to be included in the pname:pNext chain of this 421structure to identify the specific video session parameters to retrieve 422encoded parameter data for, as described in the corresponding sections. 423 424include::{generated}/validity/structs/VkVideoEncodeSessionParametersGetInfoKHR.adoc[] 425-- 426 427[open,refpage='VkVideoEncodeSessionParametersFeedbackInfoKHR',desc='Structure providing feedback about the requested video session parameters',type='structs'] 428-- 429The sname:VkVideoEncodeSessionParametersFeedbackInfoKHR structure is defined 430as: 431 432include::{generated}/api/structs/VkVideoEncodeSessionParametersFeedbackInfoKHR.adoc[] 433 434 * pname:sType is a elink:VkStructureType value identifying this structure. 435 * pname:pNext is `NULL` or a pointer to a structure extending this 436 structure. 437 * pname:hasOverrides indicates whether any of the requested parameter data 438 were overridden by the implementation. 439 440Depending on the used video encode operation, additional codec-specific 441structures may: need to be included in the pname:pNext chain of this 442structure to capture feedback information about the requested parameter 443data, as described in the corresponding sections. 444 445include::{generated}/validity/structs/VkVideoEncodeSessionParametersFeedbackInfoKHR.adoc[] 446-- 447 448 449=== Video Encode Commands 450 451[open,refpage='vkCmdEncodeVideoKHR',desc='Encode operation for bitstream generation',type='protos'] 452-- 453To launch video encode operations, call: 454 455include::{generated}/api/protos/vkCmdEncodeVideoKHR.adoc[] 456 457 * pname:commandBuffer is the command buffer to be filled with this 458 function for encoding to generate a bitstream. 459 * pname:pEncodeInfo is a pointer to a slink:VkVideoEncodeInfoKHR 460 structure. 461 462Each call issues one or more video encode operations. 463The implicit parameter pname:opCount corresponds to the number of video 464encode operations issued by the command. 465After calling this command, the 466<<queries-operation-active-query-index,active query index>> of each 467<<queries-operation-active,active>> query is incremented by pname:opCount. 468 469Currently each call to this command results in the issue of a single video 470encode operation. 471 472.Valid Usage 473**** 474 * [[VUID-vkCmdEncodeVideoKHR-opCount-07174]] 475 For each <<queries-operation-active,active>> query, the 476 <<queries-operation-active-query-index,active query index>> 477 corresponding to the query type of that query plus pname:opCount must: 478 be less than or equal to the 479 <<queries-operation-last-activatable-query-index,last activatable query 480 index>> corresponding to the query type of that query plus one 481**** 482 483include::{generated}/validity/protos/vkCmdEncodeVideoKHR.adoc[] 484-- 485 486[open,refpage='VkVideoEncodeInfoKHR',desc='Structure to chain codec-specific structures to',type='structs'] 487-- 488The sname:VkVideoEncodeInfoKHR structure is defined as: 489 490include::{generated}/api/structs/VkVideoEncodeInfoKHR.adoc[] 491 492 * pname:sType is a elink:VkStructureType value identifying this structure. 493 * pname:pNext is a pointer to a structure extending this structure. 494 A codec-specific extension structure must: be chained to specify what 495 bitstream unit to generate with this encode operation. 496 * pname:flags is reserved for future use. 497 * pname:dstBuffer is the destination video bitstream buffer to write the 498 encoded bitstream to. 499 * pname:dstBufferOffset is the starting offset in bytes from the start of 500 pname:dstBuffer to write the encoded bitstream to. 501 pname:dstBufferOffset's value must: be aligned to 502 slink:VkVideoCapabilitiesKHR::pname:minBitstreamBufferOffsetAlignment, 503 as reported by the implementation. 504 * pname:dstBufferRange is the maximum size in bytes of the encoded 505 bitstream written to pname:dstBuffer, starting from 506 pname:dstBufferOffset. 507 pname:dstBufferRange's value must: be aligned to 508 slink:VkVideoCapabilitiesKHR::pname:minBitstreamBufferSizeAlignment, as 509 reported by the implementation. 510 * pname:srcPictureResource is the Picture Resource of the 511 <<encode-input-picture,Input Picture>> to be encoded by the operation. 512 * pname:pSetupReferenceSlot is a pointer to a 513 slink:VkVideoReferenceSlotInfoKHR structure used for generating a 514 reconstructed reference slot and Picture Resource. 515 pname:pSetupReferenceSlot->slotIndex specifies the slot index number to 516 use as a target for producing the Reconstructed (DPB) data. 517 pname:pSetupReferenceSlot must: be one of the entries provided in 518 slink:VkVideoBeginCodingInfoKHR via the pname:pReferenceSlots within the 519 flink:vkCmdBeginVideoCodingKHR command that established the Vulkan Video 520 Encode Context for this command. 521 * pname:referenceSlotCount is the number of Reconstructed Reference 522 Pictures that will be used when this encoding operation is executing. 523 * pname:pReferenceSlots is `NULL` or a pointer to an array of 524 slink:VkVideoReferenceSlotInfoKHR structures that will be used when this 525 encoding operation is executing. 526 Each entry in pname:pReferenceSlots must: be one of the entries provided 527 in slink:VkVideoBeginCodingInfoKHR via the pname:pReferenceSlots within 528 the flink:vkCmdBeginVideoCodingKHR command that established the Vulkan 529 Video Encode Context for this command. 530 * pname:precedingExternallyEncodedBytes is the number of bytes externally 531 encoded for insertion in the active video encode session overall 532 bitstream prior to the bitstream that will be generated by the 533 implementation for this instance of sname:VkVideoEncodeInfoKHR. 534 The value provided is used to update the implementation's rate control 535 algorithm for the rate control layer this instance of 536 sname:VkVideoEncodeInfoKHR belongs to, by accounting for the bitrate 537 budget consumed by these externally encoded bytes. 538 See slink:VkVideoEncodeRateControlInfoKHR for additional information 539 about encode rate control. 540 541The coded size of the encode operation is specified in pname:codedExtent of 542pname:srcPictureResource. 543 544Multiple flink:vkCmdEncodeVideoKHR commands may: be recorded within a Vulkan 545Video Encode Context. 546The execution of each flink:vkCmdEncodeVideoKHR command will result in 547generating codec-specific bitstream units. 548These bitstream units are generated consecutively into the bitstream buffer 549specified in pname:dstBuffer of a sname:VkVideoEncodeInfoKHR structure 550within the flink:vkCmdBeginVideoCodingKHR command. 551The produced bitstream is the sum of all these bitstream units, including 552any padding between the bitstream units. 553Any bitstream padding must: be filled with data compliant to the codec 554standard so as not to cause any syntax errors during decoding of the 555bitstream units with the padding included. 556The range of the bitstream buffer written can: be queried via 557<<queries-video-encode-feedback, video encode feedback queries>>. 558 559.Valid Usage 560**** 561 * [[VUID-VkVideoEncodeInfoKHR-None-07012]] 562 The bound video session must: not be in <<video-session-uninitialized, 563 uninitialized>> state at the time the command is executed on the device 564**** 565 566include::{generated}/validity/structs/VkVideoEncodeInfoKHR.adoc[] 567-- 568 569[open,refpage='VkVideoEncodeFlagsKHR',desc='Reserved for future use',type='flags'] 570-- 571include::{generated}/api/flags/VkVideoEncodeFlagsKHR.adoc[] 572 573tlink:VkVideoEncodeFlagsKHR is a bitmask type for setting a mask, but is 574currently reserved for future use. 575-- 576 577[open,refpage='VkVideoEncodeRateControlInfoKHR',desc='Structure to set encode stream rate control parameters',type='structs'] 578-- 579The sname:VkVideoEncodeRateControlInfoKHR structure is defined as: 580 581include::{generated}/api/structs/VkVideoEncodeRateControlInfoKHR.adoc[] 582 583 * pname:sType is a elink:VkStructureType value identifying this structure. 584 * pname:pNext is `NULL` or a pointer to a structure extending this 585 structure. 586 * pname:flags is reserved for future use. 587 * pname:rateControlMode is a elink:VkVideoEncodeRateControlModeFlagBitsKHR 588 value specifying the encode stream rate control mode. 589 * pname:layerCount specifies the number of rate control layers in the 590 video encode stream. 591 * pname:pLayers is a pointer to an array of 592 slink:VkVideoEncodeRateControlLayerInfoKHR structures specifying the 593 rate control configurations of pname:layerCount rate control layers. 594 * pname:virtualBufferSizeInMs is the leaky bucket model virtual buffer 595 size in milliseconds, with respect to peak bitrate. 596 Valid when rate control mode is 597 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR or 598 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR. 599 For example, virtual buffer size is (pname:virtualBufferSizeInMs {times} 600 pname:maxBitrate / 1000). 601 * pname:initialVirtualBufferSizeInMs is the initial occupancy in 602 milliseconds of the virtual buffer in the leaky bucket model. 603 Valid when rate control mode is 604 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR or 605 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR. 606 607Including this structure in the pname:pNext chain of 608slink:VkVideoCodingControlInfoKHR and including 609ename:VK_VIDEO_CODING_CONTROL_ENCODE_RATE_CONTROL_BIT_KHR in 610slink:VkVideoCodingControlInfoKHR::pname:flags will define stream rate 611control settings for video encoding. 612 613Additional structures providing codec-specific rate control parameters may: 614need to be included in the pname:pNext chain of 615sname:VkVideoCodingControlInfoKHR depending on the codec profile the bound 616video session was created with and the parameters specified in 617sname:VkVideoEncodeRateControlInfoKHR (see <<video-coding-control,Video 618Coding Control>>). 619 620To ensure that the video session is properly initialized with stream-level 621rate control settings, the application must: call 622flink:vkCmdControlVideoCodingKHR with stream-level rate control settings at 623least once in execution order before the first flink:vkCmdEncodeVideoKHR 624command that is executed after video session reset. 625If not provided, default implementation-specific stream rate control 626settings will be used. 627 628Stream rate control settings can: also be re-initialized during an active 629video encoding session. 630The re-initialization takes effect whenever the 631sname:VkVideoEncodeRateControlInfoKHR structure is included in the 632pname:pNext chain of the slink:VkVideoCodingControlInfoKHR structure in the 633call to flink:vkCmdControlVideoCodingKHR, and only impacts 634flink:vkCmdEncodeVideoKHR operations that follow in execution order. 635 636include::{generated}/validity/structs/VkVideoEncodeRateControlInfoKHR.adoc[] 637-- 638 639[open,refpage='VkVideoEncodeRateControlFlagsKHR',desc='Reserved for future use',type='flags'] 640-- 641include::{generated}/api/flags/VkVideoEncodeRateControlFlagsKHR.adoc[] 642 643tname:VkVideoEncodeRateControlFlagsKHR is a bitmask type for setting a mask, 644but currently reserved for future use. 645-- 646 647[open,refpage='VkVideoEncodeRateControlModeFlagBitsKHR',desc='Video encode rate control modes',type='enums'] 648-- 649The rate control modes are defined with the following enums: 650 651include::{generated}/api/enums/VkVideoEncodeRateControlModeFlagBitsKHR.adoc[] 652 653 * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DEFAULT_KHR specifies the use of 654 implementation-specific rate control. 655 * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR specifies that 656 rate control is disabled and any quality control parameters for the 657 encoding are provided on a per-picture basis. 658 In this mode implementations will encode pictures independently of the 659 output bitrate of prior video encode operations. 660ifdef::VK_EXT_video_encode_h264[] 661 When using an H.264 encode profile, implementations will use the QP 662 values specified in the slink:VkVideoEncodeH264RateControlInfoEXT 663 structure for the encoded picture. 664endif::VK_EXT_video_encode_h264[] 665ifdef::VK_EXT_video_encode_h265[] 666 When using an H.265 encode profile, implementations will use the QP 667 values specified in the slink:VkVideoEncodeH265RateControlInfoEXT 668 structure for the encoded picture. 669endif::VK_EXT_video_encode_h265[] 670 * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR specifies the use of 671 constant bitrate rate control mode. 672 * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR specifies the use of 673 variable bitrate rate control mode. 674-- 675 676[open,refpage='VkVideoEncodeRateControlModeFlagsKHR',desc='Bitmask of VkVideoEncodeRateControlModeFlagBitsKHR', type='flags'] 677-- 678include::{generated}/api/flags/VkVideoEncodeRateControlModeFlagsKHR.adoc[] 679 680tname:VkVideoEncodeRateControlModeFlagsKHR is a bitmask type for setting a 681mask of zero or more elink:VkVideoEncodeRateControlModeFlagBitsKHR. 682-- 683 684[open,refpage='VkVideoEncodeRateControlLayerInfoKHR',desc='Structure to set encode per-layer rate control parameters',type='structs'] 685-- 686The sname:VkVideoEncodeRateControlLayerInfoKHR structure is defined as: 687 688include::{generated}/api/structs/VkVideoEncodeRateControlLayerInfoKHR.adoc[] 689 690 * pname:sType is a elink:VkStructureType value identifying this structure. 691 * pname:pNext is a pointer to a structure extending this structure. 692 * pname:averageBitrate is the average bitrate in bits/second. 693 Valid when rate control mode is 694 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR or 695 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR. 696 * pname:maxBitrate is the peak bitrate in bits/second. 697 Valid when rate control mode is 698 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR. 699 * pname:frameRateNumerator is the numerator of the frame rate. 700 Valid when rate control mode is 701 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR or 702 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR. 703 * pname:frameRateDenominator is the denominator of the frame rate. 704 Valid when rate control mode is 705 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR or 706 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR. 707 708A codec-specific structure specifying additional per-layer rate control 709settings must: be chained to sname:VkVideoEncodeRateControlLayerInfoKHR. 710If multiple rate control layers are enabled 711(slink:VkVideoEncodeRateControlInfoKHR::pname:layerCount is greater than 1), 712then the chained codec-specific extension structure also identifies the 713specific video coding layer its parent 714sname:VkVideoEncodeRateControlLayerInfoKHR applies to. 715If multiple rate control layers are enabled, the number of rate control 716layers must: match the number of video coding layers. 717The specification for an encode codec-specific extension would describe how 718multiple video coding layers are enabled for the corresponding codec. 719 720Per-layer rate control settings for all enabled rate control layers must: be 721initialized or re-initialized whenever stream rate control settings are 722provided via slink:VkVideoEncodeRateControlInfoKHR. 723This is done by specifying settings for all enabled rate control layers in 724slink:VkVideoEncodeRateControlInfoKHR::pname:pLayers. 725 726It is possible for an application to enable multiple video coding layers 727(via codec-specific extensions to encoding operations) while only enabling a 728single layer of rate control for the entire video stream. 729To achieve this, pname:layerCount in slink:VkVideoEncodeRateControlInfoKHR 730must: be set to 1, and the single sname:VkVideoEncodeRateControlLayerInfoKHR 731provided in pname:pLayers would apply to all encoded segments of the video 732stream, regardless of which codec-defined video coding layer they belong to. 733In this case, the implementation decides bitrate distribution across video 734coding layers (if applicable to the specified stream rate control mode). 735 736include::{generated}/validity/structs/VkVideoEncodeRateControlLayerInfoKHR.adoc[] 737-- 738 739