1// Copyright 2018-2022 The Khronos Group Inc. 2// 3// SPDX-License-Identifier: CC-BY-4.0 4 5[[encode-h264]] 6== Encode H.264 7 8The `apiext:VK_EXT_video_encode_h264` extension adds H.264 codec specific 9structures/types needed to support H.264 encoding. 10Unless otherwise noted, all references to the H.264 specification are to the 112010 edition published by the ITU-T, dated March 2010. 12This specification is available at https://www.itu.int/rec/T-REC-H.264. 13 14[NOTE] 15.Note 16==== 17Refer to the <<preamble, Preamble>> for information on how the Khronos 18Intellectual Property Rights Policy relates to normative references to 19external materials not created by Khronos. 20==== 21 22 23=== H.264 encode profile 24 25[open,refpage='VkVideoEncodeH264ProfileInfoEXT',desc='Structure specifying H.264 encode profile',type='structs'] 26-- 27The sname:VkVideoEncodeH264ProfileInfoEXT structure is defined as: 28 29include::{generated}/api/structs/VkVideoEncodeH264ProfileInfoEXT.adoc[] 30 31 * pname:sType is the type of this structure. 32 * pname:pNext is `NULL` or a pointer to a structure extending this 33 structure. 34 * pname:stdProfileIdc is a code:StdVideoH264ProfileIdc value specifying 35 the H.264 codec profile IDC. 36 37An H.264 encode profile is specified by including a 38sname:VkVideoEncodeH264ProfileInfoEXT structure in the pname:pNext chain of 39the slink:VkVideoProfileInfoKHR structure when 40slink:VkVideoProfileInfoKHR::pname:videoCodecOperation is 41ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_EXT. 42 43include::{generated}/validity/structs/VkVideoEncodeH264ProfileInfoEXT.adoc[] 44-- 45 46 47=== Capabilities 48 49[open,refpage='VkVideoEncodeH264CapabilitiesEXT',desc='Structure specifying H.264 encode capabilities',type='structs'] 50-- 51When calling flink:vkGetPhysicalDeviceVideoCapabilitiesKHR with 52pname:pVideoProfile->videoCodecOperation specified as 53ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_EXT, the 54slink:VkVideoEncodeH264CapabilitiesEXT structure must: be included in the 55pname:pNext chain of the slink:VkVideoCapabilitiesKHR structure to retrieve 56more capabilities specific to H.264 video encoding. 57 58The slink:VkVideoEncodeH264CapabilitiesEXT structure is defined as: 59 60include::{generated}/api/structs/VkVideoEncodeH264CapabilitiesEXT.adoc[] 61 62 * pname:sType is the type of this structure. 63 * pname:pNext is `NULL` or a pointer to a structure extending this 64 structure. 65 * pname:flags is a bitmask of elink:VkVideoEncodeH264CapabilityFlagBitsEXT 66 describing supported encoding tools. 67 * pname:inputModeFlags is a bitmask of 68 elink:VkVideoEncodeH264InputModeFlagBitsEXT describing supported command 69 buffer input granularities/modes. 70 * pname:outputModeFlags is a bitmask of 71 elink:VkVideoEncodeH264OutputModeFlagBitsEXT describing supported output 72 (bitstream size reporting) granularities/modes. 73 * pname:maxPPictureL0ReferenceCount reports the maximum number of 74 reference pictures the implementation supports in the reference list L0 75 for P pictures. 76 * pname:maxBPictureL0ReferenceCount reports the maximum number of 77 reference pictures the implementation supports in the reference list L0 78 for B pictures. 79 The reported value is `0` if encoding of B pictures is not supported. 80 * pname:maxL1ReferenceCount reports the maximum number of reference 81 pictures the implementation supports in the reference list L1 if 82 encoding of B pictures is supported. 83 The reported value is `0` if encoding of B pictures is not supported. 84 * pname:motionVectorsOverPicBoundariesFlag if ename:VK_TRUE, indicates 85 motion_vectors_over_pic_boundaries_flag will be enabled if 86 bitstream_restriction_flag is enabled in StdVideoH264SpsVuiFlags. 87 * pname:maxBytesPerPicDenom reports the value that will be used for 88 max_bytes_per_pic_denom if bitstream_restriction_flag is enabled in 89 StdVideoH264SpsVuiFlags. 90 * pname:maxBitsPerMbDenom reports the value that will be used for 91 max_bits_per_mb_denom if bitstream_restriction_flag is enabled in 92 StdVideoH264SpsVuiFlags. 93 * pname:log2MaxMvLengthHorizontal reports the value that will be used for 94 log2_max_mv_length_horizontal if bitstream_restriction_flag is enabled 95 in StdVideoH264SpsVuiFlags. 96 * pname:log2MaxMvLengthVertical reports the value that will be used for 97 log2_max_mv_length_vertical if bitstream_restriction_flag is enabled in 98 StdVideoH264SpsVuiFlags. 99 100When flink:vkGetPhysicalDeviceVideoCapabilitiesKHR is called to query the 101capabilities with parameter pname:videoCodecOperation specified as 102ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_EXT, a 103sname:VkVideoEncodeH264CapabilitiesEXT structure can: be chained to 104slink:VkVideoCapabilitiesKHR to retrieve H.264 extension specific 105capabilities. 106 107include::{generated}/validity/structs/VkVideoEncodeH264CapabilitiesEXT.adoc[] 108-- 109 110[open,refpage='VkVideoEncodeH264CapabilityFlagsEXT',desc='Bitmask of VkVideoEncodeH264CapabilityFlagBitsEXT',type='flags'] 111-- 112include::{generated}/api/flags/VkVideoEncodeH264CapabilityFlagsEXT.adoc[] 113 114tname:VkVideoEncodeH264CapabilityFlagsEXT is a bitmask type for setting a 115mask of zero or more elink:VkVideoEncodeH264CapabilityFlagBitsEXT. 116-- 117 118[open,refpage='VkVideoEncodeH264CapabilityFlagBitsEXT',desc='Video encode H.264 capability flags',type='enums'] 119-- 120Bits which may: be set in 121slink:VkVideoEncodeH264CapabilitiesEXT::pname:flags, indicating the encoding 122tools supported, are: 123 124include::{generated}/api/enums/VkVideoEncodeH264CapabilityFlagBitsEXT.adoc[] 125 126 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DIRECT_8X8_INFERENCE_ENABLED_BIT_EXT 127 reports if enabling direct_8x8_inference_flag in StdVideoH264SpsFlags is 128 supported. 129 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DIRECT_8X8_INFERENCE_DISABLED_BIT_EXT 130 reports if disabling direct_8x8_inference_flag in StdVideoH264SpsFlags 131 is supported. 132 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_SEPARATE_COLOUR_PLANE_BIT_EXT 133 reports if enabling separate_colour_plane_flag in StdVideoH264SpsFlags 134 is supported. 135 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_QPPRIME_Y_ZERO_TRANSFORM_BYPASS_BIT_EXT 136 reports if enabling qpprime_y_zero_transform_bypass_flag in 137 StdVideoH264SpsFlags is supported. 138 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_SCALING_LISTS_BIT_EXT reports if 139 enabling seq_scaling_matrix_present_flag in StdVideoH264SpsFlags or 140 pic_scaling_matrix_present_flag in StdVideoH264PpsFlags are supported. 141 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_HRD_COMPLIANCE_BIT_EXT reports if 142 the implementation guarantees generating a HRD compliant bitstream if 143 nal_hrd_parameters_present_flag or vcl_hrd_parameters_present_flag are 144 enabled in StdVideoH264SpsVuiFlags. 145 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_CHROMA_QP_OFFSET_BIT_EXT reports 146 if setting non-zero chroma_qp_index_offset in 147 StdVideoH264PictureParameterSet is supported. 148 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_SECOND_CHROMA_QP_OFFSET_BIT_EXT 149 reports if setting non-zero second_chroma_qp_index_offset in 150 StdVideoH264PictureParameterSet is supported. 151 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PIC_INIT_QP_MINUS26_BIT_EXT 152 reports if setting non-zero pic_init_qp_minus26 in 153 StdVideoH264PictureParameterSet is supported. 154 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_WEIGHTED_PRED_BIT_EXT reports if 155 enabling weighted_pred_flag in StdVideoH264PpsFlags is supported. 156 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_WEIGHTED_BIPRED_EXPLICIT_BIT_EXT 157 reports if using STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_EXPLICIT from 158 StdVideoH264WeightedBipredIdc is supported. 159 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_WEIGHTED_BIPRED_IMPLICIT_BIT_EXT 160 reports if using STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_IMPLICIT from 161 StdVideoH264WeightedBipredIdc is supported. 162 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_WEIGHTED_PRED_NO_TABLE_BIT_EXT 163 reports that when weighted_pred_flag is enabled or 164 STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_EXPLICIT from 165 StdVideoH264WeightedBipredIdc is used, the implementation is able to 166 internally decide syntax for pred_weight_table. 167 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_TRANSFORM_8X8_BIT_EXT reports if 168 enabling transform_8x8_mode_flag in StdVideoH264PpsFlags is supported. 169 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_CABAC_BIT_EXT reports if CABAC 170 entropy coding is supported. 171 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_CAVLC_BIT_EXT reports if CAVLC 172 entropy coding is supported. 173 An implementation must: support at least one entropy coding mode. 174 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DEBLOCKING_FILTER_DISABLED_BIT_EXT 175 reports if using STD_VIDEO_H264_DISABLE_DEBLOCKING_FILTER_IDC_DISABLED 176 from StdVideoH264DisableDeblockingFilterIdc is supported. 177 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DEBLOCKING_FILTER_ENABLED_BIT_EXT 178 reports if using STD_VIDEO_H264_DISABLE_DEBLOCKING_FILTER_IDC_ENABLED 179 from StdVideoH264DisableDeblockingFilterIdc is supported. 180 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DEBLOCKING_FILTER_PARTIAL_BIT_EXT 181 reports if using STD_VIDEO_H264_DISABLE_DEBLOCKING_FILTER_IDC_PARTIAL 182 from StdVideoH264DisableDeblockingFilterIdc is supported. 183 An implementation must: support at least one deblocking filter mode. 184 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DISABLE_DIRECT_SPATIAL_MV_PRED_BIT_EXT 185 reports if disabling 186 code:StdVideoEncodeH264SliceHeaderFlags::direct_spatial_mv_pred_flag is 187 supported when it is present in the slice header. 188 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_MULTIPLE_SLICE_PER_FRAME_BIT_EXT 189 reports if encoding multiple slices per frame is supported. 190 If not set, the implementation is only able to encode a single slice for 191 the entire frame. 192 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_SLICE_MB_COUNT_BIT_EXT reports 193 support for configuring 194 slink:VkVideoEncodeH264NaluSliceInfoEXT::pname:mbCount and 195 first_mb_in_slice in StdVideoEncodeH264SliceHeader for each slice in a 196 frame with multiple slices. 197 If not supported, the implementation decides the number of macroblocks 198 in each slice based on 199 slink:VkVideoEncodeH264VclFrameInfoEXT::pname:naluSliceEntryCount. 200 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_ROW_UNALIGNED_SLICE_BIT_EXT 201 reports that each slice in a frame with multiple slices may begin or 202 finish at any offset in a macroblock row. 203 If not supported, all slices in the frame must: begin at the start of a 204 macroblock row (and hence each slice must: finish at the end of a 205 macroblock row). 206 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DIFFERENT_SLICE_TYPE_BIT_EXT 207 reports that when 208 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_MULTIPLE_SLICE_PER_FRAME_BIT_EXT 209 is supported and a frame is encoded with multiple slices, the 210 implementation allows encoding each slice with a different 211 code:StdVideoEncodeH264SliceHeader::slice_type. 212 If not supported, all slices of the frame must: be encoded with the same 213 code:slice_type which corresponds to the picture type of the frame. 214 For example, all slices of a P-frame would be encoded as P-slices. 215 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_B_FRAME_IN_L1_LIST_BIT_EXT reports 216 support for using a B frame as L1 reference. 217-- 218 219[open,refpage='VkVideoEncodeH264InputModeFlagsEXT',desc='Bitmask of VkVideoEncodeH264InputModeFlagBitsEXT',type='flags'] 220-- 221include::{generated}/api/flags/VkVideoEncodeH264InputModeFlagsEXT.adoc[] 222 223tname:VkVideoEncodeH264InputModeFlagsEXT is a bitmask type for setting a 224mask of zero or more elink:VkVideoEncodeH264InputModeFlagBitsEXT. 225-- 226 227[open,refpage='VkVideoEncodeH264InputModeFlagBitsEXT',desc='Video encode H.264 input modes',type='enums'] 228-- 229The pname:inputModeFlags field reports the various command buffer input 230granularities supported by the implementation as follows: 231 232include::{generated}/api/enums/VkVideoEncodeH264InputModeFlagBitsEXT.adoc[] 233 234 * ename:VK_VIDEO_ENCODE_H264_INPUT_MODE_FRAME_BIT_EXT indicates that a 235 single command buffer must: at least encode an entire frame. 236 Any non-VCL NALUs must: be encoded using the same command buffer as the 237 frame if ename:VK_VIDEO_ENCODE_H264_INPUT_MODE_NON_VCL_BIT_EXT is not 238 supported. 239 * ename:VK_VIDEO_ENCODE_H264_INPUT_MODE_SLICE_BIT_EXT indicates that a 240 single command buffer must: at least encode a single slice. 241 Any non-VCL NALUs must: be encoded using the same command buffer as the 242 first slice of the frame if 243 ename:VK_VIDEO_ENCODE_H264_INPUT_MODE_NON_VCL_BIT_EXT is not supported. 244 * ename:VK_VIDEO_ENCODE_H264_INPUT_MODE_NON_VCL_BIT_EXT indicates that a 245 single command buffer may: encode a non-VCL NALU by itself. 246 247An implementation must: support at least one of 248ename:VK_VIDEO_ENCODE_H264_INPUT_MODE_FRAME_BIT_EXT or 249ename:VK_VIDEO_ENCODE_H264_INPUT_MODE_SLICE_BIT_EXT. 250 251If ename:VK_VIDEO_ENCODE_H264_INPUT_MODE_SLICE_BIT_EXT is not supported, the 252following two additional restrictions apply for frames encoded with multiple 253slices. 254First, all frame slices must: have the same pRefList0ModOperations and the 255same pRefList1ModOperations. 256Second, the order in which slices appear in 257slink:VkVideoEncodeH264VclFrameInfoEXT::pname:pNaluSliceEntries or in the 258command buffer must: match the placement order of the slices in the frame. 259-- 260 261[open,refpage='VkVideoEncodeH264OutputModeFlagsEXT',desc='Bitmask of VkVideoEncodeH264OutputModeFlagBitsEXT',type='flags'] 262-- 263include::{generated}/api/flags/VkVideoEncodeH264OutputModeFlagsEXT.adoc[] 264 265tname:VkVideoEncodeH264OutputModeFlagsEXT is a bitmask type for setting a 266mask of zero or more elink:VkVideoEncodeH264InputModeFlagBitsEXT. 267-- 268 269[open,refpage='VkVideoEncodeH264OutputModeFlagBitsEXT',desc='Video encode H.264 output modes',type='enums'] 270-- 271Bits which may: be set in 272slink:VkVideoEncodeH264CapabilitiesEXT::pname:outputModeFlags, indicating 273the minimum bitstream generation commands that must: be included between 274each flink:vkCmdBeginVideoCodingKHR and flink:vkCmdEndVideoCodingKHR pair 275(henceforth simply begin/end pair), are: 276 277include::{generated}/api/enums/VkVideoEncodeH264OutputModeFlagBitsEXT.adoc[] 278 279 * ename:VK_VIDEO_ENCODE_H264_OUTPUT_MODE_FRAME_BIT_EXT indicates that 280 calls to generate all NALUs of a frame must: be included within a single 281 begin/end pair. 282 Any non-VCL NALUs must: be encoded within the same begin/end pair if 283 ename:VK_VIDEO_ENCODE_H264_OUTPUT_MODE_NON_VCL_BIT_EXT is not supported. 284 * ename:VK_VIDEO_ENCODE_H264_OUTPUT_MODE_SLICE_BIT_EXT indicates that each 285 begin/end pair must: encode at least one slice. 286 Any non-VCL NALUs must: be encoded within the same begin/end pair as the 287 first slice of the frame if 288 ename:VK_VIDEO_ENCODE_H264_OUTPUT_MODE_NON_VCL_BIT_EXT is not supported. 289 * ename:VK_VIDEO_ENCODE_H264_OUTPUT_MODE_NON_VCL_BIT_EXT indicates that 290 each begin/end pair may: encode only a non-VCL NALU by itself. 291 An implementation must: support at least one of 292 ename:VK_VIDEO_ENCODE_H264_OUTPUT_MODE_FRAME_BIT_EXT or 293 ename:VK_VIDEO_ENCODE_H264_OUTPUT_MODE_SLICE_BIT_EXT. 294 295A single begin/end pair must: not encode more than a single frame. 296 297The bitstreams of NALUs generated within a single begin/end pair are written 298continuously into the same bitstream buffer (any padding between the NALUs 299must: be compliant to the H.264 standard). 300 301The supported input modes must: be coarser or equal to the supported output 302modes. 303For example, it is illegal to report slice input is supported but only frame 304output is supported. 305 306An implementation must: report one of the following combinations of 307input/output modes: 308 309 ** Input: Frame, Output: Frame 310 ** Input: Frame, Output: Frame and Non-VCL 311 ** Input: Frame, Output: Slice 312 ** Input: Frame, Output: Slice and Non-VCL 313 ** Input: Slice, Output: Slice 314 ** Input: Slice, Output: Slice and Non-VCL 315 ** Input: Frame and Non-VCL, Output: Frame and Non-VCL 316 ** Input: Frame and Non-VCL, Output: Slice and Non-VCL 317 ** Input: Slice and Non-VCL, Output: Slice and Non-VCL 318-- 319 320 321=== Encoder Parameter Sets 322 323To reduce parameter traffic during encoding, the encoder parameter set 324object supports storing H.264 SPS/PPS parameter sets that may: be later 325referenced during encoding. 326 327[open,refpage='VkVideoEncodeH264SessionParametersCreateInfoEXT',desc='Structure specifies H.264 encoder parameter set information',type='structs'] 328-- 329The slink:VkVideoEncodeH264SessionParametersCreateInfoEXT structure is 330defined as: 331 332include::{generated}/api/structs/VkVideoEncodeH264SessionParametersCreateInfoEXT.adoc[] 333 334 * pname:sType is the type of this structure. 335 * pname:pNext is `NULL` or a pointer to a structure extending this 336 structure. 337 * pname:maxStdSPSCount is the maximum number of SPS parameters that the 338 sname:VkVideoSessionParametersKHR can contain. 339 * pname:maxStdPPSCount is the maximum number of PPS parameters that the 340 sname:VkVideoSessionParametersKHR can contain. 341 * pname:pParametersAddInfo is `NULL` or a pointer to a 342 sname:VkVideoEncodeH264SessionParametersAddInfoEXT structure specifying 343 H.264 parameters to add upon object creation. 344 345A sname:VkVideoEncodeH264SessionParametersCreateInfoEXT structure holding 346one H.264 SPS and at least one H.264 PPS parameter set must: be chained to 347slink:VkVideoSessionParametersCreateInfoKHR when calling 348flink:vkCreateVideoSessionParametersKHR to store these parameter set(s) with 349the encoder parameter set object for later reference. 350The provided H.264 SPS/PPS parameters must: be within the limits specified 351during encoder creation for the encoder specified in 352slink:VkVideoSessionParametersCreateInfoKHR. 353 354include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersCreateInfoEXT.adoc[] 355-- 356 357[open,refpage='VkVideoEncodeH264SessionParametersAddInfoEXT',desc='Structure specifies H.264 encoder parameter set information',type='structs'] 358-- 359The sname:VkVideoEncodeH264SessionParametersAddInfoEXT structure is defined 360as: 361 362include::{generated}/api/structs/VkVideoEncodeH264SessionParametersAddInfoEXT.adoc[] 363 364 * pname:sType is the type of this structure. 365 * pname:pNext is `NULL` or a pointer to a structure extending this 366 structure. 367 * pname:stdSPSCount is the number of SPS elements in the pname:pStdSPSs. 368 Its value must: be less than or equal to the value of 369 pname:maxStdSPSCount. 370 * pname:pStdSPSs is a pointer to an array of 371 code:StdVideoH264SequenceParameterSet structures representing H.264 372 sequence parameter sets. 373 Each element of the array must: have a unique H.264 SPS ID. 374 * pname:stdPPSCount is the number of PPS provided in pname:pStdPPSs. 375 Its value must: be less than or equal to the value of 376 pname:maxStdPPSCount. 377 * pname:pStdPPSs is a pointer to an array of 378 code:StdVideoH264PictureParameterSet structures representing H.264 379 picture parameter sets. 380 Each element of the array must: have a unique H.264 SPS-PPS ID pair. 381 382.Valid Usage 383**** 384 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-stdSPSCount-04837]] 385 The values of pname:stdSPSCount and pname:stdPPSCount must: be less than 386 or equal to the values of pname:maxStdSPSCount and pname:maxStdPPSCount, 387 respectively 388 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-maxStdSPSCount-04838]] 389 When the pname:maxStdSPSCount number of parameters of type 390 StdVideoH264SequenceParameterSet in the Video Session Parameters object 391 is reached, no additional parameters of that type can be added to the 392 object. 393 ename:VK_ERROR_TOO_MANY_OBJECTS will be returned if an attempt is made 394 to add additional data to this object at this point 395 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-maxStdPPSCount-04839]] 396 When the pname:maxStdPPSCount number of parameters of type 397 StdVideoH264PictureParameterSet in the Video Session Parameters object 398 is reached, no additional parameters of that type can be added to the 399 object. 400 ename:VK_ERROR_TOO_MANY_OBJECTS will be returned if an attempt is made 401 to add additional data to this object at this point 402 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-None-04840]] 403 Each entry to be added must: have a unique, to the rest of the parameter 404 array entries and the existing parameters in the Video Session 405 Parameters Object that is being updated, SPS-PPS IDs 406 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-None-04841]] 407 Parameter entries that already exist in Video Session Parameters object 408 with a particular SPS-PPS IDs cannot: be replaced nor updated 409 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-None-04842]] 410 When creating a new object using a Video Session Parameters as a 411 template, the array's parameters with the same SPS-PPS IDs as the ones 412 from the template take precedence 413 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-None-04843]] 414 SPS/PPS parameters must: comply with the limits specified in 415 slink:VkVideoSessionCreateInfoKHR during Video Session creation 416**** 417 418include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersAddInfoEXT.adoc[] 419-- 420 421 422=== Frame Encoding 423 424In order to encode a frame, add a slink:VkVideoEncodeH264VclFrameInfoEXT 425structure to the pname:pNext chain of the slink:VkVideoEncodeInfoKHR 426structure passed to the flink:vkCmdEncodeVideoKHR command. 427 428[open,refpage='VkVideoEncodeH264VclFrameInfoEXT',desc='Structure specifies H.264 encode frame parameters',type='structs'] 429-- 430The slink:VkVideoEncodeH264VclFrameInfoEXT structure representing a frame 431encode operation is defined as: 432 433include::{generated}/api/structs/VkVideoEncodeH264VclFrameInfoEXT.adoc[] 434 435 * pname:sType is the type of this structure. 436 * pname:pNext is `NULL` or a pointer to a structure extending this 437 structure. 438 * pname:pReferenceFinalLists is `NULL` or a pointer to a 439 slink:VkVideoEncodeH264ReferenceListsInfoEXT structure specifying the 440 reference lists to be used for the current picture. 441 * pname:naluSliceEntryCount is the number of slice NALUs in the frame. 442 * pname:pNaluSliceEntries is a pointer to an array of 443 pname:naluSliceEntryCount slink:VkVideoEncodeH264NaluSliceInfoEXT 444 structures specifying the division of the current picture into slices 445 and the properties of these slices. 446 This is an ordered sequence; the NALUs are generated consecutively in 447 slink:VkVideoEncodeInfoKHR::pname:dstBitstreamBuffer in the same order 448 as in this array. 449 * pname:pCurrentPictureInfo is a pointer to a 450 code:StdVideoEncodeH264PictureInfo structure specifying the syntax and 451 other codec-specific information from the H.264 specification associated 452 with this picture. 453 The information provided must: reflect the decoded picture marking 454 operations that are applicable to this frame. 455 456include::{generated}/validity/structs/VkVideoEncodeH264VclFrameInfoEXT.adoc[] 457-- 458 459[open,refpage='VkVideoEncodeH264NaluSliceInfoEXT',desc='Structure specifies H.264 encode slice NALU parameters',type='structs'] 460-- 461The slink:VkVideoEncodeH264NaluSliceInfoEXT structure representing a slice 462is defined as: 463 464include::{generated}/api/structs/VkVideoEncodeH264NaluSliceInfoEXT.adoc[] 465 466 * pname:sType is the type of this structure. 467 * pname:pNext is `NULL` or a pointer to a structure extending this 468 structure. 469 * pname:mbCount is the number of macroblocks in this slice. 470 * pname:pReferenceFinalLists is `NULL` or a pointer to a 471 slink:VkVideoEncodeH264ReferenceListsInfoEXT structure specifying the 472 reference lists to be used for the current slice. 473 If pname:pReferenceFinalLists is not `NULL`, these reference lists 474 override the reference lists provided in 475 slink:VkVideoEncodeH264VclFrameInfoEXT::pname:pReferenceFinalLists. 476 * pname:pSliceHeaderStd is a pointer to a 477 code:StdVideoEncodeH264SliceHeader structure specifying the slice header 478 for the current slice. 479 480include::{generated}/validity/structs/VkVideoEncodeH264NaluSliceInfoEXT.adoc[] 481-- 482 483[open,refpage='VkVideoEncodeH264DpbSlotInfoEXT',desc='Structure specifies H.264 encode DPB picture information',type='structs'] 484-- 485The slink:VkVideoEncodeH264DpbSlotInfoEXT structure, representing a 486reconstructed picture that is being used as a reference picture, is defined 487as: 488 489include::{generated}/api/structs/VkVideoEncodeH264DpbSlotInfoEXT.adoc[] 490 491 * pname:sType is the type of this structure. 492 * pname:pNext is `NULL` or a pointer to a structure extending this 493 structure. 494 * pname:slotIndex is the <<dpb-slot, DPB Slot>> index for this picture. 495 pname:slotIndex must: match the pname:slotIndex in 496 pname:pSetupReferenceSlot of slink:VkVideoEncodeInfoKHR in the command 497 used to encode the corresponding picture. 498 * pname:pStdReferenceInfo is a pointer to a 499 code:StdVideoEncodeH264ReferenceInfo structure specifying the syntax and 500 other codec-specific information from the H.264 specification associated 501 with this reference picture. 502 503include::{generated}/validity/structs/VkVideoEncodeH264DpbSlotInfoEXT.adoc[] 504-- 505 506 507[open,refpage='VkVideoEncodeH264ReferenceListsInfoEXT',desc='Structure specifies H.264 reference frame lists',type='structs'] 508-- 509The slink:VkVideoEncodeH264ReferenceListsInfoEXT structure representing 510reference lists is defined as: 511 512include::{generated}/api/structs/VkVideoEncodeH264ReferenceListsInfoEXT.adoc[] 513 514 * pname:sType is the type of this structure. 515 * pname:pNext is `NULL` or a pointer to a structure extending this 516 structure. 517 * pname:referenceList0EntryCount is the number of reference pictures in 518 reference list L0 and is identical to 519 code:StdVideoEncodeH264SliceHeader::pname:num_ref_idx_l0_active_minus1 + 1. 520 * pname:pReferenceList0Entries is a pointer to an array of 521 pname:referenceList0EntryCount slink:VkVideoEncodeH264DpbSlotInfoEXT 522 structures specifying the reference list L0 entries for the current 523 picture. 524 The entries provided must: be ordered after all reference list L0 525 modification operations are applied (i.e. final list order). 526 The entries provided must: not reflect decoded picture marking 527 operations in this frame that are applicable to references; the impact 528 of such operations must: be reflected in future frame encode commands. 529 The slot index in each entry must: match one of the slot indexes 530 provided in the pname:pReferenceSlots of the parent 531 slink:VkVideoEncodeInfoKHR structure. 532 * pname:referenceList1EntryCount is the number of reference pictures in 533 reference list L1 and is identical to 534 code:StdVideoEncodeH264SliceHeader::pname:num_ref_idx_l1_active_minus1 + 1. 535 * pname:pReferenceList1Entries is a pointer to an array of 536 pname:referenceList1EntryCount slink:VkVideoEncodeH264DpbSlotInfoEXT 537 structures specifying the reference list L1 entries for the current 538 picture. 539 The entries provided must: be ordered after all reference list L1 540 modification operations are applied (i.e. final list order). 541 The entries provided must: not reflect decoded picture marking 542 operations in this frame that are applicable to references; the impact 543 of such operations must: be reflected in future frame encode commands. 544 The slot index in each entry must: match one of the slot indexes 545 provided in the pname:pReferenceSlots of the parent 546 slink:VkVideoEncodeInfoKHR structure. 547 * pname:pMemMgmtCtrlOperations is a pointer to a 548 code:StdVideoEncodeH264RefMemMgmtCtrlOperations structure specifying 549 reference lists modifications and decoded picture marking operations. 550 551include::{generated}/validity/structs/VkVideoEncodeH264ReferenceListsInfoEXT.adoc[] 552-- 553 554[open,refpage='VkVideoEncodeH264EmitPictureParametersInfoEXT',desc='Structure specifies H.264 encode SPS NALU insertion parameters',type='structs'] 555-- 556The sname:VkVideoEncodeH264EmitPictureParametersInfoEXT structure is defined 557as: 558 559include::{generated}/api/structs/VkVideoEncodeH264EmitPictureParametersInfoEXT.adoc[] 560 561 * pname:sType is the type of this structure. 562 * pname:pNext is `NULL` or a pointer to a structure extending this 563 structure. 564 * pname:spsId is the H.264 SPS ID for the H.264 SPS to insert in the 565 bitstream. 566 The SPS ID must: match the SPS provided in pname:spsStd of 567 slink:VkVideoEncodeH264SessionParametersCreateInfoEXT. 568 This is retrieved from the slink:VkVideoSessionParametersKHR object 569 provided in slink:VkVideoBeginCodingInfoKHR. 570 * pname:emitSpsEnable enables the emitting of the SPS structure with id of 571 pname:spsId. 572 * pname:ppsIdEntryCount is the number of entries in the 573 pname:ppsIdEntries. 574 If this parameter is `0` then no pps entries are going to be emitted in 575 the bitstream. 576 * pname:ppsIdEntries is a pointer to an array of H.264 PPS IDs for the 577 H.264 PPS to insert in the bitstream. 578 The PPS IDs must: match one of the IDs of the PPS(s) provided in 579 pname:pStdPPSs of slink:VkVideoEncodeH264SessionParametersCreateInfoEXT 580 to identify the PPS parameter set to insert in the bitstream. 581 This is retrieved from the slink:VkVideoSessionParametersKHR object 582 provided in slink:VkVideoBeginCodingInfoKHR. 583 584include::{generated}/validity/structs/VkVideoEncodeH264EmitPictureParametersInfoEXT.adoc[] 585-- 586 587 588=== Rate control 589 590[open,refpage='VkVideoEncodeH264RateControlInfoEXT',desc='Structure describing H.264 stream rate control parameters',type='structs'] 591-- 592The sname:VkVideoEncodeH264RateControlInfoEXT structure is defined as: 593 594include::{generated}/api/structs/VkVideoEncodeH264RateControlInfoEXT.adoc[] 595 596 * pname:sType is the type of this structure. 597 * pname:pNext is `NULL` or a pointer to a structure extending this 598 structure. 599 * pname:gopFrameCount is the number of frames contained within the group 600 of pictures (GOP), starting from an intra frame and until the next intra 601 frame. 602 If it is set to 0, the implementation chooses a suitable value. 603 If it is set to code:UINT32_MAX, the GOP length is treated as infinite. 604 * pname:idrPeriod is the interval, in terms of number of frames, between 605 two IDR frames. 606 If it is set to 0, the implementation chooses a suitable value. 607 If it is set to code:UINT32_MAX, the IDR period is treated as infinite. 608 * pname:consecutiveBFrameCount is the number of consecutive B-frames 609 between I- and/or P-frames within the GOP. 610 * pname:rateControlStructure is a 611 elink:VkVideoEncodeH264RateControlStructureEXT value specifying the 612 expected encode stream reference structure, to aid in rate control 613 calculations. 614 * pname:temporalLayerCount specifies the number of temporal layers enabled 615 in the stream. 616 617In order to provide H.264-specific stream rate control parameters, add a 618sname:VkVideoEncodeH264RateControlInfoEXT structure to the pname:pNext chain 619of the slink:VkVideoEncodeRateControlInfoKHR structure in the pname:pNext 620chain of the slink:VkVideoCodingControlInfoKHR structure passed to the 621flink:vkCmdControlVideoCodingKHR command. 622 623The parameters from this structure act as a guidance for implementations to 624apply various rate control heuristics. 625 626It is possible to infer the picture type to be used when encoding a frame, 627on the basis of the values provided for pname:consecutiveBFrameCount, 628pname:idrPeriod, and pname:gopFrameCount, but this inferred picture type 629will not be used by implementations to override the picture type provided in 630flink:vkCmdEncodeVideoKHR. 631Additionally, it is not required for the video session to be reset if the 632inferred picture type does not match the actual picture type. 633 634include::{generated}/validity/structs/VkVideoEncodeH264RateControlInfoEXT.adoc[] 635-- 636 637[open,refpage='VkVideoEncodeH264RateControlStructureEXT',desc='Specifies the video encode H.264 rate control structure',type='enums'] 638-- 639The pname:rateControlStructure in slink:VkVideoEncodeH264RateControlInfoEXT 640specifies one of the following video stream reference structures as a hint 641for the rate control implementation: 642 643include::{generated}/api/enums/VkVideoEncodeH264RateControlStructureEXT.adoc[] 644 645 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_STRUCTURE_UNKNOWN_EXT specifies 646 a reference structure unknown at the time of stream rate control 647 configuration. 648 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_STRUCTURE_FLAT_EXT specifies a 649 flat reference structure. 650 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_STRUCTURE_DYADIC_EXT specifies a 651 dyadic reference structure. 652-- 653 654[open,refpage='VkVideoEncodeH264RateControlLayerInfoEXT',desc='Structure describing H.264 per-layer rate control parameters',type='structs'] 655-- 656The sname:VkVideoEncodeH264RateControlLayerInfoEXT structure is defined as: 657 658include::{generated}/api/structs/VkVideoEncodeH264RateControlLayerInfoEXT.adoc[] 659 660 * pname:sType is the type of this structure. 661 * pname:pNext is `NULL` or a pointer to a structure extending this 662 structure. 663 * pname:temporalLayerId specifies the H.264 temporal layer ID of the video 664 coding layer that settings provided in this structure and its parent 665 slink:VkVideoEncodeRateControlLayerInfoKHR structure apply to. 666 * pname:useInitialRcQp indicates whether the values within 667 pname:initialRcQp should be used by the implementation. 668 * pname:initialRcQp provides the QP values for each picture type, to be 669 used in rate control calculations at the start of video encode 670 operations on a newly-created video session, or immediately after a 671 session reset. 672 These values are ignored when 673 slink:VkVideoEncodeRateControlInfoKHR::pname:rateControlMode is 674 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR. 675 * pname:useMinQp indicates whether the values within pname:minQp should be 676 used by the implementation. 677 When it is set to ename:VK_FALSE, the implementation ignores the values 678 in pname:minQp and chooses suitable values. 679 * pname:minQp provides the lower bound on the QP values for each picture 680 type, to be used in rate control calculations. 681 * pname:useMaxQp indicates whether the values within pname:maxQp should be 682 used by the implementation. 683 When it is set to ename:VK_FALSE, the implementation ignores the values 684 in pname:maxQp and chooses suitable values. 685 * pname:maxQp provides the upper bound on the QP values for each picture 686 type, to be used in rate control calculations. 687 * pname:useMaxFrameSize indicates whether the values within 688 pname:maxFrameSize should be used by the implementation. 689 * pname:maxFrameSize provides the upper bound on the encoded frame size 690 for each picture type. 691 The implementation does not guarantee the encoded frame sizes will be 692 within the specified limits, however these limits may: be used as a 693 guide in rate control calculations. 694 If enabled and not set properly, the pname:maxQp limit may prevent the 695 implementation from respecting the pname:maxFrameSize limit. 696 697H.264-specific per-layer rate control parameters must: be specified by 698adding a sname:VkVideoEncodeH264RateControlLayerInfoEXT structure to the 699pname:pNext chain of each slink:VkVideoEncodeRateControlLayerInfoKHR 700structure in a call to flink:vkCmdControlVideoCodingKHR command, when the 701command buffer context has an active video encode H.264 session. 702 703.Valid Usage 704**** 705 * [[VUID-VkVideoEncodeH264RateControlLayerInfoEXT-rateControlMode-06474]] 706 When slink:VkVideoEncodeRateControlInfoKHR::pname:rateControlMode is 707 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR, both 708 pname:useMinQp and pname:useMaxQp must be set to ename:VK_TRUE 709 * [[VUID-VkVideoEncodeH264RateControlLayerInfoEXT-rateControlMode-06475]] 710 When slink:VkVideoEncodeRateControlInfoKHR::pname:rateControlMode is 711 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR, the values 712 provided in pname:minQP must be identical to those provided in 713 pname:maxQp 714**** 715 716include::{generated}/validity/structs/VkVideoEncodeH264RateControlLayerInfoEXT.adoc[] 717-- 718 719[open,refpage='VkVideoEncodeH264QpEXT',desc='Structure describing H.264 QP values per picture type',type='structs'] 720-- 721The sname:VkVideoEncodeH264QpEXT structure is defined as: 722 723include::{generated}/api/structs/VkVideoEncodeH264QpEXT.adoc[] 724 725 * pname:qpI is the QP to be used for I-frames. 726 * pname:qpP is the QP to be used for P-frames. 727 * pname:qpB is the QP to be used for B-frames. 728 729include::{generated}/validity/structs/VkVideoEncodeH264QpEXT.adoc[] 730-- 731 732[open,refpage='VkVideoEncodeH264FrameSizeEXT',desc='Structure describing frame size values per H.264 picture type',type='structs'] 733-- 734The sname:VkVideoEncodeH264FrameSizeEXT structure is defined as: 735 736include::{generated}/api/structs/VkVideoEncodeH264FrameSizeEXT.adoc[] 737 738 * pname:frameISize is the size in bytes to be used for I-frames. 739 * pname:framePSize is the size in bytes to be used for P-frames. 740 * pname:frameBSize is the size in bytes to be used for B-frames. 741 742include::{generated}/validity/structs/VkVideoEncodeH264FrameSizeEXT.adoc[] 743-- 744 745 746