1// Copyright 2018-2023 The Khronos Group Inc. 2// 3// SPDX-License-Identifier: CC-BY-4.0 4 5[[encode-h264]] 6== Encode H.264 7 8The `apiext:VK_EXT_video_encode_h264` extension adds H.264 codec specific 9structures/types needed to support H.264 encoding. 10Unless otherwise noted, all references to the H.264 specification are to the 112010 edition published by the ITU-T, dated March 2010. 12This specification is available at https://www.itu.int/rec/T-REC-H.264. 13 14[NOTE] 15.Note 16==== 17Refer to the <<preamble, Preamble>> for information on how the Khronos 18Intellectual Property Rights Policy relates to normative references to 19external materials not created by Khronos. 20==== 21 22 23=== H.264 Encode Profile 24 25[open,refpage='VkVideoEncodeH264ProfileInfoEXT',desc='Structure specifying H.264 encode profile',type='structs'] 26-- 27The sname:VkVideoEncodeH264ProfileInfoEXT structure is defined as: 28 29include::{generated}/api/structs/VkVideoEncodeH264ProfileInfoEXT.adoc[] 30 31 * pname:sType is a elink:VkStructureType value identifying this structure. 32 * pname:pNext is `NULL` or a pointer to a structure extending this 33 structure. 34 * pname:stdProfileIdc is a code:StdVideoH264ProfileIdc value specifying 35 the H.264 codec profile IDC. 36 37An H.264 encode profile is specified by including a 38sname:VkVideoEncodeH264ProfileInfoEXT structure in the pname:pNext chain of 39the slink:VkVideoProfileInfoKHR structure when 40slink:VkVideoProfileInfoKHR::pname:videoCodecOperation is 41ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_EXT. 42 43include::{generated}/validity/structs/VkVideoEncodeH264ProfileInfoEXT.adoc[] 44-- 45 46 47=== Capabilities 48 49[open,refpage='VkVideoEncodeH264CapabilitiesEXT',desc='Structure specifying H.264 encode capabilities',type='structs'] 50-- 51When calling flink:vkGetPhysicalDeviceVideoCapabilitiesKHR with 52pname:pVideoProfile->videoCodecOperation specified as 53ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_EXT, the 54slink:VkVideoEncodeH264CapabilitiesEXT structure must: be included in the 55pname:pNext chain of the slink:VkVideoCapabilitiesKHR structure to retrieve 56more capabilities specific to H.264 video encoding. 57 58The slink:VkVideoEncodeH264CapabilitiesEXT structure is defined as: 59 60include::{generated}/api/structs/VkVideoEncodeH264CapabilitiesEXT.adoc[] 61 62 * pname:sType is a elink:VkStructureType value identifying this structure. 63 * pname:pNext is `NULL` or a pointer to a structure extending this 64 structure. 65 * pname:flags is a bitmask of elink:VkVideoEncodeH264CapabilityFlagBitsEXT 66 indicating supported H.264 encoding capabilities. 67 * pname:maxLevelIdc is a code:StdVideoH264LevelIdc value indicating the 68 maximum H.264 level supported. 69 * pname:maxSliceCount indicates the maximum number of slices that can: be 70 encoded for a single picture. 71 Further restrictions may: apply to the number of slices that can: be 72 encoded for a single picture depending on other capabilities and 73 codec-specific rules. 74 * pname:maxPPictureL0ReferenceCount indicates the maximum number of 75 reference pictures the implementation supports in the reference list L0 76 for P pictures. 77 * pname:maxBPictureL0ReferenceCount indicates the maximum number of 78 reference pictures the implementation supports in the reference list L0 79 for B pictures. 80 The reported value is `0` if encoding of B pictures is not supported. 81 * pname:maxL1ReferenceCount reports the maximum number of reference 82 pictures the implementation supports in the reference list L1 if 83 encoding of B pictures is supported. 84 The reported value is `0` if encoding of B pictures is not supported. 85 * pname:maxTemporalLayerCount indicates the maximum number of H.264 86 temporal layers supported by the implementation. 87 * pname:expectDyadicTemporalLayerPattern indicates that the 88 implementation's rate control algorithms expect the application to use a 89 dyadic temporal layer pattern when encoding multiple temporal layers. 90 * pname:minQp indicates the minimum QP value supported. 91 * pname:maxQp indicates the maximum QP value supported. 92 * pname:prefersGopRemainingFrames indicates that the implementation's rate 93 control algorithm prefers the application to specify the number of 94 frames of each type remaining in the current group of pictures. 95 * pname:requiresGopRemainingFrames indicates that the implementation's 96 rate control algorithm requires the application to specify the number of 97 frames of each type remaining in the current group of pictures. 98 * pname:stdSyntaxFlags is a bitmask of 99 elink:VkVideoEncodeH264StdFlagBitsEXT indicating capabilities related to 100 H.264 syntax elements. 101 102When flink:vkGetPhysicalDeviceVideoCapabilitiesKHR is called to query the 103capabilities with parameter pname:videoCodecOperation specified as 104ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_EXT, a 105sname:VkVideoEncodeH264CapabilitiesEXT structure can: be chained to 106slink:VkVideoCapabilitiesKHR to retrieve H.264 extension specific 107capabilities. 108 109include::{generated}/validity/structs/VkVideoEncodeH264CapabilitiesEXT.adoc[] 110-- 111 112[open,refpage='VkVideoEncodeH264CapabilityFlagBitsEXT',desc='Video encode H.264 capability flags',type='enums'] 113-- 114Bits which may: be set in 115slink:VkVideoEncodeH264CapabilitiesEXT::pname:flags, indicating the 116supported H.264 encoding capabilities, are: 117 118include::{generated}/api/enums/VkVideoEncodeH264CapabilityFlagBitsEXT.adoc[] 119 120 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_HRD_COMPLIANCE_BIT_EXT indicates 121 if the implementation guarantees generating a HRD compliant bitstream if 122 code:nal_hrd_parameters_present_flag or 123 code:vcl_hrd_parameters_present_flag are enabled in 124 code:StdVideoH264SpsVuiFlags. 125 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PREDICTION_WEIGHT_TABLE_GENERATED_BIT_EXT 126 indicates that when code:weighted_pred_flag is enabled or 127 code:STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_EXPLICIT from 128 code:StdVideoH264WeightedBipredIdc is used, the implementation is able 129 to internally decide syntax for code:pred_weight_table. 130 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_ROW_UNALIGNED_SLICE_BIT_EXT 131 indicates that each slice in a frame with multiple slices may begin or 132 finish at any offset in a macroblock row. 133 If not supported, all slices in the frame must: begin at the start of a 134 macroblock row (and hence each slice must: finish at the end of a 135 macroblock row). 136 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DIFFERENT_SLICE_TYPE_BIT_EXT 137 indicates that when a frame is encoded with multiple slices, the 138 implementation allows encoding each slice with a different 139 code:StdVideoEncodeH264SliceHeader::code:slice_type. 140 If not supported, all slices of the frame must: be encoded with the same 141 code:slice_type which corresponds to the picture type of the frame. 142 For example, all slices of a P-frame would be encoded as P-slices. 143 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_B_FRAME_IN_L0_LIST_BIT_EXT 144 indicates support for using a B frame as L0 reference. 145 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_B_FRAME_IN_L1_LIST_BIT_EXT 146 indicates support for using a B frame as L1 reference. 147 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PER_PICTURE_TYPE_MIN_MAX_QP_BIT_EXT 148 indicates support for specifying different QP values in the members of 149 slink:VkVideoEncodeH264QpEXT. 150 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PER_SLICE_CONSTANT_QP_BIT_EXT 151 indicates support for specifying different constant QP values for each 152 slice. 153 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_GENERATE_PREFIX_NALU_BIT_EXT 154 indicates support for generating prefix NALUs by setting 155 slink:VkVideoEncodeH264PictureInfoEXT::pname:generatePrefixNalu to 156 ename:VK_TRUE. 157-- 158 159[open,refpage='VkVideoEncodeH264CapabilityFlagsEXT',desc='Bitmask of VkVideoEncodeH264CapabilityFlagBitsEXT',type='flags'] 160-- 161include::{generated}/api/flags/VkVideoEncodeH264CapabilityFlagsEXT.adoc[] 162 163tname:VkVideoEncodeH264CapabilityFlagsEXT is a bitmask type for setting a 164mask of zero or more elink:VkVideoEncodeH264CapabilityFlagBitsEXT. 165-- 166 167[open,refpage='VkVideoEncodeH264StdFlagBitsEXT',desc='Video encode H.264 syntax capability flags',type='enums'] 168-- 169Bits which may: be set in 170slink:VkVideoEncodeH264CapabilitiesEXT::pname:stdSyntaxFlags, indicating the 171capabilities related to the H.264 syntax elements, are: 172 173include::{generated}/api/enums/VkVideoEncodeH264StdFlagBitsEXT.adoc[] 174 175 * ename:VK_VIDEO_ENCODE_H264_STD_SEPARATE_COLOR_PLANE_FLAG_SET_BIT_EXT 176 indicates if enabling code:separate_colour_plane_flag in 177 code:StdVideoH264SpsFlags is supported. 178 * ename:VK_VIDEO_ENCODE_H264_STD_QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG_SET_BIT_EXT 179 indicates if enabling code:qpprime_y_zero_transform_bypass_flag in 180 code:StdVideoH264SpsFlags is supported. 181 * ename:VK_VIDEO_ENCODE_H264_STD_SCALING_MATRIX_PRESENT_FLAG_SET_BIT_EXT 182 indicates if enabling code:seq_scaling_matrix_present_flag in 183 code:StdVideoH264SpsFlags or code:pic_scaling_matrix_present_flag in 184 code:StdVideoH264PpsFlags are supported. 185 * ename:VK_VIDEO_ENCODE_H264_STD_CHROMA_QP_INDEX_OFFSET_BIT_EXT indicates 186 if setting non-zero code:chroma_qp_index_offset in 187 code:StdVideoH264PictureParameterSet is supported. 188 * ename:VK_VIDEO_ENCODE_H264_STD_SECOND_CHROMA_QP_INDEX_OFFSET_BIT_EXT 189 indicates if setting non-zero code:second_chroma_qp_index_offset in 190 code:StdVideoH264PictureParameterSet is supported. 191 * ename:VK_VIDEO_ENCODE_H264_STD_PIC_INIT_QP_MINUS26_BIT_EXT indicates if 192 setting non-zero code:pic_init_qp_minus26 in 193 code:StdVideoH264PictureParameterSet is supported. 194 * ename:VK_VIDEO_ENCODE_H264_STD_WEIGHTED_PRED_FLAG_SET_BIT_EXT indicates 195 if enabling code:weighted_pred_flag in code:StdVideoH264PpsFlags is 196 supported. 197 * ename:VK_VIDEO_ENCODE_H264_STD_WEIGHTED_BIPRED_IDC_EXPLICIT_BIT_EXT 198 indicates if using code:STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_EXPLICIT from 199 code:StdVideoH264WeightedBipredIdc is supported. 200 * ename:VK_VIDEO_ENCODE_H264_STD_WEIGHTED_BIPRED_IDC_IMPLICIT_BIT_EXT 201 indicates if using code:STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_IMPLICIT from 202 code:StdVideoH264WeightedBipredIdc is supported. 203 * ename:VK_VIDEO_ENCODE_H264_STD_TRANSFORM_8X8_MODE_FLAG_SET_BIT_EXT 204 indicates if enabling code:transform_8x8_mode_flag in 205 code:StdVideoH264PpsFlags is supported. 206 * ename:VK_VIDEO_ENCODE_H264_STD_DIRECT_SPATIAL_MV_PRED_FLAG_UNSET_BIT_EXT 207 indicates if disabling 208 code:StdVideoEncodeH264SliceHeaderFlags::code:direct_spatial_mv_pred_flag 209 is supported when it is present in the slice header. 210 * ename:VK_VIDEO_ENCODE_H264_STD_ENTROPY_CODING_MODE_FLAG_UNSET_BIT_EXT 211 indicates if CAVLC entropy coding is supported. 212 * ename:VK_VIDEO_ENCODE_H264_STD_ENTROPY_CODING_MODE_FLAG_SET_BIT_EXT 213 indicates if CABAC entropy coding is supported. 214 An implementation must: support at least one entropy coding mode. 215 * ename:VK_VIDEO_ENCODE_H264_STD_DIRECT_8X8_INFERENCE_FLAG_UNSET_BIT_EXT 216 indicates if disabling code:direct_8x8_inference_flag in 217 code:StdVideoH264SpsFlags is supported. 218 * ename:VK_VIDEO_ENCODE_H264_STD_CONSTRAINED_INTRA_PRED_FLAG_SET_BIT_EXT 219 indicates if enabling code:constrained_intra_pred_flag in 220 code:StdVideoH264PpsFlags is supported. 221 * ename:VK_VIDEO_ENCODE_H264_STD_DEBLOCKING_FILTER_DISABLED_BIT_EXT 222 indicates if using 223 code:STD_VIDEO_H264_DISABLE_DEBLOCKING_FILTER_IDC_DISABLED from 224 StdVideoH264DisableDeblockingFilterIdc is supported. 225 * ename:VK_VIDEO_ENCODE_H264_STD_DEBLOCKING_FILTER_ENABLED_BIT_EXT 226 indicates if using 227 code:STD_VIDEO_H264_DISABLE_DEBLOCKING_FILTER_IDC_ENABLED from 228 StdVideoH264DisableDeblockingFilterIdc is supported. 229 * ename:VK_VIDEO_ENCODE_H264_STD_DEBLOCKING_FILTER_PARTIAL_BIT_EXT 230 indicates if using 231 code:STD_VIDEO_H264_DISABLE_DEBLOCKING_FILTER_IDC_PARTIAL from 232 StdVideoH264DisableDeblockingFilterIdc is supported. 233 An implementation must: support at least one deblocking filter mode. 234 * ename:VK_VIDEO_ENCODE_H264_STD_SLICE_QP_DELTA_BIT_EXT indicates whether 235 the implementation supports using the application-provided value for 236 code:StdVideoEncodeH264SliceHeader::code:slice_qp_delta when that value 237 is identical across the slices of the encoded frame. 238 * ename:VK_VIDEO_ENCODE_H264_STD_DIFFERENT_SLICE_QP_DELTA_BIT_EXT 239 indicates whether the implementation supports using the 240 application-provided value for 241 code:StdVideoEncodeH264SliceHeader::code:slice_qp_delta when that value 242 is different across the slices of the encoded frame. 243-- 244 245[open,refpage='VkVideoEncodeH264StdFlagsEXT',desc='Bitmask of VkVideoEncodeH264StdFlagBitsEXT',type='flags'] 246-- 247include::{generated}/api/flags/VkVideoEncodeH264StdFlagsEXT.adoc[] 248 249tname:VkVideoEncodeH264StdFlagsEXT is a bitmask type for setting a mask of 250zero or more elink:VkVideoEncodeH264StdFlagBitsEXT. 251-- 252 253 254=== H.264 Encode Quality Level Properties 255 256[open,refpage='VkVideoEncodeH264QualityLevelPropertiesEXT',desc='Structure describing the H.264 encode quality level properties',type='structs'] 257-- 258When calling flink:vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR 259with pname:pVideoProfile->videoCodecOperation specified as 260ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_EXT, the 261slink:VkVideoEncodeH264QualityLevelPropertiesEXT structure can: be included 262in the pname:pNext chain of the slink:VkVideoEncodeQualityLevelPropertiesKHR 263structure to retrieve additional video encode quality level properties 264specific to H.264 encoding. 265 266The slink:VkVideoEncodeH264QualityLevelPropertiesEXT structure is defined 267as: 268 269include::{generated}/api/structs/VkVideoEncodeH264QualityLevelPropertiesEXT.adoc[] 270 271 * pname:sType is a elink:VkStructureType value identifying this structure. 272 * pname:pNext is `NULL` or a pointer to a structure extending this 273 structure. 274 * pname:preferredRateControlFlags is a bitmask of 275 elink:VkVideoEncodeH264RateControlFlagBitsEXT values indicating the 276 preferred flags to use for 277 slink:VkVideoEncodeH264RateControlInfoEXT::pname:flags. 278 * pname:preferredGopFrameCount indicates the preferred value to use for 279 slink:VkVideoEncodeH264RateControlInfoEXT::pname:gopFrameCount. 280 * pname:preferredIdrPeriod indicates the preferred value to use for 281 slink:VkVideoEncodeH264RateControlInfoEXT::pname:idrPeriod. 282 * pname:preferredConsecutiveBFrameCount indicates the preferred value to 283 use for 284 slink:VkVideoEncodeH264RateControlInfoEXT::pname:consecutiveBFrameCount. 285 * pname:preferredTemporalLayerCount indicates the preferred value to use 286 for slink:VkVideoEncodeH264RateControlInfoEXT::pname:temporalLayerCount. 287 * pname:preferredConstantQp indicates the preferred values to use for 288 slink:VkVideoEncodeH264NaluSliceInfoEXT::pname:constantQp for each 289 picture type when using rate control mode 290 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR. 291 * pname:preferredMaxL0ReferenceCount indicates the preferred maximum 292 number of reference pictures to use in the reference list L0. 293 * pname:preferredMaxL1ReferenceCount indicates the preferred maximum 294 number of reference pictures to use in the reference list L1. 295 * pname:preferredStdEntropyCodingModeFlag indicates the preferred value to 296 use for code:entropy_coding_mode_flag in code:StdVideoH264PpsFlags. 297 298include::{generated}/validity/structs/VkVideoEncodeH264QualityLevelPropertiesEXT.adoc[] 299-- 300 301 302=== H.264 Encode Session 303 304Additional parameters can be specified when creating a video session with an 305H.264 encode profile by including an instance of the 306slink:VkVideoEncodeH264SessionCreateInfoEXT structure in the pname:pNext 307chain of slink:VkVideoSessionCreateInfoKHR. 308 309[open,refpage='VkVideoEncodeH264SessionCreateInfoEXT',desc='Structure specifies H.264 encode session parameters',type='structs'] 310-- 311The sname:VkVideoEncodeH264SessionCreateInfoEXT structure is defined as: 312 313include::{generated}/api/structs/VkVideoEncodeH264SessionCreateInfoEXT.adoc[] 314 315 * pname:sType is a elink:VkStructureType value identifying this structure. 316 * pname:pNext is `NULL` or a pointer to a structure extending this 317 structure. 318 * pname:useMaxLevelIdc indicates whether the value of pname:maxLevelIdc 319 should be used by the implementation. 320 When it is set to ename:VK_FALSE, the implementation ignores the value 321 of pname:maxLevelIdc and uses the value of 322 slink:VkVideoEncodeH264CapabilitiesEXT::pname:maxLevelIdc, as reported 323 by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video profile. 324 * pname:maxLevelIdc provides the upper bound on the H.264 level for the 325 video bitstreams produced by the created video session. 326 327include::{generated}/validity/structs/VkVideoEncodeH264SessionCreateInfoEXT.adoc[] 328-- 329 330 331=== Encoder Parameter Sets 332 333To reduce parameter traffic during encoding, the encoder parameter set 334object supports storing H.264 SPS/PPS parameter sets that may: be later 335referenced during encoding. 336 337[open,refpage='VkVideoEncodeH264SessionParametersCreateInfoEXT',desc='Structure specifies H.264 encoder parameter set information',type='structs'] 338-- 339The slink:VkVideoEncodeH264SessionParametersCreateInfoEXT structure is 340defined as: 341 342include::{generated}/api/structs/VkVideoEncodeH264SessionParametersCreateInfoEXT.adoc[] 343 344 * pname:sType is a elink:VkStructureType value identifying this structure. 345 * pname:pNext is `NULL` or a pointer to a structure extending this 346 structure. 347 * pname:maxStdSPSCount is the maximum number of SPS parameters that the 348 sname:VkVideoSessionParametersKHR can contain. 349 * pname:maxStdPPSCount is the maximum number of PPS parameters that the 350 sname:VkVideoSessionParametersKHR can contain. 351 * pname:pParametersAddInfo is `NULL` or a pointer to a 352 sname:VkVideoEncodeH264SessionParametersAddInfoEXT structure specifying 353 H.264 parameters to add upon object creation. 354 355include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersCreateInfoEXT.adoc[] 356-- 357 358[open,refpage='VkVideoEncodeH264SessionParametersAddInfoEXT',desc='Structure specifies H.264 encoder parameter set information',type='structs'] 359-- 360The sname:VkVideoEncodeH264SessionParametersAddInfoEXT structure is defined 361as: 362 363include::{generated}/api/structs/VkVideoEncodeH264SessionParametersAddInfoEXT.adoc[] 364 365 * pname:sType is a elink:VkStructureType value identifying this structure. 366 * pname:pNext is `NULL` or a pointer to a structure extending this 367 structure. 368 * pname:stdSPSCount is the number of SPS elements in the pname:pStdSPSs. 369 Its value must: be less than or equal to the value of 370 pname:maxStdSPSCount. 371 * pname:pStdSPSs is a pointer to an array of 372 code:StdVideoH264SequenceParameterSet structures representing H.264 373 sequence parameter sets. 374 Each element of the array must: have a unique H.264 SPS ID. 375 * pname:stdPPSCount is the number of PPS provided in pname:pStdPPSs. 376 Its value must: be less than or equal to the value of 377 pname:maxStdPPSCount. 378 * pname:pStdPPSs is a pointer to an array of 379 code:StdVideoH264PictureParameterSet structures representing H.264 380 picture parameter sets. 381 Each element of the array must: have a unique H.264 SPS-PPS ID pair. 382 383.Valid Usage 384**** 385 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-stdSPSCount-04837]] 386 The values of pname:stdSPSCount and pname:stdPPSCount must: be less than 387 or equal to the values of pname:maxStdSPSCount and pname:maxStdPPSCount, 388 respectively 389 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-maxStdSPSCount-04838]] 390 When the pname:maxStdSPSCount number of parameters of type 391 StdVideoH264SequenceParameterSet in the Video Session Parameters object 392 is reached, no additional parameters of that type can be added to the 393 object. 394 ename:VK_ERROR_TOO_MANY_OBJECTS will be returned if an attempt is made 395 to add additional data to this object at this point 396 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-maxStdPPSCount-04839]] 397 When the pname:maxStdPPSCount number of parameters of type 398 StdVideoH264PictureParameterSet in the Video Session Parameters object 399 is reached, no additional parameters of that type can be added to the 400 object. 401 ename:VK_ERROR_TOO_MANY_OBJECTS will be returned if an attempt is made 402 to add additional data to this object at this point 403 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-None-04840]] 404 Each entry to be added must: have a unique, to the rest of the parameter 405 array entries and the existing parameters in the Video Session 406 Parameters Object that is being updated, SPS-PPS IDs 407 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-None-04841]] 408 Parameter entries that already exist in Video Session Parameters object 409 with a particular SPS-PPS IDs cannot: be replaced nor updated 410 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-None-04842]] 411 When creating a new object using a Video Session Parameters as a 412 template, the array's parameters with the same SPS-PPS IDs as the ones 413 from the template take precedence 414 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoEXT-None-04843]] 415 SPS/PPS parameters must: comply with the limits specified in 416 slink:VkVideoSessionCreateInfoKHR during Video Session creation 417**** 418 419include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersAddInfoEXT.adoc[] 420-- 421 422[open,refpage='VkVideoEncodeH264SessionParametersGetInfoEXT',desc='Structure specifying parameters for retrieving encoded H.264 parameter set data',type='structs'] 423-- 424The sname:VkVideoEncodeH264SessionParametersGetInfoEXT structure is defined 425as: 426 427include::{generated}/api/structs/VkVideoEncodeH264SessionParametersGetInfoEXT.adoc[] 428 429 * pname:sType is a elink:VkStructureType value identifying this structure. 430 * pname:pNext is `NULL` or a pointer to a structure extending this 431 structure. 432 * pname:writeStdSPS indicates whether the encoded H.264 sequence parameter 433 set identified by pname:stdSPSId is requested to be retrieved. 434 * pname:writeStdPPS indicates whether the encoded H.264 picture parameter 435 set identified by the pair constructed from pname:stdSPSId and 436 pname:stdPPSId is requested to be retrieved. 437 * pname:stdSPSId specifies the H.264 sequence parameter set ID used to 438 identify the retrieved H.264 sequence and/or picture parameter set(s). 439 * pname:stdPPSId specifies the H.264 picture parameter set ID used to 440 identify the retrieved H.264 picture parameter set when 441 pname:writeStdPPS is set to ename:VK_TRUE. 442 443When this structure is specified in the pname:pNext chain of the 444slink:VkVideoEncodeSessionParametersGetInfoKHR structure passed to 445flink:vkGetEncodedVideoSessionParametersKHR, the command will write encoded 446parameter data to the output buffer in the following order: 447 448 . The H.264 sequence parameter set identified by pname:stdSPSId, if 449 pname:writeStdSPS is set to ename:VK_TRUE. 450 . The H.264 picture parameter set identified by the pair constructed from 451 pname:stdSPSId and pname:stdPPSId, if pname:writeStdPPS is set to 452 ename:VK_TRUE. 453 454include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersGetInfoEXT.adoc[] 455-- 456 457[open,refpage='VkVideoEncodeH264SessionParametersFeedbackInfoEXT',desc='Structure providing feedback about the requested H.264 video session parameters',type='structs'] 458-- 459The sname:VkVideoEncodeH264SessionParametersFeedbackInfoEXT structure is 460defined as: 461 462include::{generated}/api/structs/VkVideoEncodeH264SessionParametersFeedbackInfoEXT.adoc[] 463 464 * pname:sType is a elink:VkStructureType value identifying this structure. 465 * pname:pNext is `NULL` or a pointer to a structure extending this 466 structure. 467 * pname:hasStdSPSOverrides indicates whether any of the parameters of the 468 requested H.264 sequence parameter set, if one was requested via 469 slink:VkVideoEncodeH264SessionParametersGetInfoEXT::pname:writeStdSPS, 470 were overridden by the implementation. 471 * pname:hasStdPPSOverrides indicates whether any of the parameters of the 472 requested H.264 picture parameter set, if one was requested via 473 slink:VkVideoEncodeH264SessionParametersGetInfoEXT::pname:writeStdPPS, 474 were overridden by the implementation. 475 476include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersFeedbackInfoEXT.adoc[] 477-- 478 479 480=== Frame Encoding 481 482In order to encode a frame, add a slink:VkVideoEncodeH264PictureInfoEXT 483structure to the pname:pNext chain of the slink:VkVideoEncodeInfoKHR 484structure passed to the flink:vkCmdEncodeVideoKHR command. 485 486[open,refpage='VkVideoEncodeH264PictureInfoEXT',desc='Structure specifies H.264 encode frame parameters',type='structs'] 487-- 488The slink:VkVideoEncodeH264PictureInfoEXT structure representing a frame 489encode operation is defined as: 490 491include::{generated}/api/structs/VkVideoEncodeH264PictureInfoEXT.adoc[] 492 493 * pname:sType is a elink:VkStructureType value identifying this structure. 494 * pname:pNext is `NULL` or a pointer to a structure extending this 495 structure. 496 * pname:naluSliceEntryCount is the number of slice NALUs in the frame. 497 * pname:pNaluSliceEntries is a pointer to an array of 498 pname:naluSliceEntryCount slink:VkVideoEncodeH264NaluSliceInfoEXT 499 structures specifying the division of the current picture into slices 500 and the properties of these slices. 501 This is an ordered sequence; the NALUs are generated consecutively in 502 slink:VkVideoEncodeInfoKHR::pname:dstBuffer in the same order as in this 503 array. 504 * pname:pStdPictureInfo is a pointer to a 505 code:StdVideoEncodeH264PictureInfo structure specifying the syntax and 506 other codec-specific information from the H.264 specification associated 507 with this picture. 508 The information provided must: reflect the decoded picture marking 509 operations that are applicable to this frame. 510 * pname:generatePrefixNalu controls whether prefix NALUs are generated 511 before slice NALUs into the target bitstream. 512 513include::{generated}/validity/structs/VkVideoEncodeH264PictureInfoEXT.adoc[] 514-- 515 516[open,refpage='VkVideoEncodeH264NaluSliceInfoEXT',desc='Structure specifies H.264 encode slice NALU parameters',type='structs'] 517-- 518The slink:VkVideoEncodeH264NaluSliceInfoEXT structure representing a slice 519is defined as: 520 521include::{generated}/api/structs/VkVideoEncodeH264NaluSliceInfoEXT.adoc[] 522 523 * pname:sType is a elink:VkStructureType value identifying this structure. 524 * pname:pNext is `NULL` or a pointer to a structure extending this 525 structure. 526 * pname:constantQp is the QP to use for the slice if the current rate 527 control mode configured for the video session is 528 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR. 529 * pname:pStdSliceHeader is a pointer to a 530 code:StdVideoEncodeH264SliceHeader structure specifying the slice header 531 for the current slice. 532 533include::{generated}/validity/structs/VkVideoEncodeH264NaluSliceInfoEXT.adoc[] 534-- 535 536[open,refpage='VkVideoEncodeH264DpbSlotInfoEXT',desc='Structure specifies H.264 encode DPB picture information',type='structs'] 537-- 538The slink:VkVideoEncodeH264DpbSlotInfoEXT structure, representing a 539reconstructed picture that is being used as a reference picture, is defined 540as: 541 542include::{generated}/api/structs/VkVideoEncodeH264DpbSlotInfoEXT.adoc[] 543 544 * pname:sType is a elink:VkStructureType value identifying this structure. 545 * pname:pNext is `NULL` or a pointer to a structure extending this 546 structure. 547 * pname:pStdReferenceInfo is a pointer to a 548 code:StdVideoEncodeH264ReferenceInfo structure specifying the syntax and 549 other codec-specific information from the H.264 specification associated 550 with this reference picture. 551 552include::{generated}/validity/structs/VkVideoEncodeH264DpbSlotInfoEXT.adoc[] 553-- 554 555 556=== Rate Control 557 558[open,refpage='VkVideoEncodeH264RateControlInfoEXT',desc='Structure describing H.264 stream rate control parameters',type='structs'] 559-- 560The sname:VkVideoEncodeH264RateControlInfoEXT structure is defined as: 561 562include::{generated}/api/structs/VkVideoEncodeH264RateControlInfoEXT.adoc[] 563 564 * pname:sType is a elink:VkStructureType value identifying this structure. 565 * pname:pNext is `NULL` or a pointer to a structure extending this 566 structure. 567 * pname:flags is a bitmask of 568 elink:VkVideoEncodeH264RateControlFlagBitsEXT specifying H.264 rate 569 control flags. 570 * pname:gopFrameCount is the number of frames contained within the group 571 of pictures (GOP), starting from an intra frame and until the next intra 572 frame. 573 If it is set to 0, the implementation chooses a suitable value. 574 If it is set to code:UINT32_MAX, the GOP length is treated as infinite. 575 * pname:idrPeriod is the interval, in terms of number of frames, between 576 two IDR frames. 577 If it is set to 0, the implementation chooses a suitable value. 578 If it is set to code:UINT32_MAX, the IDR period is treated as infinite. 579 * pname:consecutiveBFrameCount is the number of consecutive B-frames 580 between I- and/or P-frames within the GOP. 581 * pname:temporalLayerCount specifies the number of temporal layers enabled 582 in the stream. 583 584In order to provide H.264-specific stream rate control parameters, add a 585sname:VkVideoEncodeH264RateControlInfoEXT structure to the pname:pNext chain 586of the slink:VkVideoEncodeRateControlInfoKHR structure in the pname:pNext 587chain of the slink:VkVideoCodingControlInfoKHR structure passed to the 588flink:vkCmdControlVideoCodingKHR command. 589 590The parameters from this structure act as a guidance for implementations to 591apply various rate control heuristics. 592 593It is possible to infer the picture type to be used when encoding a frame, 594on the basis of the values provided for pname:consecutiveBFrameCount, 595pname:idrPeriod, and pname:gopFrameCount, but this inferred picture type 596will not be used by implementations to override the picture type provided in 597flink:vkCmdEncodeVideoKHR. 598Additionally, it is not required for the video session to be reset if the 599inferred picture type does not match the actual picture type. 600 601include::{generated}/validity/structs/VkVideoEncodeH264RateControlInfoEXT.adoc[] 602-- 603 604[open,refpage='VkVideoEncodeH264RateControlFlagBitsEXT',desc='H.264 encode rate control bits',type='enums'] 605-- 606Bits which can: be set in 607slink:VkVideoEncodeH264RateControlInfoEXT::pname:flags, specifying H.264 608rate control flags, are: 609 610include::{generated}/api/enums/VkVideoEncodeH264RateControlFlagBitsEXT.adoc[] 611 612 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_ATTEMPT_HRD_COMPLIANCE_BIT_EXT 613 specifies that rate control should: attempt to produce an HRD compliant 614 bitstream. 615 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REGULAR_GOP_BIT_EXT specifies 616 that the application intends to use a regular GOP structure according to 617 the parameters specified in the pname:gopFrameCount, pname:idrPeriod, 618 and pname:consecutiveBFrameCount members of the 619 slink:VkVideoEncodeH264RateControlInfoEXT structure. 620 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REFERENCE_PATTERN_FLAT_BIT_EXT 621 specifies that the application intends to follow a flat reference 622 pattern. 623 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REFERENCE_PATTERN_DYADIC_BIT_EXT 624 specifies that the application intends to follow a dyadic reference 625 pattern. 626 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_TEMPORAL_LAYER_PATTERN_DYADIC_BIT_EXT 627 specifies that the application intends to follow a dyadic temporal layer 628 pattern. 629-- 630 631 632[open,refpage='VkVideoEncodeH264RateControlFlagsEXT',desc='Bitmask specifying H.264 encode rate control flags',type='flags'] 633-- 634include::{generated}/api/flags/VkVideoEncodeH264RateControlFlagsEXT.adoc[] 635 636tname:VkVideoEncodeH264RateControlFlagsEXT is a bitmask type for setting a 637mask of zero or more elink:VkVideoEncodeH264RateControlFlagBitsEXT. 638-- 639 640 641[open,refpage='VkVideoEncodeH264RateControlLayerInfoEXT',desc='Structure describing H.264 per-layer rate control parameters',type='structs'] 642-- 643The sname:VkVideoEncodeH264RateControlLayerInfoEXT structure is defined as: 644 645include::{generated}/api/structs/VkVideoEncodeH264RateControlLayerInfoEXT.adoc[] 646 647 * pname:sType is a elink:VkStructureType value identifying this structure. 648 * pname:pNext is `NULL` or a pointer to a structure extending this 649 structure. 650 * pname:useMinQp indicates whether the values within pname:minQp should be 651 used by the implementation. 652 When it is set to ename:VK_FALSE, the implementation ignores the values 653 in pname:minQp and chooses suitable values. 654 * pname:minQp provides the lower bound on the QP values for each picture 655 type, to be used in rate control calculations. 656 * pname:useMaxQp indicates whether the values within pname:maxQp should be 657 used by the implementation. 658 When it is set to ename:VK_FALSE, the implementation ignores the values 659 in pname:maxQp and chooses suitable values. 660 * pname:maxQp provides the upper bound on the QP values for each picture 661 type, to be used in rate control calculations. 662 * pname:useMaxFrameSize indicates whether the values within 663 pname:maxFrameSize should be used by the implementation. 664 * pname:maxFrameSize provides the upper bound on the encoded frame size 665 for each picture type. 666 The implementation does not guarantee the encoded frame sizes will be 667 within the specified limits, however these limits may: be used as a 668 guide in rate control calculations. 669 If enabled and not set properly, the pname:maxQp limit may prevent the 670 implementation from respecting the pname:maxFrameSize limit. 671 672H.264-specific per-layer rate control parameters must: be specified by 673adding a sname:VkVideoEncodeH264RateControlLayerInfoEXT structure to the 674pname:pNext chain of each slink:VkVideoEncodeRateControlLayerInfoKHR 675structure in a call to flink:vkCmdControlVideoCodingKHR command, when the 676command buffer context has an active video encode H.264 session. 677 678include::{generated}/validity/structs/VkVideoEncodeH264RateControlLayerInfoEXT.adoc[] 679-- 680 681[open,refpage='VkVideoEncodeH264QpEXT',desc='Structure describing H.264 QP values per picture type',type='structs'] 682-- 683The sname:VkVideoEncodeH264QpEXT structure is defined as: 684 685include::{generated}/api/structs/VkVideoEncodeH264QpEXT.adoc[] 686 687 * pname:qpI is the QP to be used for I-frames. 688 * pname:qpP is the QP to be used for P-frames. 689 * pname:qpB is the QP to be used for B-frames. 690 691include::{generated}/validity/structs/VkVideoEncodeH264QpEXT.adoc[] 692-- 693 694[open,refpage='VkVideoEncodeH264FrameSizeEXT',desc='Structure describing frame size values per H.264 picture type',type='structs'] 695-- 696The sname:VkVideoEncodeH264FrameSizeEXT structure is defined as: 697 698include::{generated}/api/structs/VkVideoEncodeH264FrameSizeEXT.adoc[] 699 700 * pname:frameISize is the size in bytes to be used for I-frames. 701 * pname:framePSize is the size in bytes to be used for P-frames. 702 * pname:frameBSize is the size in bytes to be used for B-frames. 703 704include::{generated}/validity/structs/VkVideoEncodeH264FrameSizeEXT.adoc[] 705-- 706 707[open,refpage='VkVideoEncodeH264GopRemainingFrameInfoEXT',desc='Structure specifying H.264 encode rate control GOP remaining frame counts',type='structs'] 708-- 709The sname:VkVideoEncodeH264GopRemainingFrameInfoEXT structure is defined as: 710 711include::{generated}/api/structs/VkVideoEncodeH264GopRemainingFrameInfoEXT.adoc[] 712 713 * pname:sType is a elink:VkStructureType value identifying this structure. 714 * pname:pNext is `NULL` or a pointer to a structure extending this 715 structure. 716 * pname:useGopRemainingFrames indicates whether the implementation's rate 717 control algorithm should: use the values specified in 718 pname:gopRemainingI, pname:gopRemainingP, and pname:gopRemainingB. 719 If pname:useGopRemainingFrames is ename:VK_FALSE, then the values of 720 pname:gopRemainingI, pname:gopRemainingP, and pname:gopRemainingB are 721 ignored. 722 * pname:gopRemainingI specifies the number of I-frames the 723 implementation's rate control algorithm should: assume to be remaining 724 in the GOP prior to executing the video encode operation. 725 * pname:gopRemainingP specifies the number of P-frames the 726 implementation's rate control algorithm should: assume to be remaining 727 in the GOP prior to executing the video encode operation. 728 * pname:gopRemainingB specifies the number of B-frames the 729 implementation's rate control algorithm should: assume to be remaining 730 in the GOP prior to executing the video encode operation. 731 732Setting pname:useGopRemainingFrames to ename:VK_TRUE and including this 733structure in the pname:pNext chain of slink:VkVideoEncodeInfoKHR is only 734mandatory if the 735slink:VkVideoEncodeH264CapabilitiesEXT::pname:requiresGopRemainingFrames 736reported for the used <<video-profiles,video profile>> is ename:VK_TRUE. 737However, implementations may: use these remaining frame counts, when 738specified, even when it is not required. 739In particular, when the application does not use a regular GOP structure, 740these values may: provide additional guidance for the implementation's rate 741control algorithm. 742 743The slink:VkVideoEncodeH264CapabilitiesEXT::pname:prefersGopRemainingFrames 744capability is also used to indicate that the implementation's rate control 745algorithm may: operate more accurately if the application specifies the 746remaining frame counts using this structure. 747 748As with other rate control guidance values, if the effective order and 749number of frames encoded by the application are not in line with the 750remaining frame counts specified in this structure at any given point, then 751the behavior of the implementation's rate control algorithm may: deviate 752from the one expected by the application. 753 754include::{generated}/validity/structs/VkVideoEncodeH264GopRemainingFrameInfoEXT.adoc[] 755-- 756