1// Copyright 2018-2024 The Khronos Group Inc. 2// 3// SPDX-License-Identifier: CC-BY-4.0 4 5[[encode-h264]] 6== H.264 Encode Operations 7 8Video encode operations using an <<encode-h264-profile,H.264 encode 9profile>> can: be used to encode elementary video stream sequences compliant 10to the <<itu-t-h264,ITU-T H.264 Specification>>. 11 12[NOTE] 13.Note 14==== 15Refer to the <<preamble, Preamble>> for information on how the Khronos 16Intellectual Property Rights Policy relates to normative references to 17external materials not created by Khronos. 18==== 19 20This process is performed according to the <<encode-operation-steps,video 21encode operation steps>> with the codec-specific semantics defined in 22section 8 of the <<itu-t-h264,ITU-T H.264 Specification>> as follows: 23 24 * Syntax elements, derived values, and other parameters are applied from 25 the following structures: 26 ** The code:StdVideoH264SequenceParameterSet structure corresponding to 27 the <<encode-h264-active-sps,active SPS>> specifying the 28 <<encode-h264-sps, H.264 sequence parameter set>>. 29 ** The code:StdVideoH264PictureParameterSet structure corresponding to the 30 <<encode-h264-active-pps,active PPS>> specifying the <<encode-h264-pps, 31 H.264 picture parameter set>>. 32 ** The code:StdVideoEncodeH264PictureInfo structure specifying the 33 <<encode-h264-picture-info,H.264 picture information>>. 34 ** The code:StdVideoEncodeH264SliceHeader structures specifying the 35 <<encode-h264-slice-header-params,H.264 slice header parameters>> for 36 each encoded H.264 slice. 37 ** The code:StdVideoEncodeH264ReferenceInfo structures specifying the 38 <<encode-h264-reference-info,H.264 reference information>> 39 corresponding to the optional <<reconstructed-picture,reconstructed 40 picture>> and any <<active-reference-pictures,active reference 41 pictures>>. 42 * The encoded bitstream data is written to the destination video bitstream 43 buffer range as defined in the 44 <<encode-h264-bitstream-data-access,H.264 Encode Bitstream Data 45 Access>> section. 46 * Picture data in the <<video-picture-resources,video picture resources>> 47 corresponding to the used <<encode-input-picture,encode input 48 picture>>, <<active-reference-pictures,active reference pictures>>, and 49 optional <<reconstructed-picture,reconstructed picture>> is accessed as 50 defined in the <<encode-h264-picture-data-access,H.264 Encode Picture 51 Data Access>> section. 52 * The decision on <<encode-ref-pic-setup,reference picture setup>> is made 53 according to the parameters specified in the 54 <<encode-h264-ref-pic-setup,H.264 picture information>>. 55 56If the parameters adhere to the syntactic and semantic requirements defined 57in the corresponding sections of the <<itu-t-h264,ITU-T H.264 58Specification>>, as described above, and the <<dpb-slot,DPB slots>> 59associated with the <<active-reference-pictures,active reference pictures>> 60all refer to <<dpb-slot-states,valid picture references>>, then the video 61encode operation will complete successfully. 62Otherwise, the video encode operation may: complete 63<<encode-unsuccessful,unsuccessfully>>. 64 65 66[[encode-h264-overrides]] 67=== H.264 Encode Parameter Overrides 68 69Implementations may: override, unless otherwise specified, any of the H.264 70encode parameters specified in the following Video Std structures: 71 72 * code:StdVideoH264SequenceParameterSet 73 * code:StdVideoH264PictureParameterSet 74 * code:StdVideoEncodeH264PictureInfo 75 * code:StdVideoEncodeH264SliceHeader 76 * code:StdVideoEncodeH264ReferenceInfo 77 78All such H.264 encode parameter overrides must: fulfill the conditions 79defined in the <<encode-overrides,Video Encode Parameter Overrides>> 80section. 81 82In addition, implementations must: not override any of the following H.264 83encode parameters: 84 85 * code:StdVideoEncodeH264PictureInfo::code:primary_pic_type 86 * code:StdVideoEncodeH264SliceHeader::code:slice_type 87 88In case of H.264 encode parameters stored in 89<<encode-h264-parameter-sets,video session parameters>> objects, 90applications need to use the flink:vkGetEncodedVideoSessionParametersKHR 91command to determine whether any implementation overrides happened. 92If the query indicates that implementation overrides were applied, then the 93application needs to retrieve and use the encoded H.264 parameter sets in 94the bitstream in order to be able to produce a compliant H.264 video 95bitstream using the H.264 encode parameters stored in the video session 96parameters object. 97 98In case of any H.264 encode parameters stored in the encoded bitstream 99produced by video encode operations, if the implementation supports the 100ename:VK_VIDEO_ENCODE_FEEDBACK_BITSTREAM_HAS_OVERRIDES_BIT_KHR 101<<queries-video-encode-feedback,video encode feedback query>> flag, the 102application can: use such queries to retrieve feedback about whether any 103implementation overrides have been applied to those H.264 encode parameters. 104 105 106[[encode-h264-bitstream-data-access]] 107=== H.264 Encode Bitstream Data Access 108 109Each video encode operation writes one or more VCL NAL units comprising of 110slice headers and data of the encoded picture, in the format defined in 111sections 7.3.3 and 7.3.4, according to the semantics defined in sections 1127.4.3 and 7.4.4 of the <<itu-t-h264,ITU-T H.264 Specification>>, 113respectively. 114The number of VCL NAL units written is specified by 115slink:VkVideoEncodeH264PictureInfoKHR::pname:naluSliceEntryCount. 116 117In addition, if 118slink:VkVideoEncodeH264PictureInfoKHR::pname:generatePrefixNalu is set to 119ename:VK_TRUE for the video encode operation, then an additional prefix NAL 120unit is written before each VCL NAL unit corresponding to individual slices 121in the format defined in section 7.3.2.12, according to the semantics 122defined in section 7.4.2.12 of the <<itu-t-h264,ITU-T H.264 Specification>>, 123respectively. 124 125 126[[encode-h264-picture-data-access]] 127=== H.264 Encode Picture Data Access 128 129Accesses to image data within a video picture resource happen at the 130granularity indicated by 131slink:VkVideoCapabilitiesKHR::pname:pictureAccessGranularity, as returned by 132flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used <<video-profiles, 133video profile>>. 134Accordingly, the complete image subregion of a <<encode-input-picture,encode 135input picture>>, <<reference-picture,reference picture>>, or 136<<reconstructed-picture,reconstructed picture>> accessed by video coding 137operations using an <<encode-h264-profile,H.264 encode profile>> is defined 138as the set of texels within the coordinate range: 139 140 {empty}:: [eq]#([0,pname:endX),[0,pname:endY))# 141 142Where: 143 144 * [eq]#pname:endX# equals [eq]#pname:codedExtent.width# rounded up to the 145 nearest integer multiple of pname:pictureAccessGranularity.width and 146 clamped to the width of the image subresource 147 <<video-image-subresource-reference,referred>> to by the corresponding 148 slink:VkVideoPictureResourceInfoKHR structure; 149 * [eq]#endY# equals [eq]#pname:codedExtent.height# rounded up to the 150 nearest integer multiple of pname:pictureAccessGranularity.height and 151 clamped to the height of the image subresource 152 <<video-image-subresource-reference,referred>> to by the corresponding 153 slink:VkVideoPictureResourceInfoKHR structure; 154 155Where pname:codedExtent is the member of the 156slink:VkVideoPictureResourceInfoKHR structure corresponding to the picture. 157 158In case of video encode operations using an <<encode-h264-profile,H.264 159encode profile>>, any access to a picture at the coordinates 160[eq]#(pname:x,pname:y)#, as defined by the <<itu-t-h264,ITU-T H.264 161Specification>>, is an access to the image subresource 162<<video-image-subresource-reference,referred>> to by the corresponding 163slink:VkVideoPictureResourceInfoKHR structure at the texel coordinates 164[eq]#(pname:x,pname:y)#. 165 166Implementations may: choose not to access some or all texels within 167particular <<reference-picture,reference pictures>> available to a video 168encode operation (e.g. due to <<encode-overrides,video encode parameter 169overrides>> restricting the effective set of used reference pictures, or if 170the encoding algorithm chooses not to use certain subregions of the 171reference picture data for sample prediction). 172 173 174[[encode-h264-frame-picture-slice]] 175=== H.264 Frame, Picture, and Slice 176 177H.264 pictures are partitioned into slices, as defined in section 6.3 of the 178<<itu-t-h264,ITU-T H.264 Specification>>. 179 180Video encode operations using an <<encode-h264-profile,H.264 encode 181profile>> can: encode slices of different types, as defined in section 7.4.3 182of the <<itu-t-h264,ITU-T H.264 Specification>>, by specifying the 183corresponding enumeration constant value in 184code:StdVideoEncodeH264SliceHeader::code:slice_type in the 185<<encode-h264-slice-header-params,H.264 slice header parameters>> from the 186Video Std enumeration type code:StdVideoH264SliceType: 187 188 * [[encode-h264-p-slice]] code:STD_VIDEO_H264_SLICE_TYPE_P indicates that 189 the slice is a _P slice_ as defined in section 3.109 of the 190 <<itu-t-h264,ITU-T H.264 Specification>>. 191 * [[encode-h264-b-slice]] code:STD_VIDEO_H264_SLICE_TYPE_B indicates that 192 the slice is a _B slice_ as defined in section 3.9 of the 193 <<itu-t-h264,ITU-T H.264 Specification>>. 194 * [[encode-h264-i-slice]] code:STD_VIDEO_H264_SLICE_TYPE_I indicates that 195 the slice is an _I slice_ as defined in section 3.66 of the 196 <<itu-t-h264,ITU-T H.264 Specification>>. 197 198Pictures constructed from such slices can: be of different types, as defined 199in section 7.4.2.4 of the <<itu-t-h264,ITU-T H.264 Specification>>. 200Video encode operations using an <<encode-h264-profile,H.264 encode 201profile>> can: encode pictures of a specific type by specifying the 202corresponding enumeration constant value in 203code:StdVideoEncodeH264PictureInfo::code:primary_pic_type in the 204<<encode-h264-picture-info,H.264 picture information>> from the Video Std 205enumeration type code:StdVideoH264PictureType: 206 207 * [[encode-h264-p-pic]] code:STD_VIDEO_H264_PICTURE_TYPE_P indicates that 208 the picture is a _P picture_. 209 A frame consisting of a P picture is also referred to as a _P frame_. 210 * [[encode-h264-b-pic]] code:STD_VIDEO_H264_PICTURE_TYPE_B indicates that 211 the picture is a _B picture_. 212 A frame consisting of a B picture is also referred to as a _B frame_. 213 * [[encode-h264-i-pic]] code:STD_VIDEO_H264_PICTURE_TYPE_I indicates that 214 the picture is an _I picture_. 215 A frame consisting of an I picture is also referred to as an _I frame_. 216 * [[encode-h264-idr-pic]] code:STD_VIDEO_H264_PICTURE_TYPE_IDR indicates 217 that the picture is a special type of I picture called an _IDR picture_ 218 as defined in section 3.69 of the <<itu-t-h264,ITU-T H.264 219 Specification>>. 220 A frame consisting of an IDR picture is also referred to as an _IDR 221 frame_. 222 223 224[[encode-h264-profile]] 225=== H.264 Encode Profile 226 227[open,refpage='VkVideoEncodeH264ProfileInfoKHR',desc='Structure specifying H.264 encode-specific video profile parameters',type='structs'] 228-- 229A video profile supporting H.264 video encode operations is specified by 230setting slink:VkVideoProfileInfoKHR::pname:videoCodecOperation to 231ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR and adding a 232sname:VkVideoEncodeH264ProfileInfoKHR structure to the 233slink:VkVideoProfileInfoKHR::pname:pNext chain. 234 235The sname:VkVideoEncodeH264ProfileInfoKHR structure is defined as: 236 237include::{generated}/api/structs/VkVideoEncodeH264ProfileInfoKHR.adoc[] 238 239 * pname:sType is a elink:VkStructureType value identifying this structure. 240 * pname:pNext is `NULL` or a pointer to a structure extending this 241 structure. 242 * pname:stdProfileIdc is a code:StdVideoH264ProfileIdc value specifying 243 the H.264 codec profile IDC, as defined in section A.2 of the 244 <<itu-t-h264,ITU-T H.264 Specification>>. 245 246include::{generated}/validity/structs/VkVideoEncodeH264ProfileInfoKHR.adoc[] 247-- 248 249 250=== H.264 Encode Capabilities 251 252[open,refpage='VkVideoEncodeH264CapabilitiesKHR',desc='Structure describing H.264 encode capabilities',type='structs'] 253-- 254When calling flink:vkGetPhysicalDeviceVideoCapabilitiesKHR to query the 255capabilities for an <<encode-h264-profile,H.264 encode profile>>, the 256slink:VkVideoCapabilitiesKHR::pname:pNext chain must: include a 257sname:VkVideoEncodeH264CapabilitiesKHR structure that will be filled with 258the profile-specific capabilities. 259 260The sname:VkVideoEncodeH264CapabilitiesKHR structure is defined as: 261 262include::{generated}/api/structs/VkVideoEncodeH264CapabilitiesKHR.adoc[] 263 264 * pname:sType is a elink:VkStructureType value identifying this structure. 265 * pname:pNext is `NULL` or a pointer to a structure extending this 266 structure. 267 * pname:flags is a bitmask of elink:VkVideoEncodeH264CapabilityFlagBitsKHR 268 indicating supported H.264 encoding capabilities. 269 * pname:maxLevelIdc is a code:StdVideoH264LevelIdc value indicating the 270 maximum H.264 level supported by the profile, where enum constant 271 `STD_VIDEO_H264_LEVEL_IDC_<major>_<minor>` identifies H.264 level 272 `<major>.<minor>` as defined in section A.3 of the <<itu-t-h264,ITU-T 273 H.264 Specification>>. 274 * pname:maxSliceCount indicates the maximum number of slices that can: be 275 encoded for a single picture. 276 Further restrictions may: apply to the number of slices that can: be 277 encoded for a single picture depending on other capabilities and 278 codec-specific rules. 279 * pname:maxPPictureL0ReferenceCount indicates the maximum number of 280 reference pictures the implementation supports in the reference list L0 281 for <<encode-h264-p-pic,P pictures>>. 282+ 283[NOTE] 284.Note 285==== 286As implementations may: <<encode-overrides,override>> the reference lists, 287pname:maxPPictureL0ReferenceCount does not limit the number of elements that 288the application can: specify in the L0 reference list for P pictures. 289However, if pname:maxPPictureL0ReferenceCount is zero, then the use of P 290pictures is not allowed. 291==== 292 * pname:maxBPictureL0ReferenceCount indicates the maximum number of 293 reference pictures the implementation supports in the reference list L0 294 for <<encode-h264-b-pic,B pictures>>. 295 * pname:maxL1ReferenceCount indicates the maximum number of reference 296 pictures the implementation supports in the reference list L1 if 297 encoding of <<encode-h264-b-pic,B pictures>> is supported. 298+ 299[NOTE] 300.Note 301==== 302As implementations may: <<encode-overrides,override>> the reference lists, 303pname:maxBPictureL0ReferenceCount and pname:maxL1ReferenceCount does not 304limit the number of elements that the application can: specify in the L0 and 305L1 reference lists for B pictures. 306However, if pname:maxBPictureL0ReferenceCount and pname:maxL1ReferenceCount 307are both zero, then the use of B pictures is not allowed. 308==== 309 * pname:maxTemporalLayerCount indicates the maximum number of H.264 310 temporal layers supported by the implementation. 311 * pname:expectDyadicTemporalLayerPattern indicates that the 312 implementation's rate control algorithms expect the application to use a 313 <<encode-h264-layer-pattern-dyadic,dyadic temporal layer pattern>> when 314 encoding multiple temporal layers. 315 * pname:minQp indicates the minimum QP value supported. 316 * pname:maxQp indicates the maximum QP value supported. 317 * pname:prefersGopRemainingFrames indicates that the implementation's rate 318 control algorithm prefers the application to specify the number of 319 frames of each type <<encode-h264-gop-remaining-frames,remaining>> in 320 the current <<encode-h264-gop,group of pictures>> when beginning a 321 <<video-coding-scope,video coding scope>>. 322 * pname:requiresGopRemainingFrames indicates that the implementation's 323 rate control algorithm requires the application to specify the number of 324 frames of each type <<encode-h264-gop-remaining-frames,remaining>> in 325 the current <<encode-h264-gop,group of pictures>> when beginning a 326 <<video-coding-scope,video coding scope>>. 327 * pname:stdSyntaxFlags is a bitmask of 328 elink:VkVideoEncodeH264StdFlagBitsKHR indicating capabilities related to 329 H.264 syntax elements. 330 331include::{generated}/validity/structs/VkVideoEncodeH264CapabilitiesKHR.adoc[] 332-- 333 334 335[open,refpage='VkVideoEncodeH264CapabilityFlagBitsKHR',desc='H.264 encode capability flags',type='enums'] 336-- 337Bits which may: be set in 338slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags, indicating the H.264 339encoding capabilities supported, are: 340 341include::{generated}/api/enums/VkVideoEncodeH264CapabilityFlagBitsKHR.adoc[] 342 343 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_HRD_COMPLIANCE_BIT_KHR indicates 344 whether the implementation may: be able to generate HRD compliant 345 bitstreams if any of the code:nal_hrd_parameters_present_flag or 346 code:vcl_hrd_parameters_present_flag members of 347 code:StdVideoH264SpsVuiFlags are set to `1` in the 348 <<encode-h264-active-sps,active SPS>>. 349 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PREDICTION_WEIGHT_TABLE_GENERATED_BIT_KHR 350 indicates that if code:StdVideoH264PpsFlags::code:weighted_pred_flag is 351 set to `1` or 352 code:StdVideoH264PictureParameterSet::code:weighted_bipred_idc is set to 353 code:STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_EXPLICIT in the 354 <<encode-h264-active-pps,active PPS>> when encoding a 355 <<encode-h264-p-pic,P picture>> or <<encode-h264-b-pic,B picture>>, 356 respectively, then the implementation is able to internally decide 357 syntax for code:pred_weight_table, as defined in section 7.4.3.2 of the 358 <<itu-t-h264,ITU-T H.264 Specification>>, and the application is not 359 required: to provide a weight table in the 360 <<encode-h264-slice-header-params,H.264 slice header parameters>>. 361 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_ROW_UNALIGNED_SLICE_BIT_KHR 362 indicates that each slice in a frame with multiple slices may begin or 363 finish at any offset in a macroblock row. 364 If not supported, all slices in the frame must: begin at the start of a 365 macroblock row (and hence each slice must: finish at the end of a 366 macroblock row). 367 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DIFFERENT_SLICE_TYPE_BIT_KHR 368 indicates that when a frame is encoded with multiple slices, the 369 implementation allows encoding each slice with a different 370 code:StdVideoEncodeH264SliceHeader::code:slice_type specified in the 371 <<encode-h264-slice-header-params,H.264 slice header parameters>>. 372 If not supported, all slices of the frame must: be encoded with the same 373 code:slice_type which corresponds to the picture type of the frame. 374 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_B_FRAME_IN_L0_LIST_BIT_KHR 375 indicates support for using a <<encode-h264-b-pic,B frame>> as L0 376 reference, as specified in 377 code:StdVideoEncodeH264ReferenceListsInfo::code:RefPicList0 in the 378 <<encode-h264-picture-info,H.264 picture information>>. 379 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_B_FRAME_IN_L1_LIST_BIT_KHR 380 indicates support for using a <<encode-h264-b-pic,B frame>> as L1 381 reference, as specified in 382 code:StdVideoEncodeH264ReferenceListsInfo::code:RefPicList1 in the 383 <<encode-h264-picture-info,H.264 picture information>>. 384 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PER_PICTURE_TYPE_MIN_MAX_QP_BIT_KHR 385 indicates support for specifying different QP values in the members of 386 slink:VkVideoEncodeH264QpKHR. 387 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PER_SLICE_CONSTANT_QP_BIT_KHR 388 indicates support for specifying different constant QP values for each 389 slice. 390 * ename:VK_VIDEO_ENCODE_H264_CAPABILITY_GENERATE_PREFIX_NALU_BIT_KHR 391 indicates support for generating prefix NAL units by setting 392 slink:VkVideoEncodeH264PictureInfoKHR::pname:generatePrefixNalu to 393 ename:VK_TRUE. 394-- 395 396[open,refpage='VkVideoEncodeH264CapabilityFlagsKHR',desc='Bitmask of VkVideoEncodeH264CapabilityFlagBitsKHR',type='flags'] 397-- 398include::{generated}/api/flags/VkVideoEncodeH264CapabilityFlagsKHR.adoc[] 399 400tname:VkVideoEncodeH264CapabilityFlagsKHR is a bitmask type for setting a 401mask of zero or more elink:VkVideoEncodeH264CapabilityFlagBitsKHR. 402-- 403 404[open,refpage='VkVideoEncodeH264StdFlagBitsKHR',desc='Video encode H.264 syntax capability flags',type='enums'] 405-- 406Bits which may: be set in 407slink:VkVideoEncodeH264CapabilitiesKHR::pname:stdSyntaxFlags, indicating the 408capabilities related to the H.264 syntax elements, are: 409 410include::{generated}/api/enums/VkVideoEncodeH264StdFlagBitsKHR.adoc[] 411 412 * ename:VK_VIDEO_ENCODE_H264_STD_SEPARATE_COLOR_PLANE_FLAG_SET_BIT_KHR 413 indicates whether the implementation supports using the 414 application-provided value for 415 code:StdVideoH264SpsFlags::code:separate_colour_plane_flag in the 416 <<encode-h264-sps,SPS>> when that value is `1`. 417 * ename:VK_VIDEO_ENCODE_H264_STD_QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG_SET_BIT_KHR 418 indicates whether the implementation supports using the 419 application-provided value for 420 code:StdVideoH264SpsFlags::code:qpprime_y_zero_transform_bypass_flag in 421 the <<encode-h264-sps,SPS>> when that value is `1`. 422 * ename:VK_VIDEO_ENCODE_H264_STD_SCALING_MATRIX_PRESENT_FLAG_SET_BIT_KHR 423 indicates whether the implementation supports using the 424 application-provided values for 425 code:StdVideoH264SpsFlags::code:seq_scaling_matrix_present_flag in the 426 <<encode-h264-sps,SPS>> and 427 code:StdVideoH264PpsFlags::code:pic_scaling_matrix_present_flag in the 428 <<encode-h264-pps,PPS>> when any of those values are `1`. 429 * ename:VK_VIDEO_ENCODE_H264_STD_CHROMA_QP_INDEX_OFFSET_BIT_KHR indicates 430 whether the implementation supports using the application-provided value 431 for code:StdVideoH264PictureParameterSet::code:chroma_qp_index_offset in 432 the <<encode-h264-pps,PPS>> when that value is non-zero. 433 * ename:VK_VIDEO_ENCODE_H264_STD_SECOND_CHROMA_QP_INDEX_OFFSET_BIT_KHR 434 indicates whether the implementation supports using the 435 application-provided value for 436 code:StdVideoH264PictureParameterSet::code:second_chroma_qp_index_offset 437 in the <<encode-h264-pps,PPS>> when that value is non-zero. 438 * ename:VK_VIDEO_ENCODE_H264_STD_PIC_INIT_QP_MINUS26_BIT_KHR indicates 439 whether the implementation supports using the application-provided value 440 for code:StdVideoH264PictureParameterSet::code:pic_init_qp_minus26 in 441 the <<encode-h264-pps,PPS>> when that value is non-zero. 442 * ename:VK_VIDEO_ENCODE_H264_STD_WEIGHTED_PRED_FLAG_SET_BIT_KHR indicates 443 whether the implementation supports using the application-provided value 444 for code:StdVideoH264PpsFlags::code:weighted_pred_flag in the 445 <<encode-h264-pps,PPS>> when that value is `1`. 446 * ename:VK_VIDEO_ENCODE_H264_STD_WEIGHTED_BIPRED_IDC_EXPLICIT_BIT_KHR 447 indicates whether the implementation supports using the 448 application-provided value for 449 code:StdVideoH264PictureParameterSet::code:weighted_bipred_idc in the 450 <<encode-h264-pps,PPS>> when that value is 451 code:STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_EXPLICIT. 452 * ename:VK_VIDEO_ENCODE_H264_STD_WEIGHTED_BIPRED_IDC_IMPLICIT_BIT_KHR 453 indicates whether the implementation supports using the 454 application-provided value for 455 code:StdVideoH264PictureParameterSet::code:weighted_bipred_idc in the 456 <<encode-h264-pps,PPS>> when that value is 457 code:STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_IMPLICIT. 458 * ename:VK_VIDEO_ENCODE_H264_STD_TRANSFORM_8X8_MODE_FLAG_SET_BIT_KHR 459 indicates whether the implementation supports using the 460 application-provided value for 461 code:StdVideoH264PpsFlags::code:transform_8x8_mode_flag in the 462 <<encode-h264-pps,PPS>> when that value is `1`. 463 * ename:VK_VIDEO_ENCODE_H264_STD_DIRECT_SPATIAL_MV_PRED_FLAG_UNSET_BIT_KHR 464 indicates whether the implementation supports using the 465 application-provided value for 466 code:StdVideoEncodeH264SliceHeaderFlags::code:direct_spatial_mv_pred_flag 467 in the <<encode-h264-slice-header-params,H.264 slice header parameters>> 468 when that value is `0`. 469 * ename:VK_VIDEO_ENCODE_H264_STD_ENTROPY_CODING_MODE_FLAG_UNSET_BIT_KHR 470 indicates whether the implementation supports CAVLC entropy coding, as 471 defined in section 9.2 of the <<itu-t-h264,ITU-T H.264 Specification>>, 472 and thus supports using the application-provided value for 473 code:StdVideoH264PpsFlags::code:entropy_coding_mode_flag in the 474 <<encode-h264-pps,PPS>> when that value is `0`. 475 * ename:VK_VIDEO_ENCODE_H264_STD_ENTROPY_CODING_MODE_FLAG_SET_BIT_KHR 476 indicates whether the implementation supports CABAC entropy coding, as 477 defined in section 9.3 of the <<itu-t-h264,ITU-T H.264 Specification>>, 478 and thus supports using the application-provided value for 479 code:StdVideoH264PpsFlags::code:entropy_coding_mode_flag in the 480 <<encode-h264-pps,PPS>> when that value is `1`. 481 * ename:VK_VIDEO_ENCODE_H264_STD_DIRECT_8X8_INFERENCE_FLAG_UNSET_BIT_KHR 482 indicates whether the implementation supports using the 483 application-provided value for 484 code:StdVideoH264SpsFlags::code:direct_8x8_inference_flag in the 485 <<encode-h264-sps,SPS>> when that value is `0`. 486 * ename:VK_VIDEO_ENCODE_H264_STD_CONSTRAINED_INTRA_PRED_FLAG_SET_BIT_KHR 487 indicates whether the implementation supports using the 488 application-provided value for 489 code:StdVideoH264PpsFlags::code:constrained_intra_pred_flag in the 490 <<encode-h264-pps,PPS>> when that value is `1`. 491 * ename:VK_VIDEO_ENCODE_H264_STD_DEBLOCKING_FILTER_DISABLED_BIT_KHR 492 indicates whether the implementation supports using the 493 application-provided value for 494 code:StdVideoEncodeH264SliceHeader::code:disable_deblocking_filter_idc 495 in the <<encode-h264-slice-header-params,H.264 slice header parameters>> 496 when that value is 497 code:STD_VIDEO_H264_DISABLE_DEBLOCKING_FILTER_IDC_DISABLED. 498 * ename:VK_VIDEO_ENCODE_H264_STD_DEBLOCKING_FILTER_ENABLED_BIT_KHR 499 indicates whether the implementation supports using the 500 application-provided value for 501 code:StdVideoEncodeH264SliceHeader::code:disable_deblocking_filter_idc 502 in the <<encode-h264-slice-header-params,H.264 slice header parameters>> 503 when that value is 504 code:STD_VIDEO_H264_DISABLE_DEBLOCKING_FILTER_IDC_ENABLED. 505 * ename:VK_VIDEO_ENCODE_H264_STD_DEBLOCKING_FILTER_PARTIAL_BIT_KHR 506 indicates whether the implementation supports using the 507 application-provided value for 508 code:StdVideoEncodeH264SliceHeader::code:disable_deblocking_filter_idc 509 in the <<encode-h264-slice-header-params,H.264 slice header parameters>> 510 when that value is 511 code:STD_VIDEO_H264_DISABLE_DEBLOCKING_FILTER_IDC_PARTIAL. 512 * ename:VK_VIDEO_ENCODE_H264_STD_SLICE_QP_DELTA_BIT_KHR indicates whether 513 the implementation supports using the application-provided value for 514 code:StdVideoEncodeH264SliceHeader::code:slice_qp_delta in the 515 <<encode-h264-slice-header-params,H.264 slice header parameters>> when 516 that value is identical across the slices of the encoded frame. 517 * ename:VK_VIDEO_ENCODE_H264_STD_DIFFERENT_SLICE_QP_DELTA_BIT_KHR 518 indicates whether the implementation supports using the 519 application-provided value for 520 code:StdVideoEncodeH264SliceHeader::code:slice_qp_delta in the 521 <<encode-h264-slice-header-params,H.264 slice header parameters>> when 522 that value is different across the slices of the encoded frame. 523 524These capability flags provide information to the application about specific 525H.264 syntax element values that the implementation supports without having 526to <<encode-h264-overrides,override>> them and do not otherwise restrict the 527values that the application can: specify for any of the mentioned H.264 528syntax elements. 529-- 530 531[open,refpage='VkVideoEncodeH264StdFlagsKHR',desc='Bitmask of VkVideoEncodeH264StdFlagBitsKHR',type='flags'] 532-- 533include::{generated}/api/flags/VkVideoEncodeH264StdFlagsKHR.adoc[] 534 535tname:VkVideoEncodeH264StdFlagsKHR is a bitmask type for setting a mask of 536zero or more elink:VkVideoEncodeH264StdFlagBitsKHR. 537-- 538 539 540=== H.264 Encode Quality Level Properties 541 542[open,refpage='VkVideoEncodeH264QualityLevelPropertiesKHR',desc='Structure describing the H.264 encode quality level properties',type='structs'] 543-- 544When calling flink:vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR 545with pname:pVideoProfile->videoCodecOperation specified as 546ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, the 547slink:VkVideoEncodeH264QualityLevelPropertiesKHR structure must: be included 548in the pname:pNext chain of the slink:VkVideoEncodeQualityLevelPropertiesKHR 549structure to retrieve additional video encode quality level properties 550specific to H.264 encoding. 551 552The slink:VkVideoEncodeH264QualityLevelPropertiesKHR structure is defined 553as: 554 555include::{generated}/api/structs/VkVideoEncodeH264QualityLevelPropertiesKHR.adoc[] 556 557 * pname:sType is a elink:VkStructureType value identifying this structure. 558 * pname:pNext is `NULL` or a pointer to a structure extending this 559 structure. 560 * pname:preferredRateControlFlags is a bitmask of 561 elink:VkVideoEncodeH264RateControlFlagBitsKHR values indicating the 562 preferred flags to use for 563 slink:VkVideoEncodeH264RateControlInfoKHR::pname:flags. 564 * pname:preferredGopFrameCount indicates the preferred value to use for 565 slink:VkVideoEncodeH264RateControlInfoKHR::pname:gopFrameCount. 566 * pname:preferredIdrPeriod indicates the preferred value to use for 567 slink:VkVideoEncodeH264RateControlInfoKHR::pname:idrPeriod. 568 * pname:preferredConsecutiveBFrameCount indicates the preferred value to 569 use for 570 slink:VkVideoEncodeH264RateControlInfoKHR::pname:consecutiveBFrameCount. 571 * pname:preferredTemporalLayerCount indicates the preferred value to use 572 for slink:VkVideoEncodeH264RateControlInfoKHR::pname:temporalLayerCount. 573 * pname:preferredConstantQp indicates the preferred values to use for 574 slink:VkVideoEncodeH264NaluSliceInfoKHR::pname:constantQp for each 575 picture type when using <<encode-rate-control-modes,rate control mode>> 576 ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR. 577 * pname:preferredMaxL0ReferenceCount indicates the preferred maximum 578 number of reference pictures to use in the reference list L0. 579 * pname:preferredMaxL1ReferenceCount indicates the preferred maximum 580 number of reference pictures to use in the reference list L1. 581 * pname:preferredStdEntropyCodingModeFlag indicates the preferred value to 582 use for code:entropy_coding_mode_flag in code:StdVideoH264PpsFlags. 583 584include::{generated}/validity/structs/VkVideoEncodeH264QualityLevelPropertiesKHR.adoc[] 585-- 586 587 588=== H.264 Encode Session 589 590Additional parameters can be specified when creating a video session with an 591H.264 encode profile by including an instance of the 592slink:VkVideoEncodeH264SessionCreateInfoKHR structure in the pname:pNext 593chain of slink:VkVideoSessionCreateInfoKHR. 594 595[open,refpage='VkVideoEncodeH264SessionCreateInfoKHR',desc='Structure specifies H.264 encode session parameters',type='structs'] 596-- 597The sname:VkVideoEncodeH264SessionCreateInfoKHR structure is defined as: 598 599include::{generated}/api/structs/VkVideoEncodeH264SessionCreateInfoKHR.adoc[] 600 601 * pname:sType is a elink:VkStructureType value identifying this structure. 602 * pname:pNext is `NULL` or a pointer to a structure extending this 603 structure. 604 * pname:useMaxLevelIdc indicates whether the value of pname:maxLevelIdc 605 should be used by the implementation. 606 When it is set to ename:VK_FALSE, the implementation ignores the value 607 of pname:maxLevelIdc and uses the value of 608 slink:VkVideoEncodeH264CapabilitiesKHR::pname:maxLevelIdc, as reported 609 by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the video profile. 610 * pname:maxLevelIdc is a code:StdVideoH264LevelIdc value specifying the 611 upper bound on the H.264 level for the video bitstreams produced by the 612 created video session, where enum constant 613 `STD_VIDEO_H264_LEVEL_IDC_<major>_<minor>` identifies H.264 level 614 `<major>.<minor>` as defined in section A.3 of the <<itu-t-h264,ITU-T 615 H.264 Specification>>. 616 617include::{generated}/validity/structs/VkVideoEncodeH264SessionCreateInfoKHR.adoc[] 618-- 619 620 621[[encode-h264-parameter-sets]] 622=== H.264 Encode Parameter Sets 623 624<<video-session-parameters,Video session parameters>> objects created with 625the video codec operation ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR 626can: contain the following types of parameters: 627 628:operationType: encode 629include::{chapters}/video/h264_parameter_sets.adoc[] 630 631Implementations may: override any of these parameters according to the 632semantics defined in the <<encode-overrides,Video Encode Parameter 633Overrides>> section before storing the resulting H.264 parameter sets into 634the video session parameters object. 635Applications need to use the flink:vkGetEncodedVideoSessionParametersKHR 636command to determine whether any implementation overrides happened and to 637retrieve the encoded H.264 parameter sets in order to be able to produce a 638compliant H.264 video bitstream. 639 640Such H.264 parameter set overrides may: also have cascading effects on the 641implementation overrides applied to the encoded bitstream produced by video 642encode operations. 643If the implementation supports the 644ename:VK_VIDEO_ENCODE_FEEDBACK_BITSTREAM_HAS_OVERRIDES_BIT_KHR 645<<queries-video-encode-feedback,video encode feedback query>> flag, then the 646application can: use such queries to retrieve feedback about whether any 647implementation overrides have been applied to the encoded bitstream. 648 649[open,refpage='VkVideoEncodeH264SessionParametersCreateInfoKHR',desc='Structure specifies H.264 encoder parameter set information',type='structs'] 650-- 651When a <<video-session-parameters,video session parameters>> object is 652created with the codec operation 653ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, the 654slink:VkVideoSessionParametersCreateInfoKHR::pname:pNext chain must: include 655a sname:VkVideoEncodeH264SessionParametersCreateInfoKHR structure specifying 656the capacity and initial contents of the object. 657 658The sname:VkVideoEncodeH264SessionParametersCreateInfoKHR structure is 659defined as: 660 661include::{generated}/api/structs/VkVideoEncodeH264SessionParametersCreateInfoKHR.adoc[] 662 663 * pname:sType is a elink:VkStructureType value identifying this structure. 664 * pname:pNext is `NULL` or a pointer to a structure extending this 665 structure. 666 * pname:maxStdSPSCount is the maximum number of <<encode-h264-sps,H.264 667 SPS>> entries the created sname:VkVideoSessionParametersKHR can: 668 contain. 669 * pname:maxStdPPSCount is the maximum number of <<encode-h264-pps,H.264 670 PPS>> entries the created sname:VkVideoSessionParametersKHR can: 671 contain. 672 * pname:pParametersAddInfo is `NULL` or a pointer to a 673 slink:VkVideoEncodeH264SessionParametersAddInfoKHR structure specifying 674 H.264 parameters to add upon object creation. 675 676include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersCreateInfoKHR.adoc[] 677-- 678 679[open,refpage='VkVideoEncodeH264SessionParametersAddInfoKHR',desc='Structure specifies H.264 encoder parameter set information',type='structs'] 680-- 681The sname:VkVideoEncodeH264SessionParametersAddInfoKHR structure is defined 682as: 683 684include::{generated}/api/structs/VkVideoEncodeH264SessionParametersAddInfoKHR.adoc[] 685 686 * pname:sType is a elink:VkStructureType value identifying this structure. 687 * pname:pNext is `NULL` or a pointer to a structure extending this 688 structure. 689 * pname:stdSPSCount is the number of elements in the pname:pStdSPSs array. 690 * pname:pStdSPSs is a pointer to an array of 691 code:StdVideoH264SequenceParameterSet structures describing the 692 <<encode-h264-sps,H.264 SPS>> entries to add. 693 * pname:stdPPSCount is the number of elements in the pname:pStdPPSs array. 694 * pname:pStdPPSs is a pointer to an array of 695 code:StdVideoH264PictureParameterSet structures describing the 696 <<encode-h264-pps,H.264 PPS>> entries to add. 697 698This structure can: be specified in the following places: 699 700 * In the pname:pParametersAddInfo member of the 701 slink:VkVideoEncodeH264SessionParametersCreateInfoKHR structure 702 specified in the pname:pNext chain of 703 slink:VkVideoSessionParametersCreateInfoKHR used to create a 704 <<video-session-parameters,video session parameters>> object. 705 In this case, if the video codec operation the video session parameters 706 object is created with is 707 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then it defines the 708 set of initial parameters to add to the created object (see 709 <<creating-video-session-parameters,Creating Video Session 710 Parameters>>). 711 * In the pname:pNext chain of slink:VkVideoSessionParametersUpdateInfoKHR. 712 In this case, if the video codec operation the 713 <<video-session-parameters,video session parameters>> object to be 714 updated was created with is 715 ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, then it defines the 716 set of parameters to add to it (see 717 <<video-session-parameters-update,Updating Video Session Parameters>>). 718 719.Valid Usage 720**** 721 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoKHR-None-04837]] 722 The pname:seq_parameter_set_id member of each 723 code:StdVideoH264SequenceParameterSet structure specified in the 724 elements of pname:pStdSPSs must: be unique within pname:pStdSPSs 725 * [[VUID-VkVideoEncodeH264SessionParametersAddInfoKHR-None-04838]] 726 The pair constructed from the pname:seq_parameter_set_id and 727 pname:pic_parameter_set_id members of each 728 code:StdVideoH264PictureParameterSet structure specified in the elements 729 of pname:pStdPPSs must: be unique within pname:pStdPPSs 730**** 731 732include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersAddInfoKHR.adoc[] 733-- 734 735[open,refpage='VkVideoEncodeH264SessionParametersGetInfoKHR',desc='Structure specifying parameters for retrieving encoded H.264 parameter set data',type='structs'] 736-- 737The sname:VkVideoEncodeH264SessionParametersGetInfoKHR structure is defined 738as: 739 740include::{generated}/api/structs/VkVideoEncodeH264SessionParametersGetInfoKHR.adoc[] 741 742 * pname:sType is a elink:VkStructureType value identifying this structure. 743 * pname:pNext is `NULL` or a pointer to a structure extending this 744 structure. 745 * pname:writeStdSPS indicates whether the encoded <<encode-h264-sps,H.264 746 sequence parameter set>> identified by pname:stdSPSId is requested to be 747 retrieved. 748 * pname:writeStdPPS indicates whether the encoded <<encode-h264-pps,H.264 749 picture parameter set>> identified by the pair constructed from 750 pname:stdSPSId and pname:stdPPSId is requested to be retrieved. 751 * pname:stdSPSId specifies the H.264 sequence parameter set ID used to 752 identify the retrieved H.264 sequence and/or picture parameter set(s). 753 * pname:stdPPSId specifies the H.264 picture parameter set ID used to 754 identify the retrieved H.264 picture parameter set when 755 pname:writeStdPPS is set to ename:VK_TRUE. 756 757When this structure is specified in the pname:pNext chain of the 758slink:VkVideoEncodeSessionParametersGetInfoKHR structure passed to 759flink:vkGetEncodedVideoSessionParametersKHR, the command will write encoded 760parameter data to the output buffer in the following order: 761 762 . The <<encode-h264-sps,H.264 sequence parameter set>> identified by 763 pname:stdSPSId, if pname:writeStdSPS is set to ename:VK_TRUE. 764 . The <<encode-h264-pps,H.264 picture parameter set>> identified by the 765 pair constructed from pname:stdSPSId and pname:stdPPSId, if 766 pname:writeStdPPS is set to ename:VK_TRUE. 767 768.Valid Usage 769**** 770 * [[VUID-VkVideoEncodeH264SessionParametersGetInfoKHR-writeStdSPS-08279]] 771 At least one of pname:writeStdSPS and pname:writeStdPPS must: be set to 772 ename:VK_TRUE 773**** 774 775include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersGetInfoKHR.adoc[] 776-- 777 778[open,refpage='VkVideoEncodeH264SessionParametersFeedbackInfoKHR',desc='Structure providing feedback about the requested H.264 video session parameters',type='structs'] 779-- 780The sname:VkVideoEncodeH264SessionParametersFeedbackInfoKHR structure is 781defined as: 782 783include::{generated}/api/structs/VkVideoEncodeH264SessionParametersFeedbackInfoKHR.adoc[] 784 785 * pname:sType is a elink:VkStructureType value identifying this structure. 786 * pname:pNext is `NULL` or a pointer to a structure extending this 787 structure. 788 * pname:hasStdSPSOverrides indicates whether any of the parameters of the 789 requested <<encode-h264-sps,H.264 sequence parameter set>>, if one was 790 requested via 791 slink:VkVideoEncodeH264SessionParametersGetInfoKHR::pname:writeStdSPS, 792 were <<encode-overrides,overridden>> by the implementation. 793 * pname:hasStdPPSOverrides indicates whether any of the parameters of the 794 requested <<encode-h264-pps,H.264 picture parameter set>>, if one was 795 requested via 796 slink:VkVideoEncodeH264SessionParametersGetInfoKHR::pname:writeStdPPS, 797 were <<encode-overrides,overridden>> by the implementation. 798 799include::{generated}/validity/structs/VkVideoEncodeH264SessionParametersFeedbackInfoKHR.adoc[] 800-- 801 802=== H.264 Encoding Parameters 803 804[open,refpage='VkVideoEncodeH264PictureInfoKHR',desc='Structure specifies H.264 encode frame parameters',type='structs'] 805-- 806The slink:VkVideoEncodeH264PictureInfoKHR structure is defined as: 807 808include::{generated}/api/structs/VkVideoEncodeH264PictureInfoKHR.adoc[] 809 810 * pname:sType is a elink:VkStructureType value identifying this structure. 811 * pname:pNext is `NULL` or a pointer to a structure extending this 812 structure. 813 * pname:naluSliceEntryCount is the number of elements in 814 pname:pNaluSliceEntries. 815 * pname:pNaluSliceEntries is a pointer to an array of 816 pname:naluSliceEntryCount slink:VkVideoEncodeH264NaluSliceInfoKHR 817 structures specifying the parameters of the individual H.264 slices to 818 encode for the input picture. 819 * pname:pStdPictureInfo is a pointer to a 820 code:StdVideoEncodeH264PictureInfo structure specifying 821 <<encode-h264-picture-info,H.264 picture information>>. 822 * pname:generatePrefixNalu controls whether prefix NALUs are generated 823 before slice NALUs into the target bitstream, as defined in sections 824 7.3.2.12 and 7.4.2.12 of the <<itu-t-h264,ITU-T H.264 Specification>>. 825 826This structure is specified in the pname:pNext chain of the 827slink:VkVideoEncodeInfoKHR structure passed to flink:vkCmdEncodeVideoKHR to 828specify the codec-specific picture information for an <<encode-h264,H.264 829encode operation>>. 830 831[[encode-h264-input-picture-info]] 832Encode Input Picture Information:: 833 834When this structure is specified in the pname:pNext chain of the 835slink:VkVideoEncodeInfoKHR structure passed to flink:vkCmdEncodeVideoKHR, 836the information related to the <<encode-input-picture-info,encode input 837picture>> is defined as follows: 838 839 * The image subregion used is determined according to the 840 <<encode-h264-picture-data-access,H.264 Encode Picture Data Access>> 841 section. 842 * The encode input picture is associated with the 843 <<encode-h264-picture-info,H.264 picture information>> provided in 844 pname:pStdPictureInfo. 845 846[[encode-h264-picture-info]] 847Std Picture Information:: 848 849The members of the code:StdVideoEncodeH264PictureInfo structure pointed to 850by pname:pStdPictureInfo are interpreted as follows: 851 852 * code:flags.reserved and code:reserved1 are used only for padding 853 purposes and are otherwise ignored; 854 * code:flags.IdrPicFlag as defined in section 7.4.1 of the <<itu-t-h264, 855 ITU-T H.264 Specification>>; 856 * code:flags.is_reference as defined in section 3.136 of the <<itu-t-h264, 857 ITU-T H.264 Specification>>; 858 * code:seq_parameter_set_id and code:pic_parameter_set_id are used to 859 identify the active parameter sets, as described below; 860 * code:primary_pic_type as defined in section 7.4.2 of the <<itu-t-h264, 861 ITU-T H.264 Specification>>; 862 * code:PicOrderCnt as defined in section 8.2 of the <<itu-t-h264,ITU-T 863 H.264 Specification>>; 864 * code:temporal_id as defined in section G.7.4.1.1 of the <<itu-t-h264, 865 ITU-T H.264 Specification>>; 866 * if code:pRefLists is not `NULL`, then it is a pointer to a 867 code:StdVideoEncodeH264ReferenceListsInfo structure that is interpreted 868 as follows: 869 ** code:flags.reserved is used only for padding purposes and is otherwise 870 ignored; 871 ** code:ref_pic_list_modification_flag_l0 and 872 code:ref_pic_list_modification_flag_l1 as defined in section 7.4.3.1 of 873 the <<itu-t-h264, ITU-T H.264 Specification>>; 874 ** code:num_ref_idx_l0_active_minus1 and code:num_ref_idx_l1_active_minus1 875 as defined in section 7.4.3 of the <<itu-t-h264, ITU-T H.264 876 Specification>>; 877 ** code:RefPicList0 and code:RefPicList1 as defined in section 8.2.4 of 878 the <<itu-t-h264, ITU-T H.264 Specification>> where each element of 879 these arrays either identifies an 880 <<encode-active-reference-picture-info,active reference picture>> using 881 its <<dpb-slot,DPB slot>> index or contains the value 882 code:STD_VIDEO_H264_NO_REFERENCE_PICTURE to indicate "`no reference 883 picture`"; 884 ** if code:refList0ModOpCount is not zero, then 885 code:pRefList0ModOperations is a pointer to an array of 886 code:refList0ModOpCount number of 887 code:StdVideoEncodeH264RefListModEntry structures specifying the 888 modification parameters for the reference list L0 as defined in section 889 7.4.3.1 of the <<itu-t-h264, ITU-T H.264 Specification>>; 890 ** if code:refList1ModOpCount is not zero, then 891 code:pRefList1ModOperations is a pointer to an array of 892 code:refList1ModOpCount number of 893 code:StdVideoEncodeH264RefListModEntry structures specifying the 894 modification parameters for the reference list L1 as defined in section 895 7.4.3.1 of the <<itu-t-h264, ITU-T H.264 Specification>>; 896 ** if code:refPicMarkingOpCount is not zero, then 897 code:refPicMarkingOperations is a pointer to an array of 898 code:refPicMarkingOpCount number of 899 code:StdVideoEncodeH264RefPicMarkingEntry structures specifying the 900 reference picture marking parameters as defined in section 7.4.3.3 of 901 the <<itu-t-h264, ITU-T H.264 Specification>>; 902 * all other members are interpreted as defined in section 7.4.3 of the 903 <<itu-t-h264,ITU-T H.264 Specification>>. 904 905[[encode-h264-ref-pic-setup]] 906Reference picture setup is controlled by the value of 907code:StdVideoEncodeH264PictureInfo::pname:flags.is_reference. 908If it is set and a <<encode-reconstructed-picture-info,reconstructed 909picture>> is specified, then the latter is used as the target of picture 910reconstruction to <<dpb-slot-states,activate>> the <<dpb-slot,DPB slot>> 911specified in pname:pEncodeInfo->pSetupReferenceSlot->slotIndex. 912If code:StdVideoEncodeH264PictureInfo::pname:flags.is_reference is not set, 913but a <<encode-reconstructed-picture-info,reconstructed picture>> is 914specified, then the corresponding picture reference associated with the 915<<dpb-slot,DPB slot>> is invalidated, as described in the 916<<dpb-slot-states,DPB Slot States>> section. 917 918Active Parameter Sets:: 919 920The members of the code:StdVideoEncodeH264PictureInfo structure pointed to 921by pname:pStdPictureInfo are used to select the active parameter sets to use 922from the bound video session parameters object, as follows: 923 924 * [[encode-h264-active-sps]] The _active SPS_ is the 925 <<encode-h264-sps,SPS>> identified by the key specified in 926 code:StdVideoEncodeH264PictureInfo::code:seq_parameter_set_id. 927 * [[encode-h264-active-pps]] The _active PPS_ is the 928 <<encode-h264-pps,PPS>> identified by the key specified by the pair 929 constructed from 930 code:StdVideoEncodeH264PictureInfo::code:seq_parameter_set_id and 931 code:StdVideoEncodeH264PictureInfo::code:pic_parameter_set_id. 932 933[[encode-h264-weighted-pred]] 934H.264 encoding uses _explicit weighted sample prediction_ for a slice, as 935defined in section 8.4.2.3 of the <<itu-t-h264,ITU-T H.264 Specification>>, 936if any of the following conditions are true for the active 937<<encode-h264-pps,PPS>> and the pname:pStdSliceHeader member of the 938corresponding element of pname:pNaluSliceEntries: 939 940 * pname:pStdSliceHeader->slice_type is code:STD_VIDEO_H264_SLICE_TYPE_P 941 and code:weighted_pred_flag is enabled in the active PPS. 942 * pname:pStdSliceHeader->slice_type is code:STD_VIDEO_H264_SLICE_TYPE_B 943 and code:weighted_bipred_idc in the active PPS equals 944 code:STD_VIDEO_H264_WEIGHTED_BIPRED_IDC_EXPLICIT. 945 946.Valid Usage 947**** 948 * [[VUID-VkVideoEncodeH264PictureInfoKHR-naluSliceEntryCount-08301]] 949 pname:naluSliceEntryCount must: be between `1` and 950 slink:VkVideoEncodeH264CapabilitiesKHR::pname:maxSliceCount, inclusive, 951 as returned by flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the 952 used video profile 953 * [[VUID-VkVideoEncodeH264PictureInfoKHR-flags-08304]] 954 If slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags, as returned by 955 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video 956 profile, does not include 957 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_GENERATE_PREFIX_NALU_BIT_KHR, then 958 pname:generatePrefixNalu must: be ename:VK_FALSE 959 * [[VUID-VkVideoEncodeH264PictureInfoKHR-flags-08314]] 960 If slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags, as returned by 961 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video 962 profile, does not include 963 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PREDICTION_WEIGHT_TABLE_GENERATED_BIT_KHR 964 and the slice corresponding to any element of pname:pNaluSliceEntries 965 uses <<encode-h264-weighted-pred,explicit weighted sample prediction>>, 966 then 967 slink:VkVideoEncodeH264NaluSliceInfoKHR::pname:pStdSliceHeader->pWeightTable 968 must: not be `NULL` for that element of pname:pNaluSliceEntries 969 * [[VUID-VkVideoEncodeH264PictureInfoKHR-flags-08315]] 970 If slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags, as returned by 971 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video 972 profile, does not include 973 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_DIFFERENT_SLICE_TYPE_BIT_KHR, then 974 slink:VkVideoEncodeH264NaluSliceInfoKHR::pname:pStdSliceHeader->slice_type 975 must: be identical for all elements of pname:pNaluSliceEntries 976**** 977 978include::{generated}/validity/structs/VkVideoEncodeH264PictureInfoKHR.adoc[] 979-- 980 981[open,refpage='VkVideoEncodeH264NaluSliceInfoKHR',desc='Structure specifies H.264 encode slice NALU parameters',type='structs'] 982-- 983The slink:VkVideoEncodeH264NaluSliceInfoKHR structure is defined as: 984 985include::{generated}/api/structs/VkVideoEncodeH264NaluSliceInfoKHR.adoc[] 986 987 * pname:sType is a elink:VkStructureType value identifying this structure. 988 * pname:pNext is `NULL` or a pointer to a structure extending this 989 structure. 990 * pname:constantQp is the QP to use for the slice if the current 991 <<encode-rate-control-modes,rate control mode>> configured for the video 992 session is ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR. 993 * pname:pStdSliceHeader is a pointer to a 994 code:StdVideoEncodeH264SliceHeader structure specifying 995 <<encode-h264-slice-header-params,H.264 slice header parameters>> for 996 the slice. 997 998[[encode-h264-slice-header-params]] 999Std Slice Header Parameters:: 1000 1001The members of the code:StdVideoEncodeH264SliceHeader structure pointed to 1002by pname:pStdSliceHeader are interpreted as follows: 1003 1004 * code:flags.reserved and code:reserved1 are used only for padding 1005 purposes and are otherwise ignored; 1006 * if pname:pWeightTable is not `NULL`, then it is a pointer to a 1007 code:StdVideoEncodeH264WeightTable that is interpreted as follows: 1008 ** code:flags.reserved is used only for padding purposes and is otherwise 1009 ignored; 1010 ** all other members of code:StdVideoEncodeH264WeightTable are interpreted 1011 as defined in section 7.4.3.2 of the <<itu-t-h264,ITU-T H.264 1012 Specification>>; 1013 * all other members are interpreted as defined in section 7.4.3 of the 1014 <<itu-t-h264,ITU-T H.264 Specification>>. 1015 1016include::{generated}/validity/structs/VkVideoEncodeH264NaluSliceInfoKHR.adoc[] 1017-- 1018 1019[open,refpage='VkVideoEncodeH264DpbSlotInfoKHR',desc='Structure specifies H.264 encode DPB picture information',type='structs'] 1020-- 1021The slink:VkVideoEncodeH264DpbSlotInfoKHR structure is defined as: 1022 1023include::{generated}/api/structs/VkVideoEncodeH264DpbSlotInfoKHR.adoc[] 1024 1025 * pname:sType is a elink:VkStructureType value identifying this structure. 1026 * pname:pNext is `NULL` or a pointer to a structure extending this 1027 structure. 1028 * pname:pStdReferenceInfo is a pointer to a 1029 code:StdVideoEncodeH264ReferenceInfo structure specifying 1030 <<encode-h264-reference-info,H.264 reference information>>. 1031 1032This structure is specified in the pname:pNext chain of 1033slink:VkVideoEncodeInfoKHR::pname:pSetupReferenceSlot, if not `NULL`, and 1034the pname:pNext chain of the elements of 1035slink:VkVideoEncodeInfoKHR::pname:pReferenceSlots to specify the 1036codec-specific reference picture information for an <<encode-h264,H.264 1037encode operation>>. 1038 1039[[encode-h264-active-reference-picture-info]] 1040Active Reference Picture Information:: 1041 1042When this structure is specified in the pname:pNext chain of the elements of 1043slink:VkVideoEncodeInfoKHR::pname:pReferenceSlots, one element is added to 1044the list of <<encode-active-reference-picture-info,active reference 1045pictures>> used by the video encode operation for each element of 1046slink:VkVideoEncodeInfoKHR::pname:pReferenceSlots as follows: 1047 1048 * The image subregion used is determined according to the 1049 <<encode-h264-picture-data-access,H.264 Encode Picture Data Access>> 1050 section. 1051 * The reference picture is associated with the <<dpb-slot,DPB slot>> index 1052 specified in the pname:slotIndex member of the corresponding element of 1053 slink:VkVideoEncodeInfoKHR::pname:pReferenceSlots. 1054 * The reference picture is associated with the 1055 <<encode-h264-reference-info,H.264 reference information>> provided in 1056 pname:pStdReferenceInfo. 1057 1058[[encode-h264-reconstructed-picture-info]] 1059Reconstructed Picture Information:: 1060 1061When this structure is specified in the pname:pNext chain of 1062slink:VkVideoEncodeInfoKHR::pname:pSetupReferenceSlot, the information 1063related to the <<encode-reconstructed-picture-info,reconstructed picture>> 1064is defined as follows: 1065 1066 * The image subregion used is determined according to the 1067 <<encode-h264-picture-data-access,H.264 Encode Picture Data Access>> 1068 section. 1069 * If <<encode-h264-ref-pic-setup,reference picture setup>> is requested, 1070 then the reconstructed picture is used to <<dpb-slot-states,activate>> 1071 the <<dpb-slot,DPB slot>> with the index specified in 1072 slink:VkVideoEncodeInfoKHR::pname:pSetupReferenceSlot->slotIndex. 1073 * The reconstructed picture is associated with the 1074 <<encode-h264-reference-info,H.264 reference information>> provided in 1075 pname:pStdReferenceInfo. 1076 1077[[encode-h264-reference-info]] 1078Std Reference Information:: 1079 1080The members of the code:StdVideoEncodeH264ReferenceInfo structure pointed to 1081by pname:pStdReferenceInfo are interpreted as follows: 1082 1083 * code:flags.reserved is used only for padding purposes and is otherwise 1084 ignored; 1085 * code:flags.used_for_long_term_reference is used to indicate whether the 1086 picture is marked as "`used for long-term reference`" as defined in 1087 section 8.2.5.1 of the <<itu-t-h264,ITU-T H.264 Specification>>; 1088 * code:primary_pic_type as defined in section 7.4.2 of the <<itu-t-h264, 1089 ITU-T H.264 Specification>>; 1090 * code:long_term_pic_num and code:long_term_frame_idx as defined in 1091 section 7.4.3 of the <<itu-t-h264,ITU-T H.264 Specification>>; 1092 * code:temporal_id as defined in section G.7.4.1.1 of the <<itu-t-h264, 1093 ITU-T H.264 Specification>>; 1094 * all other members are interpreted as defined in section 8.2 of the 1095 <<itu-t-h264,ITU-T H.264 Specification>>. 1096 1097include::{generated}/validity/structs/VkVideoEncodeH264DpbSlotInfoKHR.adoc[] 1098-- 1099 1100 1101[[encode-h264-rate-control]] 1102=== H.264 Encode Rate Control 1103 1104[[encode-h264-gop]] 1105==== Group of Pictures 1106 1107In case of H.264 encoding it is common practice to follow a regular pattern 1108of different picture types in display order when encoding subsequent frames. 1109This pattern is referred to as the _group of pictures_ (GOP). 1110 1111[[encode-h264-regular-gop]] 1112A regular GOP is defined by the following parameters: 1113 1114 * The number of frames in the GOP; 1115 * The number of consecutive B frames between I and/or P frames in display 1116 order. 1117 1118GOPs are further classified as _open_ and _closed_ GOPs. 1119 1120Frame types in an open GOP follow each other in display order according to 1121the following algorithm: 1122 1123 1. The first frame is always an I frame. 1124 2. This is followed by a number of consecutive B frames, as defined above. 1125 3. If the number of frames in the GOP is not reached yet, then the next 1126 frame is a P frame and the algorithm continues from step 2. 1127 1128[[encode-h264-open-gop]] 1129image::{images}/h26x_open_gop.svg[align="center",title="H.264 open GOP",opts="{imageopts}"] 1130 1131[[encode-h264-idr-period]] 1132In case of a closed GOP, an <<encode-h264-idr-pic,IDR frame>> is used at a 1133certain period. 1134 1135[[encode-h264-closed-gop]] 1136image::{images}/h26x_closed_gop.svg[align="center",title="H.264 closed GOP",opts="{imageopts}"] 1137 1138It is also typical for H.264 encoding to use specific reference picture 1139usage patterns across the frames of the GOP. 1140The two most common reference patterns used are as follows: 1141 1142[[encode-h264-ref-pattern-flat]] 1143Flat Reference Pattern:: 1144 1145 * Each P frame uses the last non-B frame, in display order, as reference. 1146 * Each B frame uses the last non-B frame, in display order, as its 1147 backward reference, and uses the next non-B frame, in display order, as 1148 its forward reference. 1149 1150image::{images}/h26x_ref_pattern_flat.svg[align="center",title="H.264 flat reference pattern",opts="{imageopts}"] 1151 1152[[encode-h264-ref-pattern-dyadic]] 1153Dyadic Reference Pattern:: 1154 1155 * Each P frame uses the last non-B frame, in display order, as reference. 1156 * The following algorithm is applied to the sequence of consecutive B 1157 frames between I and/or P frames in display order: 1158 1159 . The B frame in the middle of this sequence uses the frame preceding the 1160 sequence as its backward reference, and uses the frame following the 1161 sequence as its forward reference. 1162 . The algorithm is executed recursively for the following frame sequences: 1163 ** The B frames of the original sequence preceding the frame in the 1164 middle, if any. 1165 ** The B frames of the original sequence following the frame in the 1166 middle, if any. 1167 1168image::{images}/h26x_ref_pattern_dyadic.svg[align="center",title="H.264 dyadic reference pattern",opts="{imageopts}"] 1169 1170The application can: provide guidance to the implementation's rate control 1171algorithm about the structure of the GOP used by the application. 1172Any such guidance about the GOP and its structure does not mandate that 1173specific GOP structure to be used by the application, as the picture type of 1174individual encoded pictures is still application-controlled, however, any 1175deviation from the provided guidance may: result in undesired rate control 1176behavior including, but not limited, to the implementation not being able to 1177conform to the expected average or target bitrates, or other rate control 1178parameters specified by the application. 1179 1180When an H.264 encode session is used to encode multiple temporal layers, it 1181is also common practice to follow a regular pattern for the H.264 temporal 1182ID for the encoded pictures in display order when encoding subsequent 1183frames. 1184This pattern is referred to as the _temporal GOP_. 1185The most common temporal layer pattern used is as follows: 1186 1187[[encode-h264-layer-pattern-dyadic]] 1188Dyadic Temporal Layer Pattern:: 1189 1190 * The number of frames in the temporal GOP is [eq]#2^n-1^#, where [eq]#n# 1191 is the number of temporal layers. 1192 * The [eq]#i#^th^ frame in the temporal GOP uses temporal ID [eq]#t#, if 1193 and only if the index of the least significant bit set in [eq]#i# equals 1194 [eq]#n-t-1#, except for the first frame, which is the only frame in the 1195 temporal GOP using temporal ID zero. 1196 * The [eq]#i#^th^ frame in the temporal GOP uses the [eq]#r#^th^ frame as 1197 reference, where [eq]#r# is calculated from [eq]#i# by clearing the 1198 least significant bit set in it, except for the first frame in the 1199 temporal GOP, which uses the first frame of the previous temporal GOP, 1200 if any, as reference. 1201 1202image::{images}/h26x_layer_pattern_dyadic.svg[align="center",title="H.264 dyadic temporal layer pattern",opts="{imageopts}"] 1203 1204[NOTE] 1205.Note 1206==== 1207Multi-layer rate control and multi-layer coding are typically used for 1208streaming cases where low latency is expected, hence B pictures with forward 1209prediction are usually not used. 1210==== 1211 1212[open,refpage='VkVideoEncodeH264RateControlInfoKHR',desc='Structure describing H.264 stream rate control parameters',type='structs'] 1213-- 1214The sname:VkVideoEncodeH264RateControlInfoKHR structure is defined as: 1215 1216include::{generated}/api/structs/VkVideoEncodeH264RateControlInfoKHR.adoc[] 1217 1218 * pname:sType is a elink:VkStructureType value identifying this structure. 1219 * pname:pNext is `NULL` or a pointer to a structure extending this 1220 structure. 1221 * pname:flags is a bitmask of 1222 elink:VkVideoEncodeH264RateControlFlagBitsKHR specifying H.264 rate 1223 control flags. 1224 * pname:gopFrameCount is the number of frames within a <<encode-h264-gop, 1225 group of pictures (GOP)>> intended to be used by the application. 1226 If it is set to 0, the rate control algorithm may: assume an 1227 implementation-dependent GOP length. 1228 If it is set to code:UINT32_MAX, the GOP length is treated as infinite. 1229 * pname:idrPeriod is the interval, in terms of number of frames, between 1230 two <<encode-h264-idr-pic,IDR frames>> (see <<encode-h264-idr-period,IDR 1231 period>>). 1232 If it is set to 0, the rate control algorithm may: assume an 1233 implementation-dependent IDR period. 1234 If it is set to code:UINT32_MAX, the IDR period is treated as infinite. 1235 * pname:consecutiveBFrameCount is the number of consecutive B frames 1236 between I and/or P frames within the <<encode-h264-gop,GOP>>. 1237 * pname:temporalLayerCount specifies the number of H.264 temporal layers 1238 that the application intends to use. 1239 1240When an instance of this structure is included in the pname:pNext chain of 1241the slink:VkVideoCodingControlInfoKHR structure passed to the 1242flink:vkCmdControlVideoCodingKHR command, and 1243slink:VkVideoCodingControlInfoKHR::pname:flags includes 1244ename:VK_VIDEO_CODING_CONTROL_ENCODE_RATE_CONTROL_BIT_KHR, the parameters in 1245this structure are used as guidance for the implementation's rate control 1246algorithm (see <<video-coding-control,Video Coding Control>>). 1247 1248If pname:flags includes 1249ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_ATTEMPT_HRD_COMPLIANCE_BIT_KHR, then 1250the rate control state is reset to an initial state to meet HRD compliance 1251requirements. 1252Otherwise the new rate control state may: be applied without a reset 1253depending on the implementation and the specified rate control parameters. 1254 1255[NOTE] 1256.Note 1257==== 1258It would be possible to infer the picture type to be used when encoding a 1259frame, on the basis of the values provided for pname:consecutiveBFrameCount, 1260pname:idrPeriod, and pname:gopFrameCount, but this inferred picture type 1261will not be used by implementations to override the picture type provided to 1262the video encode operation. 1263==== 1264 1265.Valid Usage 1266**** 1267 * [[VUID-VkVideoEncodeH264RateControlInfoKHR-flags-08280]] 1268 If slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags, as returned by 1269 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video 1270 profile, does not include 1271 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_HRD_COMPLIANCE_BIT_KHR, then 1272 pname:flags must: not contain 1273 ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_ATTEMPT_HRD_COMPLIANCE_BIT_KHR 1274 * [[VUID-VkVideoEncodeH264RateControlInfoKHR-flags-08281]] 1275 If pname:flags contains 1276 ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REFERENCE_PATTERN_FLAT_BIT_KHR 1277 or 1278 ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REFERENCE_PATTERN_DYADIC_BIT_KHR, 1279 then it must: also contain 1280 ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REGULAR_GOP_BIT_KHR 1281 * [[VUID-VkVideoEncodeH264RateControlInfoKHR-flags-08282]] 1282 If pname:flags contains 1283 ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REFERENCE_PATTERN_FLAT_BIT_KHR, 1284 then it must: not also contain 1285 ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REFERENCE_PATTERN_DYADIC_BIT_KHR 1286 * [[VUID-VkVideoEncodeH264RateControlInfoKHR-flags-08283]] 1287 If pname:flags contains 1288 ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REGULAR_GOP_BIT_KHR, then 1289 pname:gopFrameCount must: be greater than `0` 1290 * [[VUID-VkVideoEncodeH264RateControlInfoKHR-idrPeriod-08284]] 1291 If pname:idrPeriod is not `0`, then it must: be greater than or equal to 1292 pname:gopFrameCount 1293 * [[VUID-VkVideoEncodeH264RateControlInfoKHR-consecutiveBFrameCount-08285]] 1294 If pname:consecutiveBFrameCount is not `0`, then it must: be less than 1295 pname:gopFrameCount 1296**** 1297 1298include::{generated}/validity/structs/VkVideoEncodeH264RateControlInfoKHR.adoc[] 1299-- 1300 1301[open,refpage='VkVideoEncodeH264RateControlFlagBitsKHR',desc='H.264 encode rate control bits',type='enums'] 1302-- 1303Bits which can: be set in 1304slink:VkVideoEncodeH264RateControlInfoKHR::pname:flags, specifying H.264 1305rate control flags, are: 1306 1307include::{generated}/api/enums/VkVideoEncodeH264RateControlFlagBitsKHR.adoc[] 1308 1309 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_ATTEMPT_HRD_COMPLIANCE_BIT_KHR 1310 specifies that rate control should: attempt to produce an HRD compliant 1311 bitstream, as defined in annex C of the <<itu-t-h264,ITU-T H.264 1312 Specification>>. 1313 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REGULAR_GOP_BIT_KHR specifies 1314 that the application intends to use a <<encode-h264-regular-gop, regular 1315 GOP structure>> according to the parameters specified in the 1316 pname:gopFrameCount, pname:idrPeriod, and pname:consecutiveBFrameCount 1317 members of the slink:VkVideoEncodeH264RateControlInfoKHR structure. 1318 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REFERENCE_PATTERN_FLAT_BIT_KHR 1319 specifies that the application intends to follow a 1320 <<encode-h264-ref-pattern-flat,flat reference pattern>> in the GOP. 1321 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_REFERENCE_PATTERN_DYADIC_BIT_KHR 1322 specifies that the application intends to follow a 1323 <<encode-h264-ref-pattern-dyadic,dyadic reference pattern>> in the GOP. 1324 * ename:VK_VIDEO_ENCODE_H264_RATE_CONTROL_TEMPORAL_LAYER_PATTERN_DYADIC_BIT_KHR 1325 specifies that the application intends to follow a 1326 <<encode-h264-layer-pattern-dyadic,dyadic temporal layer pattern>>. 1327-- 1328 1329 1330[open,refpage='VkVideoEncodeH264RateControlFlagsKHR',desc='Bitmask specifying H.264 encode rate control flags',type='flags'] 1331-- 1332include::{generated}/api/flags/VkVideoEncodeH264RateControlFlagsKHR.adoc[] 1333 1334tname:VkVideoEncodeH264RateControlFlagsKHR is a bitmask type for setting a 1335mask of zero or more elink:VkVideoEncodeH264RateControlFlagBitsKHR. 1336-- 1337 1338 1339[[encode-h264-rate-control-layer]] 1340==== Rate Control Layers 1341 1342[open,refpage='VkVideoEncodeH264RateControlLayerInfoKHR',desc='Structure describing H.264 per-layer rate control parameters',type='structs'] 1343-- 1344The sname:VkVideoEncodeH264RateControlLayerInfoKHR structure is defined as: 1345 1346include::{generated}/api/structs/VkVideoEncodeH264RateControlLayerInfoKHR.adoc[] 1347 1348 * pname:sType is a elink:VkStructureType value identifying this structure. 1349 * pname:pNext is `NULL` or a pointer to a structure extending this 1350 structure. 1351 * pname:useMinQp indicates whether the QP values determined by rate 1352 control will be clamped to the lower bounds on the QP values specified 1353 in pname:minQp. 1354 * pname:minQp specifies the lower bounds on the QP values, for each 1355 picture type, that the implementation's rate control algorithm will use 1356 when pname:useMinQp is set to ename:VK_TRUE. 1357 * pname:useMaxQp indicates whether the QP values determined by rate 1358 control will be clamped to the upper bounds on the QP values specified 1359 in pname:maxQp. 1360 * pname:maxQp specifies the upper bounds on the QP values, for each 1361 picture type, that the implementation's rate control algorithm will use 1362 when pname:useMaxQp is set to ename:VK_TRUE. 1363 * pname:useMaxFrameSize indicates whether the implementation's rate 1364 control algorithm should: use the values specified in pname:maxFrameSize 1365 as the upper bounds on the encoded frame size for each picture type. 1366 * pname:maxFrameSize specifies the upper bounds on the encoded frame size, 1367 for each picture type, when pname:useMaxFrameSize is set to 1368 ename:VK_TRUE. 1369 1370When used, the values in pname:minQp and pname:maxQp guarantee that the 1371effective QP values used by the implementation will respect those lower and 1372upper bounds, respectively. 1373However, limiting the range of QP values that the implementation is able to 1374use will also limit the capabilities of the implementation's rate control 1375algorithm to comply to other constraints. 1376In particular, the implementation may: not be able to comply to the 1377following: 1378 1379 * The average and/or peak <<encode-bitrate,bitrate>> values to be used for 1380 the encoded bitstream specified in the pname:averageBitrate and 1381 pname:maxBitrate members of the 1382 slink:VkVideoEncodeRateControlLayerInfoKHR structure. 1383 * The upper bounds on the encoded frame size, for each picture type, 1384 specified in the pname:maxFrameSize member of 1385 sname:VkVideoEncodeH264RateControlLayerInfoKHR. 1386 1387[NOTE] 1388.Note 1389==== 1390In general, applications need to configure rate control parameters 1391appropriately in order to be able to get the desired rate control behavior, 1392as described in the <<encode-rate-control,Video Encode Rate Control>> 1393section. 1394==== 1395 1396When an instance of this structure is included in the pname:pNext chain of a 1397slink:VkVideoEncodeRateControlLayerInfoKHR structure specified in one of the 1398elements of the pname:pLayers array member of the 1399slink:VkVideoEncodeRateControlInfoKHR structure passed to the 1400flink:vkCmdControlVideoCodingKHR command, 1401slink:VkVideoCodingControlInfoKHR::pname:flags includes 1402ename:VK_VIDEO_CODING_CONTROL_ENCODE_RATE_CONTROL_BIT_KHR, and the bound 1403video session was created with the video codec operation 1404ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, it specifies the 1405H.264-specific rate control parameters of the rate control layer 1406corresponding to that element of pname:pLayers. 1407 1408.Valid Usage 1409**** 1410 * [[VUID-VkVideoEncodeH264RateControlLayerInfoKHR-useMinQp-08286]] 1411 If pname:useMinQp is ename:VK_TRUE, then the pname:qpI, pname:qpP, and 1412 pname:qpB members of pname:minQp must: all be between 1413 slink:VkVideoEncodeH264CapabilitiesKHR::pname:minQp and 1414 slink:VkVideoEncodeH264CapabilitiesKHR::pname:maxQp, as returned by 1415 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video profile 1416 * [[VUID-VkVideoEncodeH264RateControlLayerInfoKHR-useMaxQp-08287]] 1417 If pname:useMaxQp is ename:VK_TRUE, then the pname:qpI, pname:qpP, and 1418 pname:qpB members of pname:maxQp must: all be between 1419 slink:VkVideoEncodeH264CapabilitiesKHR::pname:minQp and 1420 slink:VkVideoEncodeH264CapabilitiesKHR::pname:maxQp, as returned by 1421 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video profile 1422 * [[VUID-VkVideoEncodeH264RateControlLayerInfoKHR-useMinQp-08288]] 1423 If pname:useMinQp is ename:VK_TRUE and 1424 slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags, as returned by 1425 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video 1426 profile, does not include 1427 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PER_PICTURE_TYPE_MIN_MAX_QP_BIT_KHR, 1428 then the pname:qpI, pname:qpP, and pname:qpB members of pname:minQp 1429 must: all specify the same value 1430 * [[VUID-VkVideoEncodeH264RateControlLayerInfoKHR-useMaxQp-08289]] 1431 If pname:useMaxQp is ename:VK_TRUE and 1432 slink:VkVideoEncodeH264CapabilitiesKHR::pname:flags, as returned by 1433 flink:vkGetPhysicalDeviceVideoCapabilitiesKHR for the used video 1434 profile, does not include 1435 ename:VK_VIDEO_ENCODE_H264_CAPABILITY_PER_PICTURE_TYPE_MIN_MAX_QP_BIT_KHR, 1436 then the pname:qpI, pname:qpP, and pname:qpB members of pname:maxQp 1437 must: all specify the same value 1438 * [[VUID-VkVideoEncodeH264RateControlLayerInfoKHR-useMinQp-08374]] 1439 If pname:useMinQp and pname:useMaxQp are both ename:VK_TRUE, then the 1440 pname:qpI, pname:qpP, and pname:qpB members of pname:minQp must: all be 1441 less than or equal to the respective members of pname:maxQp 1442**** 1443 1444include::{generated}/validity/structs/VkVideoEncodeH264RateControlLayerInfoKHR.adoc[] 1445-- 1446 1447[open,refpage='VkVideoEncodeH264QpKHR',desc='Structure describing H.264 QP values per picture type',type='structs'] 1448-- 1449The sname:VkVideoEncodeH264QpKHR structure is defined as: 1450 1451include::{generated}/api/structs/VkVideoEncodeH264QpKHR.adoc[] 1452 1453 * pname:qpI is the QP to be used for <<encode-h264-i-pic,I pictures>>. 1454 * pname:qpP is the QP to be used for <<encode-h264-p-pic,P pictures>>. 1455 * pname:qpB is the QP to be used for <<encode-h264-b-pic,B pictures>>. 1456 1457include::{generated}/validity/structs/VkVideoEncodeH264QpKHR.adoc[] 1458-- 1459 1460[open,refpage='VkVideoEncodeH264FrameSizeKHR',desc='Structure describing frame size values per H.264 picture type',type='structs'] 1461-- 1462The sname:VkVideoEncodeH264FrameSizeKHR structure is defined as: 1463 1464include::{generated}/api/structs/VkVideoEncodeH264FrameSizeKHR.adoc[] 1465 1466 * pname:frameISize is the size in bytes to be used for 1467 <<encode-h264-i-pic,I pictures>>. 1468 * pname:framePSize is the size in bytes to be used for 1469 <<encode-h264-p-pic,P pictures>>. 1470 * pname:frameBSize is the size in bytes to be used for 1471 <<encode-h264-b-pic,B pictures>>. 1472 1473include::{generated}/validity/structs/VkVideoEncodeH264FrameSizeKHR.adoc[] 1474-- 1475 1476 1477[[encode-h264-gop-remaining-frames]] 1478==== GOP Remaining Frames 1479 1480Besides session level rate control configuration, the application can: 1481specify the number of frames per frame type remaining in the 1482<<encode-h264-gop,group of pictures (GOP)>>. 1483 1484[open,refpage='VkVideoEncodeH264GopRemainingFrameInfoKHR',desc='Structure specifying H.264 encode rate control GOP remaining frame counts',type='structs'] 1485-- 1486The sname:VkVideoEncodeH264GopRemainingFrameInfoKHR structure is defined as: 1487 1488include::{generated}/api/structs/VkVideoEncodeH264GopRemainingFrameInfoKHR.adoc[] 1489 1490 * pname:sType is a elink:VkStructureType value identifying this structure. 1491 * pname:pNext is `NULL` or a pointer to a structure extending this 1492 structure. 1493 * pname:useGopRemainingFrames indicates whether the implementation's rate 1494 control algorithm should: use the values specified in 1495 pname:gopRemainingI, pname:gopRemainingP, and pname:gopRemainingB. 1496 If pname:useGopRemainingFrames is ename:VK_FALSE, then the values of 1497 pname:gopRemainingI, pname:gopRemainingP, and pname:gopRemainingB are 1498 ignored. 1499 * pname:gopRemainingI specifies the number of <<encode-h264-i-pic,I 1500 frames>> the implementation's rate control algorithm should: assume to 1501 be remaining in the <<encode-h264-gop,GOP>> prior to executing the video 1502 encode operation. 1503 * pname:gopRemainingP specifies the number of <<encode-h264-p-pic,P 1504 frames>> the implementation's rate control algorithm should: assume to 1505 be remaining in the <<encode-h264-gop,GOP>> prior to executing the video 1506 encode operation. 1507 * pname:gopRemainingB specifies the number of <<encode-h264-b-pic,B 1508 frames>> the implementation's rate control algorithm should: assume to 1509 be remaining in the <<encode-h264-gop,GOP>> prior to executing the video 1510 encode operation. 1511 1512Setting pname:useGopRemainingFrames to ename:VK_TRUE and including this 1513structure in the pname:pNext chain of slink:VkVideoBeginCodingInfoKHR is 1514only mandatory if the 1515slink:VkVideoEncodeH264CapabilitiesKHR::pname:requiresGopRemainingFrames 1516reported for the used <<video-profiles,video profile>> is ename:VK_TRUE. 1517However, implementations may: use these remaining frame counts, when 1518specified, even when it is not required. 1519In particular, when the application does not use a 1520<<encode-h264-regular-gop,regular GOP structure>>, these values may: provide 1521additional guidance for the implementation's rate control algorithm. 1522 1523The slink:VkVideoEncodeH264CapabilitiesKHR::pname:prefersGopRemainingFrames 1524capability is also used to indicate that the implementation's rate control 1525algorithm may: operate more accurately if the application specifies the 1526remaining frame counts using this structure. 1527 1528As with other rate control guidance values, if the effective order and 1529number of frames encoded by the application are not in line with the 1530remaining frame counts specified in this structure at any given point, then 1531the behavior of the implementation's rate control algorithm may: deviate 1532from the one expected by the application. 1533 1534include::{generated}/validity/structs/VkVideoEncodeH264GopRemainingFrameInfoKHR.adoc[] 1535-- 1536 1537 1538[[encode-h264-requirements]] 1539=== H.264 Encode Requirements 1540 1541This section described the required: H.264 encoding capabilities for 1542physical devices that have at least one queue family that supports the video 1543codec operation ename:VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR, as 1544returned by flink:vkGetPhysicalDeviceQueueFamilyProperties2 in 1545slink:VkQueueFamilyVideoPropertiesKHR::pname:videoCodecOperations. 1546 1547.Required <<video-std-header-version,Video Std Header Versions>> 1548[options="header"] 1549|==== 1550| Video Std Header Name | Version 1551| `vulkan_video_codec_h264std_encode` | 1.0.0 1552|==== 1553 1554.Required Video Capabilities 1555[width="100%",cols="<35,<14,<11",options="header"] 1556|==== 1557| Video Capability | Requirement | Requirement Type^1^ 1558| **slink:VkVideoCapabilitiesKHR** | | 1559| pname:flags | - | min 1560| pname:minBitstreamBufferOffsetAlignment | 4096 | max 1561| pname:minBitstreamBufferSizeAlignment | 4096 | max 1562| pname:pictureAccessGranularity | (64,64) | max 1563| pname:minCodedExtent | - | max 1564| pname:maxCodedExtent | - | min 1565| pname:maxDpbSlots | 0 | min 1566| pname:maxActiveReferencePictures | 0 | min 1567| **slink:VkVideoEncodeCapabilitiesKHR** | | 1568| pname:flags | - | min 1569| pname:rateControlModes | - | min 1570| pname:maxBitrate | 64000 | min 1571| pname:maxQualityLevels | 1 | min 1572| pname:encodeInputPictureGranularity | (64,64) | max 1573| pname:supportedEncodeFeedbackFlags | ename:VK_VIDEO_ENCODE_FEEDBACK_BITSTREAM_BUFFER_OFFSET_BIT_KHR + 1574 ename:VK_VIDEO_ENCODE_FEEDBACK_BITSTREAM_BYTES_WRITTEN_BIT_KHR | min 1575| **slink:VkVideoEncodeH264CapabilitiesKHR** | | 1576| pname:flags | - | min 1577| pname:maxLevelIdc | code:STD_VIDEO_H264_LEVEL_IDC_1_0 | min 1578| pname:maxSliceCount | 1 | min 1579| pname:maxPPictureL0ReferenceCount | 0 | min 1580| pname:maxBPictureL0ReferenceCount | 0 | min 1581| pname:maxL1ReferenceCount | 0 | min 1582| pname:maxTemporalLayerCount | 1 | min 1583| pname:expectDyadicTemporalLayerPattern | - | implementation-dependent 1584| pname:minQp | - | max 1585| pname:maxQp | - | min 1586| pname:prefersGopRemainingFrames | - | implementation-dependent 1587| pname:requiresGopRemainingFrames | - | implementation-dependent 1588| pname:stdSyntaxFlags | - | min 1589|==== 1590 15911:: 1592 The *Requirement Type* column specifies the requirement is either the 1593 minimum value all implementations must: support, the maximum value all 1594 implementations must: support, or the exact value all implementations 1595 must: support. 1596 For bitmasks a minimum value is the least bits all implementations must: 1597 set, but they may: have additional bits set beyond this minimum. 1598
