1// Copyright 2022-2024 The Khronos Group Inc. 2// 3// SPDX-License-Identifier: CC-BY-4.0 4 5= VK_EXT_image_compression_control 6:toc: left 7:refpage: https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/ 8:sectnums: 9 10This document proposes adding support for fixed-rate, or 'lossy', image compression. 11 12== Problem Statement 13 14Many existing implementations support some form of lossless image or framebuffer compression. 15Implementations manage this transparently to applications, which is possible since the output is bit-exact. 16(The use of image compression (or not) can result in performance differences that are visible in profiling tools etc.) 17 18Fixed-rate compression formats have so far not been supported. 19As the term implies, these compression techniques are done at defined bitrates, and may therefore lose information compared to an uncompressed result. 20Since results are generally not bit-exact when compared to an uncompressed result, we do not want implementations to enable these algorithms without application opt-ins. 21 22The fixed-rate compression algorithms are implementation-specific and not standardized. 23We want to expose an API mechanism that abstracts the implementation-specific details. 24 25Further, implementation may not support all possible compression rates and may not be able to use the requested compression rates in all cases (e.g. depending on image usage flags). 26We want to expose a query to let applications understand what compression rates are available and what rates are applied to any given image. 27 28== Solution Space 29 30To enable fixed-rate compression, two options were considered: 31 32 . Add the option to enable compression on existing formats 33 . Add new fixed-rate compressed formats 34 35Adding new formats would follow the precedent of the block compressed formats (ASTC, ETC2, BCn). 36The downside of this approach is that it would introduce a very large set of new formats, in particular because implementations typically support a few different compression rates per format. 37 38This proposal uses the existing formats, but allows the application to opt in to compression for each of them. 39 40The more difficult question is how to describe the compression rates. Options that were considered: 41 42 . Describe them as bytes per 'block' of compressed data. We would likely need to describe the dimensions of each block as well as the size. 43 . Describe them as percentages of the uncompressed size. The percentages would not always be integer sizes and hard to express as enumerants. 44 . Describe them informally as low, medium, high. This would not be very informative. 45 . Describe them as bits per pixel. This has the issue that the meaning of N bits per pixel is very different between a 1-component and a 4-component format. 46 . Describe them as bits per component. This is new terminology. 47 48In the end, the "bits per component" terminology was chosen so that the same compression rate describes the same degree of compression applied to formats that differ only in the number of channels. 49For example, `VK_FORMAT_R8G8_UNORM` compressed to half its original size is a rate of 4 bits per channel, 8 bits per pixel. 50`VK_FORMAT_R8G8B8A8_UNORM` compressed to half _its_ original size is 4 bits per channel, 16 bits per pixel. 51Both of these cases could be requested with `VK_IMAGE_COMPRESSION_FIXED_RATE_4BPC_BIT_EXT`. 52 53== Proposal 54 55=== API Features 56 57Implementations may support fixed-rate compression for any image, including swapchain images. 58To allow the implementation of the WSI to be independent from the ICD, the feature is split in two extensions: 59 . VK_EXT_image_compression_control 60 . VK_EXT_image_compression_control_swapchain 61 62The following features are exposed by the VK_EXT_image_compression_control extension: 63 64[source,c] 65---- 66typedef struct VkPhysicalDeviceImageCompressionControlFeaturesEXT { 67 VkStructureType sType; 68 void* pNext; 69 VkBool32 imageCompressionControl; 70} VkPhysicalDeviceImageCompressionControlFeaturesEXT; 71---- 72 73`imageCompressionControl` is the main feature enabling this extension's functionality and must be supported if this extension is supported. 74 75The following features are exposed by the VK_EXT_image_compression_control_swapchain extension: 76 77[source,c] 78---- 79typedef struct VkPhysicalDeviceImageCompressionControlSwapchainFeaturesEXT { 80 VkStructureType sType; 81 void* pNext; 82 VkBool32 imageCompressionControlSwapchain; 83---- 84 85`imageCompressionControlSwapchain` specifies if the compression can be controlled for swapchain images and must be supported if this extension is supported. 86 87=== Enabling compression 88 89To enable compression for an image, this structure can be passed in the pNext chain of `VkImageCreateInfo`: 90 91[source,c] 92---- 93typedef struct VkImageCompressionControlEXT { 94 VkStructureType sType; 95 const void* pNext; 96 VkImageCompressionFlagsEXT flags; 97 uint32_t compressionControlPlaneCount; 98 VkImageCompressionFixedRateFlagsEXT* pFixedRateFlags; 99} VkImageCompressionControlEXT; 100---- 101 102The `flags` parameter specifies one of the following values: 103 104[source,c] 105---- 106typedef enum VkImageCompressionFlagBitsEXT { 107 VK_IMAGE_COMPRESSION_DEFAULT_EXT = 0, 108 VK_IMAGE_COMPRESSION_FIXED_RATE_DEFAULT_EXT = 0x00000001, 109 VK_IMAGE_COMPRESSION_FIXED_RATE_EXPLICIT_EXT = 0x00000002, 110 VK_IMAGE_COMPRESSION_DISABLED_EXT = 0x00000004, 111 VK_IMAGE_COMPRESSION_FLAG_BITS_MAX_ENUM_EXT = 0x7FFFFFFF 112} VkImageCompressionFlagBitsEXT; 113---- 114 115Here: 116 117 * `VK_IMAGE_COMPRESSION_DEFAULT_EXT` specifies the default behavior, where fixed-rate compression is disallowed, and is equivalent to not passing this extension structure. 118 * `VK_IMAGE_COMPRESSION_FIXED_RATE_DEFAULT_EXT` specifies that the implementation can pick a default fixed-rate compression rate. This option can be used by applications that want to enable some level of fixed-rate compression without having to query all the implementation-specific details. 119 * `VK_IMAGE_COMPRESSION_FIXED_RATE_EXPLICIT_EXT` specifies that the fixed-rate compression rates are chosen explicitly, and provided in the `pFixedRateFlags` parameters. 120 * `VK_IMAGE_COMPRESSION_DISABLED_EXT` specifies that all compression should be disabled. This is not intended for shipping applications, but may be useful for profiling and debugging. 121 122If `flags` is VK_IMAGE_COMPRESSION_FIXED_RATE_EXPLICIT_EXT the compression rate is specifies by the `compressionControlPlaneCount` and `pFixedRateFlags` parameters. 123The `compressionControlPlaneCount` parameter is included to support YCbCr formats where implementations may allow the compression rate to be different per plane. 124If the value of this parameter is `1`, then the value of `pFixedRateFlags` specifies the compression rate for all planes. 125 126Each element of `pFixedRateFlags` can be a combination of the following values: 127 128[source,c] 129---- 130typedef enum VkImageCompressionFixedRateFlagBitsEXT { 131 VK_IMAGE_COMPRESSION_FIXED_RATE_NONE_EXT = 0, 132 VK_IMAGE_COMPRESSION_FIXED_RATE_1BPC_BIT_EXT = 0x00000001, 133 VK_IMAGE_COMPRESSION_FIXED_RATE_2BPC_BIT_EXT = 0x00000002, 134 VK_IMAGE_COMPRESSION_FIXED_RATE_3BPC_BIT_EXT = 0x00000004, 135 VK_IMAGE_COMPRESSION_FIXED_RATE_4BPC_BIT_EXT = 0x00000008, 136 VK_IMAGE_COMPRESSION_FIXED_RATE_5BPC_BIT_EXT = 0x00000010, 137 VK_IMAGE_COMPRESSION_FIXED_RATE_6BPC_BIT_EXT = 0x00000020, 138 VK_IMAGE_COMPRESSION_FIXED_RATE_7BPC_BIT_EXT = 0x00000040, 139 VK_IMAGE_COMPRESSION_FIXED_RATE_8BPC_BIT_EXT = 0x00000080, 140 VK_IMAGE_COMPRESSION_FIXED_RATE_9BPC_BIT_EXT = 0x00000100, 141 VK_IMAGE_COMPRESSION_FIXED_RATE_10BPC_BIT_EXT = 0x00000200, 142 VK_IMAGE_COMPRESSION_FIXED_RATE_11BPC_BIT_EXT = 0x00000400, 143 VK_IMAGE_COMPRESSION_FIXED_RATE_12BPC_BIT_EXT = 0x00000800, 144 VK_IMAGE_COMPRESSION_FIXED_RATE_13BPC_BIT_EXT = 0x00001000, 145 VK_IMAGE_COMPRESSION_FIXED_RATE_14BPC_BIT_EXT = 0x00002000, 146 VK_IMAGE_COMPRESSION_FIXED_RATE_15BPC_BIT_EXT = 0x00004000, 147 VK_IMAGE_COMPRESSION_FIXED_RATE_16BPC_BIT_EXT = 0x00008000, 148 VK_IMAGE_COMPRESSION_FIXED_RATE_17BPC_BIT_EXT = 0x00010000, 149 VK_IMAGE_COMPRESSION_FIXED_RATE_18BPC_BIT_EXT = 0x00020000, 150 VK_IMAGE_COMPRESSION_FIXED_RATE_19BPC_BIT_EXT = 0x00040000, 151 VK_IMAGE_COMPRESSION_FIXED_RATE_20BPC_BIT_EXT = 0x00080000, 152 VK_IMAGE_COMPRESSION_FIXED_RATE_21BPC_BIT_EXT = 0x00100000, 153 VK_IMAGE_COMPRESSION_FIXED_RATE_22BPC_BIT_EXT = 0x00200000, 154 VK_IMAGE_COMPRESSION_FIXED_RATE_23BPC_BIT_EXT = 0x00400000, 155 VK_IMAGE_COMPRESSION_FIXED_RATE_24BPC_BIT_EXT = 0x00800000, 156 VK_IMAGE_COMPRESSION_FIXED_RATE_FLAG_BITS_MAX_ENUM_EXT = 0x7FFFFFFF 157} VkImageCompressionFixedRateFlagBitsEXT; 158---- 159 160Where "BPC" is an abbreviation for "Bits Per Component". 161 162If more than one bit is set in an element of `pFixedRateFlags`, the implementation should choose the smallest (most compressed) rate supported. 163 164If the imageCompressionControlSwapchain feature is supported, the `VkImageCompressionControlEXT` structure can be passed in the pNext chain of VkSwapchainCreateInfoKHR to control the compression rate for swapchain images. 165 166 167=== Querying compression 168 169To query the compression properties that actually were applied to an image, include the following structure in the pNext chain of the VkSubresourceLayout2EXT structure in a call to vkGetImageSubresourceLayout2EXT: 170 171[source,c] 172---- 173typedef struct VkImageCompressionPropertiesEXT { 174 VkStructureType sType; 175 void* pNext; 176 VkImageCompressionFlagsEXT imageCompressionFlags; 177 VkImageCompressionFixedRateFlagsEXT imageCompressionFixedRateFlags; 178} VkImageCompressionPropertiesEXT; 179---- 180 181This structure can also be passed in the pNext chain of `VkImageFormatProperties2` and `VkSurfaceFormat2KHR` to query what compression rates are available for a given format. 182 183vkGetImageSubresourceLayout2EXT is a new command that is identical to vkGetImageSubresourceLayout but with extensible input and output structures. 184 185== Examples 186 187The least invasive way to opt-in to some form of fixed-rate compression would be: 188 189[source,c] 190---- 191VkImageCreateInfo createInfo = {}; 192// fill in createInfo as usual 193 194VkImageCompressionControlEXT compressionControl = {} 195compressionControl.flags = VK_IMAGE_COMPRESSION_FIXED_RATE_DEFAULT_EXT; 196createInfo.pNext = &compressionControl; 197 198vkCreateImage(device, &createInfo, NULL, &image); 199---- 200 201To check if what level of compression was applied: 202 203[source,c] 204---- 205VkImageCompressionPropertiesEXT compressionProperties = {}; 206VkImageSubresource2EXT imageSubresource = {}; 207imageSubsource.imageSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; 208imageSubsource.imageSubresource.mipLevel = 0; 209imageSubsource.imageSubresource.arrayLayer = 0; 210VkSubresourceLayout2EXT subresourceLayout = {}; 211subresourceLayout.pNext = &compressionProperties; 212 213vkGetImageSubresourceLayout2EXT(device, image, &imageSubresource, &subresourceLayout); 214 215if (compressionProperties.imageCompressionFlags == VK_IMAGE_COMPRESSION_FIXED_RATE_EXPLICIT_EXT) 216{ 217 // fixed-rate compression was applied 218 // the rate is given by compressionProperties.imageCompressionFixedRateFlags 219} 220---- 221 222To query what rates the implementation supports: 223 224[source,c] 225---- 226VkPhysicalDeviceImageFormatInfo2 imageFormatInfo = {}; 227// fill in imageFormatInfo as usual, but also add: 228imageFormatInfo.pNext = &compressionControl; 229 230VkImageFormatProperties2 imageFormatProperties = {}; 231VkImageCompressionPropertiesEXT compressionProperties = {}; 232imageFormatProperties.pNext = &compressionProperties; 233 234vkGetPhysicalDeviceImageFormatProperties2(physicalDevice, &imageFormatInfo, &imageFormatProperties); 235 236// compressionProperties describes the supported compression rates 237// this can be used to specify explicit compression rates when the image is created 238---- 239 240== Issues 241 242=== RESOLVED: Should we split out the swapchain functionality to a separate extension? 243 244Yes. This is done allow the implementation of the WSI to be independent from the ICD. 245 246