1// Copyright 2022-2023 The Khronos Group Inc. 2// 3// SPDX-License-Identifier: CC-BY-4.0 4 5= VK_EXT_swapchain_maintenance1 6:toc: left 7:refpage: https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/ 8:sectnums: 9 10This proposal investigates and addresses a number of practical issues with the 11`VK_KHR_swapchain` specification. 12 13== Problem Statement 14 15The following is a list of issues considered in this proposal: 16 17 * It is not directly known when a semaphore used for presentation can be 18 recycled. 19 * Switching present modes requires a swapchain recreation 20 * Upfront memory allocation is costly in startup time and memory 21 * Unknown behavior when presenting a swapchain image to a surface with 22 different dimensions 23 * Impossible to release acquired images without presenting them 24 25=== Recycling Present Semaphores 26 27With `VK_KHR_swapchain`, there is no way to provide feedback as to when a 28semaphore used for presentation can be freed or recycled. 29The application would have to infer this information from when the fence of the 30_next_ call to `vkAcquireNextImageKHR` that returns the presented image's index 31has been signaled. 32This is needlessly complicated. 33 34=== Swapchain Recreation on Present Mode Change 35 36Currently, when changing present modes, the Vulkan swapchain is required to be 37recreated. 38This is unnecessary if otherwise the swapchain is not changed, as the present 39mode does not affect the actual swapchain images and their associated memory. 40This is realistically affecting apps that switch between 41VK_PRESENT_MODE_SHARED_DEMAND_REFRESH_KHR and 42VK_PRESENT_MODE_SHARED_CONTINUOUS_REFRESH_KHR (in single-buffer applications). 43In that case, a flicker is noticed as the swapchain is recreated and the old 44contents of the swapchain is discarded. 45 46=== Upfront Memory Allocation 47 48Currently, Vulkan allows the application to record commands using any image 49from the swapchain. 50Additionally, the application may use `VkBindImageMemorySwapchainInfoKHR` at 51any time to bind an image to swapchain memory. 52This requires swapchain implementations to allocate memory for all images 53upfront. 54This is costly both in startup time, and potentially memory if all images are 55not eventually acquired. 56 57=== Scaling Behavior 58 59It is desirable for the application to specify a scaling behavior for when the 60swapchain extents do not match the surface's, such as during window resizing, 61instead of receiving `VK_ERROR_OUT_OF_DATE_KHR`. 62 63See the proposal document for `VK_EXT_surface_maintenance1` for details. 64 65=== Releasing Acquired Images 66 67When the swapchain needs to be recreated, it may be possible that an 68application has acquired images from the old swapchain that it is no longer 69interested in presenting to. 70However, it is unable to release those images before recreating the swapchain. 71As a result, the application is forced to present to an old swapchain that may 72no longer match the surface. 73Additionally, the memory allocated for the old and new swapchains coexist, 74increasing peak memory usage. 75 76== Solution Space 77 78There are generally two possible approaches to resolving these problems: 79 80 * Have the application directly interface with the window system, bypassing 81 `VK_KHR_swapchain`. 82 * Improve upon `VK_KHR_swapchain` 83 84The benefits of the first approach are: 85 86 * Total control of the swapchain implementation by the application, which 87 would allow for prompt fixes of future issue 88 * Works on existing drivers, removing any need for fallbacks 89 90However, this approach results in duplicated effort among applications. 91A potential library with a swapchain implementation can be conceived which 92could be updated and shipped with applications independently from driver 93updates, though it lacks the benefit of driver updates bringing fixes and 94optimizations to all applications. 95 96This proposal considers the second approach with the goal of improving the 97larger Vulkan ecosystem while leveraging existing swapchain implementations 98which most applications have come to rely on. 99 100=== Recycling Present Semaphores 101 102There are three potential solutions for this: 103 104 * A status query for present operations, potentially identified through ids 105 assigned with `VK_KHR_present_id`. 106 * A wait function similar to `vkWaitForPresentKHR` 107 * A fence passed to `vkQueuePresentKHR` that signals when the presentation 108 engine no longer accesses the present semaphores 109 110The last solution is adopted as it provides more power to the application 111(wait, in addition to query), as well as fitting better with event loops, other 112Vulkan APIs and future work to enable timeline semaphores. 113 114=== Swapchain Recreation on Present Mode Change 115 116Each present mode may require a different number of images. 117Additionally, the `VK_PRESENT_MODE_SHARED_DEMAND_REFRESH_KHR` and 118`VK_PRESENT_MODE_SHARED_CONTINUOUS_REFRESH_KHR` present modes may require a 119different image memory layout compared with the rest of the present modes. 120Switching between non-shared present modes may or may not be possible on some 121implementations. 122 123Regarding image count, potential solutions are: 124 125 * Specify all potential present modes at swapchain creation time 126 ** The swapchain would then allocate as many images to satisfy all present 127 modes 128 * Increase the number of swapchain images as necessary when present mode is 129 changed 130 ** This requires the application to use `vkGetSwapchainImagesKHR` after each 131 change in present mode and adjust accordingly 132 133Regarding the subsets of switchable present modes that the implementation 134supports, potential solutions are: 135 136 * Specify a particular behavior for `VkSwapchainCreateInfoKHR::oldSwapchain` 137 when the only change is in the present mode 138 ** This is not straightforward and is error-prone 139 * Allow modifying present modes as needed, such as during a 140 `vkQueuePresentKHR` call 141 ** Implementations may support switching between present modes only in 142 disjoint subsets. 143 This subsets can be queried. 144 Alternatively, Vulkan could divide the present modes into shared and 145 non-shared, with the latter conditional to a feature flag. 146 147The adopted solution in this proposal is to allow applications to specify a 148potential set of present modes at swapchain creation time. 149A query must be used to determine whether these present modes are compatible 150for dynamic switching. 151Then, the application would pass the desired present mode to 152`vkQueuePresentKHR`. 153 154With the solution to the "Upfront Memory Allocation" problem, the application 155can avoid paying any extra memory cost due to higher image counts until a 156present mode that uses that many images is used. 157 158=== Upfront Memory Allocation 159 160This can be explicitly opted in with a new swapchain create flag. 161Using this flag would prohibit problematic scenarios, such as using 162`VkBindImageMemorySwapchainInfoKHR` with an image index that has never been 163acquired, or recording command buffers using those swapchain images. 164 165This amortizes the cost of memory allocation between the first few frames, 166using CPU time that may otherwise have gone idle waiting for the GPU. 167This will additionally resolve a number of memory issues: 168 169 * If a present mode is specified at swapchain creation time which requires a 170 larger number of images, but that is never actually used, the extra 171 images would not have to consume memory. 172 * When resizing the swapchain, peak memory increase is avoided by not 173 actually allocating memory for the new swapchain. 174 First memory allocation for the new swapchain could happen after some of 175 the images from the old swapchain have been destroyed. 176 177=== Scaling Behavior 178 179The `VK_EXT_surface_maintenance1` extension introduces scaling and gravity 180behavior enums whose support can be queried from the surface. 181At swapchain creation time, one of the supported behavior can be specified by 182the application. 183 184=== Releasing Acquired Images 185 186There are a number of ways the applications could partially work around this 187issue, such as by deferring image acquisition, or recreating the swapchain only 188once all images from the old swapchain are presented. 189 190However, in all implementations, releasing an acquired image without presenting 191it is no more complex than presenting it. 192The adopted solution is to expose this functionality in this extension. 193 194== Proposal 195 196Introduced by this API are: 197 198=== Feature 199 200Advertising whether the implementation supports the functionality in this 201extension: 202 203[source,c] 204---- 205typedef struct VkPhysicalDeviceSwapchainMaintenance1FeaturesEXT { 206 VkStructureType sType; 207 void* pNext; 208 VkBool32 swapchainMaintenance1; 209} VkPhysicalDeviceSwapchainMaintenance1FeaturesEXT; 210---- 211 212=== Present Fence 213 214To associate fences with the present operations for each swapchain, chain the 215following to `VkPresentInfoKHR`: 216 217[source,c] 218---- 219typedef struct VkSwapchainPresentFenceInfoEXT { 220 VkStructureType sType; 221 void* pNext; 222 uint32_t swapchainCount; 223 VkFence* pFences; 224} VkSwapchainPresentFenceInfoEXT; 225---- 226 227With `swapchainCount` matching 228`VkSwapchainPresentFenceInfoEXT::swapchainCount`, each swapchain being 229presented to will signal the fence once the application is allowed to destroy 230or recycle the semaphores passed to `vkPresentInfoKHR::pname:pWaitSemaphores`. 231 232=== Switching Present Modes 233 234During creation of the swapchain, all potential present modes are specified by 235chaining the following to `VkSwapchainCreateInfoKHR`: 236 237[source,c] 238---- 239typedef struct VkSwapchainPresentModesCreateInfoEXT { 240 VkStructureType sType; 241 void* pNext; 242 uint32_t presentModeCount; 243 VkPresentModeKHR* pPresentModes; 244} VkSwapchainPresentModesCreateInfoEXT; 245---- 246 247The present modes given in `pPresentModes` must be compatible for mode 248switching. 249This can be queried by use of `VkSurfacePresentModeCompatibilityEXT` from the 250`VK_EXT_surface_maintenance1` extension. 251 252The present mode can be changed by chaining the following to 253`VkPresentInfoKHR`: 254 255[source,c] 256---- 257typedef struct VkSwapchainPresentModeInfoEXT { 258 VkStructureType sType; 259 void* pNext; 260 uint32_t swapchainCount; 261 VkPresentModeKHR* pPresentModes; 262} VkSwapchainPresentModeInfoEXT; 263---- 264 265Where the elements of `pPresentModes` can take any present mode specified in 266`VkSwapchainPresentModesCreateInfoEXT` during the creation of the respective 267swapchain. 268If not specified, the swapchain will continue to operate according to the last 269specified present mode. 270 271=== Swapchain Memory Allocation 272 273To allow the swapchain to defer memory allocation for each image until it is 274acquired through `vkAcquireNextImageKHR`, specify the 275`VK_SWAPCHAIN_CREATE_DEFERRED_MEMORY_ALLOCATION_BIT_EXT` flag in 276`VkSwapchainCreateInfoKHR::flags`. 277In that case, the application may still use `vkGetSwapchainImagesKHR` to 278retrieve the image handles, but it may not use any image whose index has never 279been returned by `vkAcquireNextImageKHR`; this includes recording commands 280using such an image, or binding another image to its memory through 281`VkBindImageMemorySwapchainInfoKHR`. 282 283As memory allocation for each image is deferred, so should the application 284defer the above operations for each image until it is first acquired. 285 286=== Scaling Behavior 287 288To specify the scaling behavior of the swapchain, chain the following to 289`VkSwapchainCreateInfoKHR`: 290 291[source,c] 292---- 293typedef struct VkSwapchainPresentScalingCreateInfoEXT { 294 VkStructureType sType; 295 void* pNext; 296 VkPresentScalingFlagsEXT scalingBehavior; 297 VkPresentGravityFlagsEXT presentGravityX; 298 VkPresentGravityFlagsEXT presentGravityY; 299} VkSwapchainPresentScalingCreateInfoEXT; 300---- 301 302The values specified in `scalingBehavior`, `presentGravityX` and 303`presentGravityY` must be supported by the surface. 304This can be queried by use of `VkSurfacePresentScalingCapabilitiesEXT` from the 305`VK_EXT_surface_maintenance1` extension. 306 307=== Releasing Acquired Images 308 309To release previously acquired images back to the swapchain, call 310`vkReleaseSwapchainImagesEXT`: 311 312[source,c] 313---- 314VKAPI_ATTR VkResult VKAPI_CALL vkReleaseSwapchainImagesEXT( 315 VkDevice device, 316 const VkReleaseSwapchainImagesInfoEXT* pReleaseInfo); 317---- 318 319`VkReleaseSwapchainImagesInfoEXT` is defined as: 320 321[source,c] 322---- 323typedef struct VkReleaseSwapchainImagesInfoEXT { 324 VkStructureType sType; 325 const void* pNext; 326 VkSwapchainKHR swapchain; 327 deUint32 imageIndexCount; 328 const deUint32* pImageIndices; 329} VkReleaseSwapchainImagesInfoEXT; 330---- 331 332== Issues 333 334=== RESOLVED: Should there be a surface capability bit to advertise the present fence functionality? 335 336No. 337Present fences fix a critical hole in the swapchain programming model and hence 338are a required feature of this maintenance extension. 339