• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Queue Present Wait Semaphore Management
2
3The following shorthand notations are used throughout this document:
4
5- PE: Presentation Engine
6- ANI: vkAcquireNextImageKHR
7- QS: vkQueueSubmit
8- QP: vkQueuePresentKHR
9- W: Wait
10- S: Signal
11- R: Render
12- P: Present
13- SN: Semaphore N
14- IN: Swapchain image N
15- FN: Fence N
16
17---
18
19## Introduction
20
21Vulkan requires the application (ANGLE in this case) to acquire swapchain images and queue them for
22presentation, synchronizing GPU submissions with semaphores.  A single frame looks like the
23following:
24
25    CPU: ANI  ... QS   ... QP
26         S:S1     W:S1     W:S2
27                  S:S2
28    GPU:          <------------ R ----------->
29     PE:                                      <-------- P ------>
30
31That is, the GPU starts rendering after submission, and the presentation is started when rendering is
32finished.  Note that Vulkan tries to abstract a large variety of PE architectures, some of which do
33not behave in a straight-forward manner.  As such, ANGLE cannot know what the PE is exactly doing
34with the images or when the images are visible on the screen.  The only signal out of the PE is
35received through the semaphore that's used in ANI.
36
37With multiple frames, the pipeline looks different based on present mode.  Let's focus on
38FIFO (the arguments in this document translate to all modes) with 3 images:
39
40    CPU: QS QP QS QP QS QP QS QP
41         I1 I1 I2 I2 I3 I3 I1 I1
42    GPU: <---- R I1 ----><---- R I2 ----><---- R I3 ----><---- R I1 ---->
43     PE:                 <----- P I1 -----><----- P I2 -----><----- P I3 -----><----- P I1 ----->
44
45First, an issue is evident here.  The CPU is submitting jobs and queuing images for presentation
46faster than the GPU can render them or the PE can view them.  This can cause the length of the
47submit queue to grow indefinitely, resulting in larger and larger input lag.  In FIFO mode, the PE
48present queue also grows indefinitely.
49
50To address this issue, ANGLE paces the CPU such that the length of the submit queue is kept at a
51maximum of 1 image (i.e. submission with one image is being processed, and another one is in queue):
52
53    CPU: QS   QS          W:F1 QS         W:F2 QS
54         I1   I2               I3              I1
55         S:F1 S:F2             S:F3            S:F4
56    GPU: <---- R I1 ----><---- R I2 ----><---- R I3 ----><---- R I1 ---->
57
58> Note: Ideally, the length of the PE present queue should also be kept at a maximum of 1 (i.e. one
59> image being presented, and another in queue).  However, the Vulkan WSI extension doesn't provide
60> enough control to achieve this.  In heavy application, the length of the PE present queue is
61> probably 1 anyway (as the rendering time is almost as long as the frame (i.e. present time), in
62> which case pacing the submissions similarly paces the presentation).  In theory, in FIFO mode, the
63> length of the PE present queue is below n+2 where n is the number of swapchain images.
64>
65> To understand why, imagine a FIFO swapchain with 1000 images and submissions that are
66> infinitesimally short.  In this case, the CPU pacing is effectively a no-op (as the GPU instantly
67> finishes jobs) for the first 1002 submissions.  The 1003rd submission waits for F1001 (which uses
68> I1).  However, the 1001st submission will not start until the PE switches to presenting I2 (at the
69> next V-Sync).  The CPU then waits for V-Sync before the 1003rd submission.  The CPU waits for one
70> V-Sync for every subsequent submission, keeping the length of the queue 1002.
71> [`VK_GOOGLE_display_timing`][DisplayTimingGOOGLE] is likely a solution to this problem.
72
73Associated with each QP operation is a semaphore signaled by the preceding QS and waited on by the
74PE before the image can be presented.  Currently, there's no feedback from Vulkan (See [internal
75Khronos issue][VulkanIssue1060]) regarding _when_ the PE has actually finished waiting on the
76semaphore!  This means that the application cannot generally know when to destroy the corresponding
77semaphore.  However, taking ANGLE's CPU pacing into account, we are able to destroy (or rather
78reuse) semaphores when they are provably unused.
79
80This document describes an approach for destroying semaphores that should work with all valid PE
81architectures, but will be described in terms of more common PE architectures (e.g. where the PE
82only backs each VkImage and VkSemaphore handle with one actual memory object, and where the PE
83cycles between the swapchain images in a straight-forward manner).
84
85The interested reader may follow the discussion in this abandoned [gerrit CL][CL1757018] for more
86background and ideas.
87
88[DisplayTimingGOOGLE]: https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VK_GOOGLE_display_timing.html
89[VulkanIssue1060]: https://gitlab.khronos.org/vulkan/vulkan/issues/1060
90[CL1757018]: https://chromium-review.googlesource.com/c/angle/angle/+/1757018
91
92## Determining When a QP Semaphore is Waited On
93
94Let's combine the above diagrams with all the details:
95
96    CPU: ANI   | QS    | QP    | ANI   | QS    | QP    | ANI   | W:F1 | QS    | QP    | ANI   | W:F2 | QS    | QP
97         I1    | I1    | I1    | I2    | I2    | I2    | I3    |      | I3    | I3    | I1    |      | I1    | I1
98         S:SA1 | W:SA1 |       | S:SA2 | W:SA2 |       | S:SA3 |      | W:SA3 |       | S:SA4 |      | W:SA4 |
99               | S:SP1 | W:SP1 |       | S:SP2 | W:SP2 |       |      | S:SP3 | W:SP3 |       |      | S:SP4 | W:SP4
100               | S:F1  |       |       | S:F2  |       |       |      | S:F3  |       |       |      | S:F4  |
101
102Let's focus only on sequences that return the same image:
103
104    CPU: ANI   | W:F(X-2) | QS    | QP    | ... | ANI   | W:F(Y-2) | QS    | QP
105         I1    |          | I1    | I1    |     | I1    |          | I1    | I1
106         S:SAX |          | W:SAX |       |     | S:SAY |          | W:SAY |
107               |          | S:SPX | W:SPX |     |       |          | S:SPY | W:SPY
108               |          | S:FX  |       |     |       |          | S:FY  |
109
110Note that X and Y are arbitrarily distanced (including possibly being sequential).
111
112Say we are at frame Y+2.  There's therefore a wait on FY.  The following holds:
113
114    FY is signaled
115    => SAY is signaled
116    => The PE has handed I1 back to the application
117    => The PE has already processed the *previous* QP of I1
118    => SPX is waited on
119
120At this point, we can destroy SPX.  In other words, in frame Y+2, we can destroy SPX (note that 2 is
121the number of frames the CPU pacing code uses).  If frame Y+1 is not using I1, this means the
122history of present semaphores for I1 would be `{SPX, SPY}` and we can destroy the oldest semaphore
123in this list.  If frame Y+1 is also using I1, we should still destroy SPX in frame Y+2, but the
124history of the present semaphores for I1 would be `{SPX, SPY, SP(Y+1)}`.
125
126In the Vulkan backend, we simplify destruction of semaphores by always keeping a history of 3
127present semaphores for each image (again, 3 is H+1 where H is the swap history size used in CPU
128pacing) and always reuse (instead of destroy) the oldest semaphore of the image that is about to be
129presented.
130
131To summarize, we use the completion of a submission using an image to prove when the semaphore used
132for the *previous* presentation of that image is no longer in use (and can be safely destroyed or
133reused).
134
135## Swapchain recreation
136
137When recreating the swapchain, all images are eventually freed and new ones are created, possibly
138with a different count and present mode.  For the old swapchain, we can no longer rely on the
139completion of a future submission to know when a previous presentation's semaphore can be destroyed,
140as there won't be any more submissions using images from the old swapchain.
141
142> For example, imagine the old swapchain was created in FIFO mode, and one image is being presented
143> until the next V-Sync.  Furthermore, imagine the new swapchain is created in MAILBOX mode.  Since
144> the old swapchain's image will remain presented until V-Sync, the new MAILBOX swapchain can
145> perform an arbitrarily large number of (throw-away) presentations.  The old swapchain (and its
146> associated present semaphores) cannot be destroyed until V-Sync; a signal that's not captured by
147> Vulkan.
148
149ANGLE resolves this issue by deferring the destruction of the old swapchain and its remaining
150present semaphores to the time when the semaphore corresponding to the first present of the new
151swapchain can be destroyed.  In the example in the previous section, if SPX is the present semaphore
152of the first QP performed on the new swapchain, at frame Y+2, when we know SPX can be destroyed, we
153know that the first image of the new swapchain has already been presented.  This proves that all
154previous QPs of the old swapchain have been processed.
155
156> Note: the swapchain can potentially be destroyed much earlier, but with no feedback from the
157> presentation engine, we cannot know that.  This delays means that the swapchain could be recreated
158> while there are pending old swapchains to be destroyed.  The destruction of both old swapchains
159> must now be deferred to when the first QP of the new swapchain has been processed.  If an
160> application resizes the window constantly and at a high rate, ANGLE would keep accumulating old
161> swapchains and not free them until it stops.  While a user will likely not be able to do this (as
162> the rate of window system events is lower than the framerate), this can be programmatically done
163> (as indeed done in EGL dEQP tests).  Nvidia for example fails creation of a new swapchain if there
164> are already 20 allocated (on desktop, or less than ten on Quadro).  If the backlog of old
165> swapchains get larger than a threshold, ANGLE calls `vkQueueWaitIdle()` and destroys the
166> swapchains.
167