Lines Matching +full:is +full:- +full:data +full:- +full:descriptor
1 <!-- markdownlint-disable MD041 -->
2 <!-- Copyright 2015-2019 LunarG, Inc. -->
8 # GPU-Assisted Validation
12 [3]: https://i.creativecommons.org/l/by-nd/4.0/88x31.png "Creative Commons License"
13 [4]: https://creativecommons.org/licenses/by-nd/4.0/
15 GPU-Assisted validation is implemented in the SPIR-V Tools optimizer and the `VK_LAYER_KHRONOS_vali…
16 soon-to-be-deprecated `VK_LAYER_LUNARG_core_validation` layer). This document covers the design of …
21 The basic operation of GPU-Assisted validation is comprised of instrumenting shader code to perform…
25 The layer instruments the shaders by passing the shader's SPIR-V bytecode to the SPIR-V optimizer c…
26 …erform an instrumentation pass to add the additional instructions to perform the run-time checking.
27 The layer then passes the resulting modified SPIR-V bytecode to the driver as part of the process o…
29 The layer also allocates a buffer that describes the length of all descriptor arrays and the write …
30 It only does this if the VK_EXT_descriptor_indexing extension is enabled.
32 As the shader is executed, the instrumented shader code performs the run-time checks.
34 This record is small and is on the order of a dozen 32-bit words.
36 …r records into consecutive memory locations as long as there is space available in the pre-allocat…
41 which is then reported in the same manner as other validation messages.
42 If the shader was compiled with debug information (source code and SPIR-V instruction mapping to so…
45 ## GPU-Assisted Validation Checks
47 The initial release (Jan 2019) of GPU-Assisted Validation includes checking for out-of-bounds descr…
48 for image/texel descriptor types.
50 The second release (Apr 2019) adds validation for out-of-bounds descriptor array indexing and use o…
51 VK_EXT_descriptor_indexing extension is enabled. Also added (June 2019) was validation for buffer …
53 ### Out-of-Bounds(OOB) Descriptor Array Indexing
55 Checking for correct indexing of descriptor arrays is sometimes referred to as "bind-less validatio…
56 It is called "bind-less" because a binding in a descriptor set may contain an array of like descrip…
57 And unless there is a constant or compile-time indication of which descriptor in the array is selec…
58 the descriptor binding status is considered to be ambiguous, leaving the actual binding to be deter…
67 The array of combined image samplers is `tex` and has 6 samplers in the array.
68 The complete validation error message issued when `tex_ind` indexes past the array is:
71 ERROR : VALIDATION - Message Id Number: 0 | Message Id Name: UNASSIGNED-Image descriptor index out …
72 …Index of 6 used to index descriptor array of length 6. Command buffer (CubeDrawCommandBuf)(0xbc24…
75 /home/user/src/Vulkan-ValidationLayers/external/Vulkan-Tools/cube/cube.frag at line 45.
78 The VK_EXT_descriptor_indexing extension allows a shader to declare a descriptor array without spec…
82 …r needs to tell the optimization code how big the descriptor array is so the code can determine wh…
83 bounds and what is not.
85 The extension also allows descriptor set bindings to be partially bound, meaning that as long as th…
87 …tion code needs to know which elements of a descriptor array have been written, so that it can tel…
90 Note that currently, VK_DESCRIPTOR_TYPE_INLINE_UNIFORM_BLOCK_EXT validation is not working and all …
93 ## GPU-Assisted Validation Options
95 Here are the options related to activating GPU-Assisted Validation:
97 1. Enable GPU-Assisted Validation - GPU-Assisted Validation is off by default and must be enabled.
99 …GPU-Assisted Validation is disabled by default because the shader instrumentation may introduce si…
101 GPU-Assisted Validation requires additional resources such as device memory and descriptors.
102 It is desirable for the user to opt-in to this feature because of these requirements.
106 2. Reserve a Descriptor Set Binding Slot - Modifies the value of the `VkPhysicalDeviceLimits::maxBo…
107 …property to return a value one less than the actual device's value to "reserve" a descriptor set b…
109 …This option is likely only of interest to applications that dynamically adjust their descriptor se…
114 The existing layer configuration file mechanism can be used to enable GPU-Assisted Validation.
115 This mechanism is described on the
119 To turn on GPU validation, add the following to your layer settings file, which is often
140 The `VK_EXT_validation_features` extension can be used to enable GPU-Assisted Validation at CreateI…
142 Here is sample code illustrating how to enable it:
157 ## GPU-Assisted Validation Limitations
159 There are several limitations that may impede the operation of GPU-Assisted Validation:
163 Vulkan 1.1 or later is required because the GPU instrumentation code uses SPIR-V 1.3 features.
164 Vulkan 1,1 is required to ensure that SPIR-V 1.3 is available.
166 ### Descriptor Types
168 The current implementation works with image, texel, and buffer descriptor types.
171 ### Descriptor Set Binding Limit
173 This is probably the most important limitation and is related to the
176 When applications use all the available descriptor set binding slots,
177 GPU-Assisted Validation cannot be performed because it needs a descriptor set to
180 This problem is most likely to occur on devices, often mobile, that support only the
181 minimum required value for `VkPhysicalDeviceLimits::maxBoundDescriptorSets`, which is 4.
182 Some applications may be written to use 4 slots since this is the highest value that
183 is guaranteed by the specification.
185 then GPU-Assisted Validation cannot be performed.
187 In this implementation, this condition is detected and gracefully recovered from by
188 building the graphics pipeline with non-instrumented shaders instead of instrumented ones.
189 An error message is also displayed informing the user of the condition.
191 Applications don't have many options in this situation and it is anticipated that
192 changing the application to free a slot is difficult.
196 GPU-Assisted Validation does allocate device memory for the error report buffers, and if
197 descriptor indexing is enabled, for the input buffer of descriptor sizes and write state.
199 the application is trying to use all of the available memory.
203 Note that if descriptor indexing is enabled, the input buffer size will be equal to
205 binding_count is the binding number of the largest binding in the set.
208 As a best practice, when using GPU-Assisted Validation with descriptor indexing enabled,
209 make sure descriptor bindings are densely packed.
211 If GPU-Assisted Validation device memory allocations fail, the device could become
212 unstable because some previously-built pipelines may contain instrumented shaders.
213 This is a condition that is nearly impossible to recover from, so the layer just
215 There is a reasonable chance to recover from these conditions,
220 This is roughly the same problem as the device memory problem mentioned above,
222 Any failure to allocate a descriptor set means that the instrumented shader code
244 to implement GPU-Assisted Validation without some of the limitations described above.
247 This technique removes the need to create descriptors, use a descriptor set slot,
250 This alternate implementation is under consideration.
252 ## GPU-Assisted Validation Internal Design
254 This section may be of interest to readers who are interested on how GPU-Assisted Validation is imp…
263 …If descriptor indexing is enabled, calculate the amount of memory needed to describe the descripto…
265 The Vulkan Memory Allocator is used to handle this efficiently.
267 …There is probably little advantage in providing a larger output buffer in order to obtain more deb…
268 …It is likely, especially for fragment shaders, that multiple errors occurring near each other have…
270 …A block is allocated on a per draw basis to make it possible to associate a shader debug error rec…
272 …This is done partly to give the user more information in the error report, namely the command buff…
273 An alternative design allocates this block on a per-device or per-queue basis and should work.
274 …However, it is not possible to identify the command buffer that causes the error if multiple comma…
276 * For each draw, dispatch, and trace rays call, allocate a descriptor set and update it to point to…
277 …If descriptor indexing is enabled, also update the descriptor set to point to the allocated input …
278 Fill the input buffer with the size and write state information for each descriptor array.
279 There is a descriptor set manager to handle this efficiently.
280 …ke an additional call down the chain to create a bind descriptor set command to bind our descripto…
282 writes into this buffer for when the draw is executed.
283 The end result is that each draw call has its own buffer containing GPU instrumentation error
285 * Determine the descriptor set binding index that is eventually used to bind the descriptor set jus…
286 Usually, it is `VkPhysicalDeviceLimits::maxBoundDescriptorSets` minus one.
289 * When creating a ShaderModule, pass the SPIR-V bytecode to the SPIR-V optimizer to perform the ins…
290 …Pass the desired descriptor set binding index to the optimizer via a parameter so that the instrum…
291 code knows which descriptor to use for writing error report data to the memory block.
292 …If descriptor indexing is enabled, turn on OOB and write state checking in the instrumentation pas…
294 * For all pipeline layouts, add our descriptor set to the layout, at the binding index determined e…
295 Fill any gaps with empty descriptor sets.
297 …If the incoming layout already has a descriptor set placed at our desired index, the layer must no…
298 descriptor set to the layout, replacing the one in the incoming layout.
300 non-instrumented ones when the pipeline layout is later used to create a graphics pipeline.
302 …line, ComputePipeline, or RayTracingPipeline, check to see if the pipeline is using the debug bind…
303 If it is, replace the instrumented shaders in the pipeline with non-instrumented ones.
304 * Before calling QueueSubmit, if descriptor indexing is enabled, check to see if there were any unw…
305 update-after-bind.
309 If any debug record is found, generate a validation error message for each record found.
311 The above describes only the high-level details of GPU-Assisted Validation operation.
312 More detail is found in the discussion of the individual hooked functions below.
319 From these options, the layer sets instance-scope flags in the validation layer tracking data to in…
320 GPU-Assisted Validation has been requested, along with any other associated options.
324 Much of the GPU-Assisted Validation implementation involves making "application level" Vulkan API
336 To address this, the code can "just" be written carefully so that it is "valid" Vulkan,
337 which is hard to do.
342 This sort of checking is performed by layer developers to check that the additional
343 Vulkan usage is valid.
347 * Building the validation layer with a hack to force GPU-Assisted Validation to be enabled.
352 …* Set up the layer stack so that the "khronos_validation2" layer is on top of or before the actual…
361 * The additional API calls are not state-tracked
363 This means that things like device memory allocations and descriptor allocations are not
365 For example, any device memory allocation performed by GPU-Assisted Validation won't be
367 This could lead to an early allocation failure that is not accompanied by a validation error.
369 This shortcoming is left as not addressed in this implementation because it is anticipated that
370 a later implementation of GPU-Assisted Validation using the `VK_EXT_buffer_device_address`
376 The GPU-Assisted Validation code is largely contained in one
377 [file](https://github.com/KhronosGroup/Vulkan-ValidationLayers/blob/master/layers/gpu_validation.cp…
382 if (GetEnables(dev_data)->gpu_validation) {
387 The GPU-Assisted Validation code is linked into the shared library for the khronos and core validat…
391 Each function for a Vulkan API command intercepted in the khronos validation layer is usually split…
400 The GPU-Assisted Validation functions follow this pattern not by hooking into the top-level validat…
413 * Determine and record (save in device state) the desired descriptor set binding index.
416 * Initialize descriptor set manager
417 * Make a descriptor set layout to describe our descriptor set
418 * Make a descriptor set layout to describe a "dummy" descriptor set that contains no descriptors
419 …* This is used to "pad" pipeline layouts to fill any gaps between the used bind indices and our bi…
420 * Record these objects in the per-device state
424 * Destroy descriptor set layouts created in CreateDevice
425 * Clean up descriptor set manager
432 * Get a descriptor set from the descriptor set manager
434 …* If descriptor indexing is enabled, get an input buffer and fill with descriptor array information
435 * Update (write) the descriptor set with the memory info
436 * Check to see if the layout for the pipeline just bound is using our selected bind index
437 …* If no conflict, add an additional command to the command buffer to bind our descriptor set at ou…
438 * Record the above objects in the per-CB state
445 * Give the descriptor sets back to the descriptor set manager
450 This function is called from PreCallRecordCreateShaderModule.
451 … sets up to call the SPIR-V optimizer to run the "BindlessCheckPass", replacing the original SPIR-…
452 which is then used in the call down the chain to CreateShaderModule.
454 This function generates a "unique shader ID" that is passed to the SPIR-V optimizer,
456 This ID is returned by this function so it can be recorded in the shader module at PostCallRecord t…
458 … be instrumented before creating the shader module and therefore the handle is not available to use
460 Therefore, the layer keeps a "counter" in per-device state that is incremented each time a shader i…
462 This unique ID is given to the SPIR-V optimizer and is stored in the shader module state tracker af…
464 The process of instrumenting the SPIR-V also includes passing the selected descriptor set binding i…
465 to the SPIR-V optimizer which the instrumented
467 An instrumented shader is now "hard-wired" to write error records via the descriptor set at that bi…
471 The original SPIR-V bytecode is left stored in the shader module tracking data.
472 This is important because the layer may need to replace the instrumented shader with the original s…
473 there is a binding index conflict.
475 This ensures that the original SPIR-V bytecode is available if we need it to replace the instrument…
479 This is function is called through PreCallRecordCreatePipelineLayout.
481 * Check for a descriptor set binding index conflict.
482 * If there is one, issue an error message and leave the pipeline layout unmodified
485 * Copy the original descriptor set layouts into the new pipeline layout
486 …* Pad the new pipeline layout with dummy descriptor set layouts up to but not including the last o…
487 * Add our descriptor set layout as the last one in the new pipeline layout
505 * Report an error about a possible deadlock if CmdWaitEvents is recorded with VK_PIPELINE_STAGE_HOS…
509 * Examine the pipelines to see if any use the debug descriptor set binding index
511 * Create non-instrumented shader modules from the saved original SPIR-V
512 * Modify the CreateInfo data to use these non-instrumented shaders.
513 * This prevents instrumented shaders from using the application's descriptor set.
526 This tracker is used to attach the shader bytecode to the shader in case it is needed
531 It is possible for the application to destroy the shader module after
537 so the graphics pipeline handle is also stored in this tracker so that it can
538 be looked up when the graphics pipeline is destroyed.
539 At that point, it is safe to free the bytecode since the pipeline is never used again.
547 The shader instrumentation process performed by the SPIR-V optimizer applies descriptor index bound…
560 Instrumentation is applied to the following SPIR-V operations:
596 either Uniform or StorageBuffer storage class and a type which is either a
597 struct decorated with Block, or a runtime or statically-sized array of such
605 This description includes the support for future GPU-Assisted Validation features
606 such as checking for uninitialized descriptors in the partially-bound scenario.
607 These items are not used in the current implementation for descriptor array
611 The format of this buffer is as follows:
617 uint Data[];
621 `DataWrittenLength` is the number of uint32_t words that have been attempted to be written.
624 The `Data` array is the uint32_t words written by the shaders of the pipeline to record bindless va…
625 All elements of `Data` should be initialized to 0.
626 Note that the `Data` array has runtime length.
627 The shader queries the length of the `Data` array to make sure that it does not write past the end …
629 The layer uses the length of `Data` to control the number of records written by the shaders.
631 The `DataWrittenLength` is atomically updated by the shaders so that shaders do not overwrite each …
633 If the value plus the record length is greater than the length of `Data`, it does not write the rec…
635 … protocol, the value in `DataWrittenLength` is not very meaningful if it is greater than the lengt…
636 However, the format of the written records plus the fact that `Data` is initialized to 0 should be …
641 The format of an output record is the following:
647 <Stage-Specific Words>
648 <Validation-Specific Words>
650 The Record Size is the number of words in this record, including the the Record Size.
652 The Shader ID is a handle that was provided by the layer when the shader was instrumented.
654 The Instruction Index is the instruction within the original function at which the error occurred.
655 For bindless, this will be the instruction which consumes the descriptor in question,
656 or the instruction that consumes the OpSampledImage that consumes the descriptor.
658 The Stage is the integer value used in SPIR-V for each of the Execution Models:
661 |---------------|:-----:|
681 |---------------|------------------|------------|------------|
696 ### Validation-Specific Words
701 The first word is the Error Code.
708 |-----------------------------|:----:|----------------|-----------------------|
709 |IndexOutOfBounds |0 |Descriptor Index|Descriptor Array Length|
710 |DescriptorUninitialized |1 |Descriptor Index|unused |
712 So the words written for an image descriptor bounds error in a fragment shader is:
724 If another error is encountered, that record is written starting at Word 10, if the whole record wi…
727 …an continue to read valid records until it sees a Record Length of 0 or the end of Data is reached.
731 The programmatic interface for the above informal description is codified in the
732 [SPIRV-Tools](https://github.com/KhronosGroup/SPIRV-Tools) repository in file
733 [`instrument.hpp`](https://github.com/KhronosGroup/SPIRV-Tools/blob/master/include/spirv-tools/inst…
737 ## GPU-Assisted Validation Error Report
739 This is a fairly simple process of mapping the debug report buffer associated with
744 The layer clears the buffer to zeros when it is allocated and after processing any
751 * command buffer handle - This is easily obtained because we are looping over the command
753 * draw number - keep track of how many draws we've processed for a given command buffer.
754 * pipeline handle - The shader tracker discussed earlier contains this handle
755 * shader module handle - The "Shader ID" (Word 1 in the record) is used to lookup
756 the shader tracker which is then used to obtain the shader module and pipeline handles
757 * instruction index - This is the SPIR-V instruction index where the invalid array access occurred.
758 …It is not that useful by itself, since the user would have to use it to locate a SPIR-V instruction
759 in a SPIR-V disassembly and somehow relate it back to the shader source code.
760 But it could still be useful to some and it is easy to report.
761 The user can build the shader with debug information to get source-level information.
765 If a name exists for that object, it is included in the error message.
767 The layer then adds on error message text obtained from decoding the stage-specific and
768 validation-specific data as described earlier.
770 This completes the error report when there is no source-level debug information in the shader.
772 ### Source-Level Debug Information
774 This is one of the more complicated and code-heavy parts of the GPU-Assisted Validation feature
775 and all it really does is display source-level information when the shader is compiled
776 with debugging info (`-g` option in the case of `glslangValidator`).
782 The SPIR-V generator (e.g., glslangValidator) places an OpLine SPIR-V instruction in the
786 It is possible to have two source code statements on the same line in the source file,
789 The layer scans the SPIR-V looking for the last OpLine instruction that appears before the instruct…
793 The filename itself is obtained by scanning the SPIR-V again for an OpString instruction that
796 This information is added to the validation error message.
798 For online compilation when there is no "file", only the line number information is reported.
802 The SPIR-V built with source-level debug info also contains OpSource instructions that
804 Due to possible pre-processing, the layer just cannot simply use the source file line number
807 Instead, the correct source code line is found by first locating the "#line" directive in the
813 Then the difference between the "#line" line number and the OpLine line number is added
814 to the place where the "#line" was found to locate the actual line of source, which is
817 For example, if the OpLine line number is 15, and there is a "#line 10" on line 40
822 Although the input buffer is a linear array of unsigned integers, conceptually there are arrays wit…
824 Word 1 starts an array (denoted by sets_to_sizes) that is number_of_sets long, with an index that i…
826 …ray is the sizes array, that contains the array size (or 1 if descriptor is not an array) of each …
828 … array is the sets_to_bindings array that for each descriptor set, indexes into the bindings_to_wr…
830 After the sets_to_bindings array, is the bindings_to_written array that for each binding in the set…
836 Assume Descriptor Set 0 looks like: And Descriptor Set 1 looks like:
842 Here is what the input buffer should look like:
863 Alternately, you could describe the array size and write state data as:
864 (set = s, binding = b, index = i) is not initialized if
870 ## GPU-Assisted Validation Testing
872 Validation Layer Tests (VLTs) exist for GPU-Assisted Validation.
878 They activate GPU-Assisted Validation via the programmatic
882 is built with debug info.