Lines Matching +full:shared +full:- +full:dma +full:- +full:pool

1 .. SPDX-License-Identifier: GPL-2.0
4 DMA and swiotlb
7 swiotlb is a memory buffer allocator used by the Linux kernel DMA layer. It is
8 typically used when a device doing DMA can't directly access the target memory
10 the DMA layer calls swiotlb to allocate a temporary memory buffer that conforms
11 to the limitations. The DMA is done to/from this temporary memory buffer, and
17 the DMA layer of the DMA attributes of the devices they are managing, and use
18 the normal DMA map, unmap, and sync APIs when programming a device to do DMA.
19 These APIs use the device DMA attributes and kernel-wide settings to determine
20 if bounce buffering is necessary. If so, the DMA layer manages the allocation,
21 freeing, and sync'ing of bounce buffers. Since the DMA attributes are per
25 memory buffer, doing bounce buffering is slower than doing DMA directly to the
27 when necessary for providing DMA functionality.
30 ---------------
31 swiotlb was originally created to handle DMA for devices with addressing
33 only provide 32-bit DMA addresses. By allocating bounce buffer memory below
35 do DMA.
40 directed to guest memory that is unencrypted. CoCo VMs set a kernel-wide option
41 to force all DMA I/O to use bounce buffers, and the bounce buffer memory is set
42 up as unencrypted. The host does DMA I/O to/from the bounce buffer memory, and
43 the Linux kernel DMA layer does "sync" operations to cause the CPU to copy the
50 mappings are set up for a DMA operation to/from a device that is considered
54 IOMMU access control is per-granule, the untrusted device can gain access to
55 the unrelated kernel data. This problem is solved by bounce buffering the DMA
60 ------------------
64 buffer memory is physically contiguous. The expectation is that the DMA layer
65 maps the physical memory address to a DMA address, and returns the DMA address
66 to the driver for programming into the device. If a DMA operation specifies
72 swiotlb_tbl_unmap_single() does the reverse. If the DMA operation might have
85 ------------------------------
87 called by the corresponding DMA APIs which may run in contexts that cannot
88 block. Hence the default memory pool for swiotlb allocations must be
89 pre-allocated at boot time (but see Dynamic swiotlb below). Because swiotlb
90 allocations must be physically contiguous, the entire default memory pool is
93 The need to pre-allocate the default swiotlb pool creates a boot-time tradeoff.
94 The pool should be large enough to ensure that bounce buffer requests can
95 always be satisfied, as the non-blocking requirement means requests can't wait
96 for space to become available. But a large pool potentially wastes memory, as
97 this pre-allocated memory is not available for other uses in the system. The
98 tradeoff is particularly acute in CoCo VMs that use bounce buffers for all DMA
99 I/O. These VMs use a heuristic to set the default pool size to ~6% of memory,
104 default memory pool size remains an open issue.
107 bytes, which is 256 KiB with current definitions. When a device's DMA settings
108 are such that the device might use swiotlb, the maximum size of a DMA segment
109 must be limited to that 256 KiB. This value is communicated to higher-level
111 higher-level code fails to account for this limit, it may make requests that
114 A key device DMA setting is "min_align_mask", which is a power of 2 minus 1
118 min_align_mask is non-zero, it may produce an "alignment offset" in the address
124 swiotlb, max_sectors_kb will be 256 KiB. When min_align_mask is non-zero,
130 bounce buffer might start at a larger address if min_align_mask is non-zero.
131 Hence there may be pre-padding space that is allocated prior to the start of
133 alloc_align_mask boundary, potentially resulting in post-padding space. Any
134 pre-padding or post-padding space is not initialized by swiotlb code. The
136 devices. It is set to the granule size - 1 so that the bounce buffer is
140 ------------------------
142 as one or more "pools". The default pool is allocated during system boot with a
143 default size of 64 MiB. The default pool size may be modified with the
147 the life of the system. Each pool must be a contiguous range of physical
148 memory. The default pool is allocated below the 4 GiB physical address line so
149 it works for devices that can only address 32-bits of physical memory (unless
150 architecture-specific code provides the SWIOTLB_ANY flag). In a CoCo VM, the
151 pool memory must be decrypted before swiotlb is used.
153 Each pool is divided into "slots" of size IO_TLB_SIZE, which is 2 KiB with
156 occupies one or more contiguous slots. A slot is never shared by multiple
159 IO_TLB_SEGSIZE. Multiple smaller bounce buffers may co-exist in a single slot
183 The default pool is allocated with PAGE_SIZE alignment. If an alloc_align_mask
190 change in the future, the initial pool allocation might need to be done with
194 ---------------
195 When CONFIG_SWIOTLB_DYNAMIC is enabled, swiotlb can do on-demand expansion of
199 into an swiotlb pool. Creating an additional pool must be done asynchronously
202 buffer request creates a "transient pool" to avoid returning an "swiotlb full"
203 error. A transient pool has the size of the bounce buffer request, and is
204 deleted when the bounce buffer is freed. Memory for this transient pool comes
205 from the general system memory atomic pool so that creation does not block.
206 Creating a transient pool has relatively high cost, particularly in a CoCo VM
208 background task can add another non-transient pool.
210 Adding a dynamic pool has limitations. Like with the default pool, the memory
213 allocation may not be available. The dynamic pool allocator tries smaller sizes
215 memory fragmentation, dynamically adding a pool might not succeed at all.
217 The number of areas in a dynamic pool may be different from the number of areas
218 in the default pool. Because the new pool size is typically a few MiB at most,
219 the number of areas will likely be smaller. For example, with a new pool size
225 swiotlb code frequently must search for the pool containing a particular
231 few CPUs. It allows the default swiotlb pool to be smaller so that memory is
236 ----------------------
239 which includes the default memory pool and any dynamic or transient pools
244 io_tlb_pool describes a memory pool, either the default pool, a dynamic pool,
245 or a transient pool. The description includes the start and end addresses of
246 the memory in the pool, a pointer to an array of io_tlb_area structures, and a
247 pointer to an array of io_tlb_slot structures that are associated with the pool.
250 serialize access to slots in the area. The io_tlb_area array for a pool has an
251 entry for each area, and is accessed using a 0-based area index derived from the
255 io_tlb_slot describes an individual memory slot in the pool, with size
258 address of the pool. The size of struct io_tlb_slot is 24 bytes, so the
261 The io_tlb_slot array is designed to meet several requirements. First, the DMA
292 a new bounce buffer and when freeing a bounce buffer. At pool creation time, the
299 requirements, it may allocate pre-padding space across zero or more slots. But
304 The "pad_slots" value is recorded only in the first non-padding slot allocated
308 ----------------
310 memory separate from the default swiotlb pool, and that are dedicated for DMA
311 use by a particular device. Restricted pools provide a level of DMA memory
314 requires that CONFIG_DMA_RESTRICTED_POOL is set. Each restricted pool is based
320 allocate/free slots from/to the restricted pool directly and do not go through