1.. SPDX-License-Identifier: GPL-2.0 2 3.. _inline_encryption: 4 5================= 6Inline Encryption 7================= 8 9Background 10========== 11 12Inline encryption hardware sits logically between memory and the disk, and can 13en/decrypt data as it goes in/out of the disk. Inline encryption hardware has a 14fixed number of "keyslots" - slots into which encryption contexts (i.e. the 15encryption key, encryption algorithm, data unit size) can be programmed by the 16kernel at any time. Each request sent to the disk can be tagged with the index 17of a keyslot (and also a data unit number to act as an encryption tweak), and 18the inline encryption hardware will en/decrypt the data in the request with the 19encryption context programmed into that keyslot. This is very different from 20full disk encryption solutions like self encrypting drives/TCG OPAL/ATA 21Security standards, since with inline encryption, any block on disk could be 22encrypted with any encryption context the kernel chooses. 23 24 25Objective 26========= 27 28We want to support inline encryption (IE) in the kernel. 29To allow for testing, we also want a crypto API fallback when actual 30IE hardware is absent. We also want IE to work with layered devices 31like dm and loopback (i.e. we want to be able to use the IE hardware 32of the underlying devices if present, or else fall back to crypto API 33en/decryption). 34 35 36Constraints and notes 37===================== 38 39- IE hardware has a limited number of "keyslots" that can be programmed 40 with an encryption context (key, algorithm, data unit size, etc.) at any time. 41 One can specify a keyslot in a data request made to the device, and the 42 device will en/decrypt the data using the encryption context programmed into 43 that specified keyslot. When possible, we want to make multiple requests with 44 the same encryption context share the same keyslot. 45 46- We need a way for upper layers like filesystems to specify an encryption 47 context to use for en/decrypting a struct bio, and a device driver (like UFS) 48 needs to be able to use that encryption context when it processes the bio. 49 50- We need a way for device drivers to expose their inline encryption 51 capabilities in a unified way to the upper layers. 52 53 54Design 55====== 56 57We add a struct bio_crypt_ctx to struct bio that can 58represent an encryption context, because we need to be able to pass this 59encryption context from the upper layers (like the fs layer) to the 60device driver to act upon. 61 62While IE hardware works on the notion of keyslots, the FS layer has no 63knowledge of keyslots - it simply wants to specify an encryption context to 64use while en/decrypting a bio. 65 66We introduce a keyslot manager (KSM) that handles the translation from 67encryption contexts specified by the FS to keyslots on the IE hardware. 68This KSM also serves as the way IE hardware can expose its capabilities to 69upper layers. The generic mode of operation is: each device driver that wants 70to support IE will construct a KSM and set it up in its struct request_queue. 71Upper layers that want to use IE on this device can then use this KSM in 72the device's struct request_queue to translate an encryption context into 73a keyslot. The presence of the KSM in the request queue shall be used to mean 74that the device supports IE. 75 76The KSM uses refcounts to track which keyslots are idle (either they have no 77encryption context programmed, or there are no in-flight struct bios 78referencing that keyslot). When a new encryption context needs a keyslot, it 79tries to find a keyslot that has already been programmed with the same 80encryption context, and if there is no such keyslot, it evicts the least 81recently used idle keyslot and programs the new encryption context into that 82one. If no idle keyslots are available, then the caller will sleep until there 83is at least one. 84 85 86blk-mq changes, other block layer changes and blk-crypto-fallback 87================================================================= 88 89We add a pointer to a ``bi_crypt_context`` and ``keyslot`` to 90struct request. These will be referred to as the ``crypto fields`` 91for the request. This ``keyslot`` is the keyslot into which the 92``bi_crypt_context`` has been programmed in the KSM of the ``request_queue`` 93that this request is being sent to. 94 95We introduce ``block/blk-crypto-fallback.c``, which allows upper layers to remain 96blissfully unaware of whether or not real inline encryption hardware is present 97underneath. When a bio is submitted with a target ``request_queue`` that doesn't 98support the encryption context specified with the bio, the block layer will 99en/decrypt the bio with the blk-crypto-fallback. 100 101If the bio is a ``WRITE`` bio, a bounce bio is allocated, and the data in the bio 102is encrypted stored in the bounce bio - blk-mq will then proceed to process the 103bounce bio as if it were not encrypted at all (except when blk-integrity is 104concerned). ``blk-crypto-fallback`` sets the bounce bio's ``bi_end_io`` to an 105internal function that cleans up the bounce bio and ends the original bio. 106 107If the bio is a ``READ`` bio, the bio's ``bi_end_io`` (and also ``bi_private``) 108is saved and overwritten by ``blk-crypto-fallback`` to 109``bio_crypto_fallback_decrypt_bio``. The bio's ``bi_crypt_context`` is also 110overwritten with ``NULL``, so that to the rest of the stack, the bio looks 111as if it was a regular bio that never had an encryption context specified. 112``bio_crypto_fallback_decrypt_bio`` will decrypt the bio, restore the original 113``bi_end_io`` (and also ``bi_private``) and end the bio again. 114 115Regardless of whether real inline encryption hardware is used or the 116blk-crypto-fallback is used, the ciphertext written to disk (and hence the 117on-disk format of data) will be the same (assuming the hardware's implementation 118of the algorithm being used adheres to spec and functions correctly). 119 120If a ``request queue``'s inline encryption hardware claimed to support the 121encryption context specified with a bio, then it will not be handled by the 122``blk-crypto-fallback``. We will eventually reach a point in blk-mq when a 123struct request needs to be allocated for that bio. At that point, 124blk-mq tries to program the encryption context into the ``request_queue``'s 125keyslot_manager, and obtain a keyslot, which it stores in its newly added 126``keyslot`` field. This keyslot is released when the request is completed. 127 128When the first bio is added to a request, ``blk_crypto_rq_bio_prep`` is called, 129which sets the request's ``crypt_ctx`` to a copy of the bio's 130``bi_crypt_context``. bio_crypt_do_front_merge is called whenever a subsequent 131bio is merged to the front of the request, which updates the ``crypt_ctx`` of 132the request so that it matches the newly merged bio's ``bi_crypt_context``. In particular, the request keeps a copy of the ``bi_crypt_context`` of the first 133bio in its bio-list (blk-mq needs to be careful to maintain this invariant 134during bio and request merges). 135 136To make it possible for inline encryption to work with request queue based 137layered devices, when a request is cloned, its ``crypto fields`` are cloned as 138well. When the cloned request is submitted, blk-mq programs the 139``bi_crypt_context`` of the request into the clone's request_queue's keyslot 140manager, and stores the returned keyslot in the clone's ``keyslot``. 141 142 143API presented to users of the block layer 144========================================= 145 146``struct blk_crypto_key`` represents a crypto key (the raw key, size of the 147key, the crypto algorithm to use, the data unit size to use, and the number of 148bytes required to represent data unit numbers that will be specified with the 149``bi_crypt_context``). 150 151``blk_crypto_init_key`` allows upper layers to initialize such a 152``blk_crypto_key``. 153 154``bio_crypt_set_ctx`` should be called on any bio that a user of 155the block layer wants en/decrypted via inline encryption (or the 156blk-crypto-fallback, if hardware support isn't available for the desired 157crypto configuration). This function takes the ``blk_crypto_key`` and the 158data unit number (DUN) to use when en/decrypting the bio. 159 160``blk_crypto_config_supported`` allows upper layers to query whether or not the 161an encryption context passed to request queue can be handled by blk-crypto 162(either by real inline encryption hardware, or by the blk-crypto-fallback). 163This is useful e.g. when blk-crypto-fallback is disabled, and the upper layer 164wants to use an algorithm that may not supported by hardware - this function 165lets the upper layer know ahead of time that the algorithm isn't supported, 166and the upper layer can fallback to something else if appropriate. 167 168``blk_crypto_start_using_key`` - Upper layers must call this function on 169``blk_crypto_key`` and a ``request_queue`` before using the key with any bio 170headed for that ``request_queue``. This function ensures that either the 171hardware supports the key's crypto settings, or the crypto API fallback has 172transforms for the needed mode allocated and ready to go. Note that this 173function may allocate an ``skcipher``, and must not be called from the data 174path, since allocating ``skciphers`` from the data path can deadlock. 175 176``blk_crypto_evict_key`` *must* be called by upper layers before a 177``blk_crypto_key`` is freed. Further, it *must* only be called only once 178there are no more in-flight requests that use that ``blk_crypto_key``. 179``blk_crypto_evict_key`` will ensure that a key is removed from any keyslots in 180inline encryption hardware that the key might have been programmed into (or the blk-crypto-fallback). 181 182API presented to device drivers 183=============================== 184 185A :c:type:``struct blk_keyslot_manager`` should be set up by device drivers in 186the ``request_queue`` of the device. The device driver needs to call 187``blk_ksm_init`` (or its resource-managed variant ``devm_blk_ksm_init``) on the 188``blk_keyslot_manager``, while specifying the number of keyslots supported by 189the hardware. 190 191The device driver also needs to tell the KSM how to actually manipulate the 192IE hardware in the device to do things like programming the crypto key into 193the IE hardware into a particular keyslot. All this is achieved through the 194struct blk_ksm_ll_ops field in the KSM that the device driver 195must fill up after initing the ``blk_keyslot_manager``. 196 197The KSM also handles runtime power management for the device when applicable 198(e.g. when it wants to program a crypto key into the IE hardware, the device 199must be runtime powered on) - so the device driver must also set the ``dev`` 200field in the ksm to point to the `struct device` for the KSM to use for runtime 201power management. 202 203``blk_ksm_reprogram_all_keys`` can be called by device drivers if the device 204needs each and every of its keyslots to be reprogrammed with the key it 205"should have" at the point in time when the function is called. This is useful 206e.g. if a device loses all its keys on runtime power down/up. 207 208If the driver used ``blk_ksm_init`` instead of ``devm_blk_ksm_init``, then 209``blk_ksm_destroy`` should be called to free up all resources used by a 210``blk_keyslot_manager`` once it is no longer needed. 211 212Layered Devices 213=============== 214 215Request queue based layered devices like dm-rq that wish to support IE need to 216create their own keyslot manager for their request queue, and expose whatever 217functionality they choose. When a layered device wants to pass a clone of that 218request to another ``request_queue``, blk-crypto will initialize and prepare the 219clone as necessary - see ``blk_crypto_insert_cloned_request`` in 220``blk-crypto.c``. 221 222 223Future Optimizations for layered devices 224======================================== 225 226Creating a keyslot manager for a layered device uses up memory for each 227keyslot, and in general, a layered device merely passes the request on to a 228"child" device, so the keyslots in the layered device itself are completely 229unused, and don't need any refcounting or keyslot programming. We can instead 230define a new type of KSM; the "passthrough KSM", that layered devices can use 231to advertise an unlimited number of keyslots, and support for any encryption 232algorithms they choose, while not actually using any memory for each keyslot. 233Another use case for the "passthrough KSM" is for IE devices that do not have a 234limited number of keyslots. 235 236 237Interaction between inline encryption and blk integrity 238======================================================= 239 240At the time of this patch, there is no real hardware that supports both these 241features. However, these features do interact with each other, and it's not 242completely trivial to make them both work together properly. In particular, 243when a WRITE bio wants to use inline encryption on a device that supports both 244features, the bio will have an encryption context specified, after which 245its integrity information is calculated (using the plaintext data, since 246the encryption will happen while data is being written), and the data and 247integrity info is sent to the device. Obviously, the integrity info must be 248verified before the data is encrypted. After the data is encrypted, the device 249must not store the integrity info that it received with the plaintext data 250since that might reveal information about the plaintext data. As such, it must 251re-generate the integrity info from the ciphertext data and store that on disk 252instead. Another issue with storing the integrity info of the plaintext data is 253that it changes the on disk format depending on whether hardware inline 254encryption support is present or the kernel crypto API fallback is used (since 255if the fallback is used, the device will receive the integrity info of the 256ciphertext, not that of the plaintext). 257 258Because there isn't any real hardware yet, it seems prudent to assume that 259hardware implementations might not implement both features together correctly, 260and disallow the combination for now. Whenever a device supports integrity, the 261kernel will pretend that the device does not support hardware inline encryption 262(by essentially setting the keyslot manager in the request_queue of the device 263to NULL). When the crypto API fallback is enabled, this means that all bios with 264and encryption context will use the fallback, and IO will complete as usual. 265When the fallback is disabled, a bio with an encryption context will be failed. 266