1# Mbed TLS PSA keystore format stability testing strategy 2 3## Introduction 4 5The PSA crypto subsystem includes a persistent key store. It is possible to create a persistent key and read it back later. This must work even if Mbed TLS has been upgraded in the meantime (except for deliberate breaks in the backward compatibility of the storage). 6 7The goal of this document is to define a test strategy for the key store that not only validates that it's possible to load a key that was saved with the version of Mbed TLS under test, but also that it's possible to load a key that was saved with previous versions of Mbed TLS. 8 9Interoperability is not a goal: PSA crypto implementations are not intended to have compatible storage formats. Downgrading is not required to work. 10 11## General approach 12 13### Limitations of a direct approach 14 15The goal of storage format stability testing is: as a user of Mbed TLS, I want to store a key under version V and read it back under version W, with W ≥ V. 16 17Doing the testing this way would be difficult because we'd need to have version V of Mbed TLS available when testing version W. 18 19An alternative, semi-direct approach consists of generating test data under version V, and reading it back under version W. Done naively, this would require keeping a large amount of test data (full test coverage multiplied by the number of versions that we want to preserve backward compatibility with). 20 21### Save-and-compare approach 22 23Importing and saving a key is deterministic. Therefore we can ensure the stability of the storage format by creating test cases under a version V of Mbed TLS, where the test case parameters include both the parameters to pass to key creation and the expected state of the storage after the key is created. The test case creates a key as indicated by the parameters, then compares the actual state of the storage with the expected state. 24 25In addition, the test case also loads the key and checks that it has the expected data and metadata. Import-and-save testing and load-and-check testing can be split into separate test functions with the same payloads. 26 27If the test passes with version V, this means that the test data is consistent with what the implementation does. When the test later runs under version W ≥ V, it creates and reads back a storage state which is known to be identical to the state that V would have produced. Thus, this approach validates that W can read storage states created by V. 28 29Note that it is the combination of import-and-save passing on version V and load-and-check passing on version W with the same data that proves that version W can read back what version V wrote. From the perspective of a particular version of the library, the import-and-save tests guarantee forward compatibility while the load-and-check tests guarantee backward compatibility. 30 31Use a similar approach for files other than keys where possible and relevant. 32 33### Keeping up with storage format evolution 34 35Test cases should normally not be removed from the code base: if something has worked before, it should keep working in future versions, so we should keep testing it. 36 37This cannot be enforced solely by looking at a single version of Mbed TLS, since there would be no indication that more test cases used to exist. It can only be enforced through review of library changes. The review may be assisted by a tool that compares the old and the new version, in the same way that `abi-check.py` compares the library's API and ABI. 38 39If the way certain keys are stored changes, and we don't deliberately decide to stop supporting old keys (which should only be done by retiring a version of the storage format), then we should keep the corresponding test cases in load-only mode: create a file with the expected content, load it and check the data that it contains. 40 41## Storage architecture overview 42 43The PSA subsystem provides storage on top of the PSA trusted storage interface. The state of the storage is a mapping from file identifer (a 64-bit number) to file content (a byte array). These files include: 44 45* [Key files](#key-storage) (files containing one key's metadata and, except for some secure element keys, key material). 46* The [random generator injected seed or state file](#random-generator-state) (`PSA_CRYPTO_ITS_RANDOM_SEED_UID`). 47* [Storage transaction file](#storage-transaction-resumption). 48* [Driver state files](#driver-state-files). 49 50For a more detailed description, refer to the [Mbed Crypto storage specification](../mbed-crypto-storage-specification.md). 51 52In addition, Mbed TLS includes an implementation of the PSA trusted storage interface on top of C stdio. This document addresses the test strategy for [PSA ITS over file](#psa-its-over-file) in a separate section below. 53 54## Key storage testing 55 56This section describes the desired test cases for keys created with the current storage format version. When the storage format changes, if backward compatibility is desired, old test data should be kept as described under [“Keeping up with storage format evolution”](#keeping-up-with-storage-format-evolution). 57 58### Keystore layout 59 60Objective: test that the key file name corresponds to the key identifier. 61 62Method: Create a key with a given identifier (using `psa_import_key`) and verify that a file with the expected name is created, and no other. Repeat for different identifiers. 63 64### General key format 65 66Objective: test the format of the key file: which field goes where and how big it is. 67 68Method: Create a key with certain metadata with `psa_import_key`. Read the file content and validate that it has the expected layout, deduced from the storage specification. Repeat with different metadata. Ensure that there are test cases covering all fields. 69 70### Enumeration of test cases for keys 71 72Objective: ensure that the coverage is sufficient to have assurance that all keys are stored correctly. This requires a sufficient selection of key types, sizes, policies, etc. 73 74In particular, the tests must validate that each `PSA_xxx` constant that is stored in a key is covered by at least one test case: 75 76* Lifetimes: `PSA_KEY_LIFETIME_xxx`, `PSA_KEY_PERSISTENCE_xxx`, `PSA_KEY_LOCATION_xxx`. 77* Usage flags: `PSA_KEY_USAGE_xxx`. 78* Algorithms in policies: `PSA_ALG_xxx`. 79* Key types: `PSA_KEY_TYPE_xxx`, `PSA_ECC_FAMILY_xxx`, `PSA_DH_FAMILY_xxx`. 80 81In addition, the coverage of key material must ensure that any variation in key representation is detected. See [“Considerations on key material representations”](#Considerations-on-key-material-representations) for considerations regarding key types. 82 83Method: Each test case creates a key with `psa_import_key`, purges it from memory, then reads it back and exercises it. 84 85Generate test cases automatically based on an enumeration of available constants and some knowledge of what attributes (sizes, algorithms, …) and content to use for keys of a certain type. 86 87### Testing with alternative lifetime values 88 89Objective: have test coverage for lifetimes other than the default persistent lifetime (`PSA_KEY_LIFETIME_PERSISTENT`). 90 91Method: 92 93* For alternative locations: have tests conditional on the presence of a driver for that location. 94* For alternative persistence levels: have load-and-check tests for supported persistence levels. We may also want to have negative tests ensuring that keys with a not-supported persistence level are not accidentally created. 95 96### Considerations on key material representations 97 98The risks of incompatibilities in key representations depends on the key type and on the presence of drivers. Compatibility of and with drivers is currently out of scope of this document. 99 100Some types only have one plausible representation. Others admit alternative plausible representations (different encodings, or non-canonical representations). 101Here are some areas to watch for, with an identified risk of incompatibilities. 102 103* HMAC keys longer than the block size: pre-hashed or not? 104* DES keys: was parity enforced? 105* RSA keys: can invalid DER encodings (e.g. leading zeros, ignored sign bit) have been stored? 106* RSA private keys: can invalid CRT parameters have been stored? 107* Montgomery private keys: were they stored in masked form? 108 109## Random generator state 110 111TODO 112 113## Driver state files 114 115Not yet implemented. 116 117TODO 118 119## Storage transaction resumption 120 121Only relevant for secure element support. Not yet fully implemented. 122 123TODO 124 125## PSA ITS over file 126 127TODO 128