1Engines 2======= 3 4Deprecation Note 5---------------- 6 7The ENGINE API was introduced in OpenSSL version 0.9.6 as a low level 8interface for adding alternative implementations of cryptographic 9primitives, most notably for integrating hardware crypto devices. 10 11The ENGINE interface has its limitations and it has been superseeded 12by the [PROVIDER API](README-PROVIDERS.md), it is deprecated in OpenSSL 13version 3.0. The following documentation is retained as an aid for 14users who need to maintain or support existing ENGINE implementations. 15Support for new hardware devices or new algorithms should be added 16via providers, and existing engines should be converted to providers 17as soon as possible. 18 19Built-in ENGINE implementations 20------------------------------- 21 22There are currently built-in ENGINE implementations for the following 23crypto devices: 24 25 * Microsoft CryptoAPI 26 * VIA Padlock 27 * nCipher CHIL 28 29In addition, dynamic binding to external ENGINE implementations is now 30provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE" 31section below for details. 32 33At this stage, a number of things are still needed and are being worked on: 34 35 1. Integration of EVP support. 36 2. Configuration support. 37 3. Documentation! 38 39Integration of EVP support 40-------------------------- 41 42With respect to EVP, this relates to support for ciphers and digests in 43the ENGINE model so that alternative implementations of existing 44algorithms/modes (or previously unimplemented ones) can be provided by 45ENGINE implementations. 46 47Configuration support 48--------------------- 49 50Configuration support currently exists in the ENGINE API itself, in the 51form of "control commands". These allow an application to expose to the 52user/admin the set of commands and parameter types a given ENGINE 53implementation supports, and for an application to directly feed string 54based input to those ENGINEs, in the form of name-value pairs. This is an 55extensible way for ENGINEs to define their own "configuration" mechanisms 56that are specific to a given ENGINE (eg. for a particular hardware 57device) but that should be consistent across *all* OpenSSL-based 58applications when they use that ENGINE. Work is in progress (or at least 59in planning) for supporting these control commands from the CONF (or 60NCONF) code so that applications using OpenSSL's existing configuration 61file format can have ENGINE settings specified in much the same way. 62Presently however, applications must use the ENGINE API itself to provide 63such functionality. To see first hand the types of commands available 64with the various compiled-in ENGINEs (see further down for dynamic 65ENGINEs), use the "engine" openssl utility with full verbosity, i.e.: 66 67 openssl engine -vvvv 68 69Documentation 70------------- 71 72Documentation? Volunteers welcome! The source code is reasonably well 73self-documenting, but some summaries and usage instructions are needed - 74moreover, they are needed in the same POD format the existing OpenSSL 75documentation is provided in. Any complete or incomplete contributions 76would help make this happen. 77 78STABILITY & BUG-REPORTS 79======================= 80 81What already exists is fairly stable as far as it has been tested, but 82the test base has been a bit small most of the time. For the most part, 83the vendors of the devices these ENGINEs support have contributed to the 84development and/or testing of the implementations, and *usually* (with no 85guarantees) have experience in using the ENGINE support to drive their 86devices from common OpenSSL-based applications. Bugs and/or inexplicable 87behaviour in using a specific ENGINE implementation should be sent to the 88author of that implementation (if it is mentioned in the corresponding C 89file), and in the case of implementations for commercial hardware 90devices, also through whatever vendor support channels are available. If 91none of this is possible, or the problem seems to be something about the 92ENGINE API itself (ie. not necessarily specific to a particular ENGINE 93implementation) then you should mail complete details to the relevant 94OpenSSL mailing list. For a definition of "complete details", refer to 95the OpenSSL "README" file. As for which list to send it to: 96 97 * openssl-users: if you are *using* the ENGINE abstraction, either in an 98 pre-compiled application or in your own application code. 99 100 * openssl-dev: if you are discussing problems with OpenSSL source code. 101 102USAGE 103===== 104 105The default "openssl" ENGINE is always chosen when performing crypto 106operations unless you specify otherwise. You must actively tell the 107openssl utility commands to use anything else through a new command line 108switch called "-engine". Also, if you want to use the ENGINE support in 109your own code to do something similar, you must likewise explicitly 110select the ENGINE implementation you want. 111 112Depending on the type of hardware, system, and configuration, "settings" 113may need to be applied to an ENGINE for it to function as expected/hoped. 114The recommended way of doing this is for the application to support 115ENGINE "control commands" so that each ENGINE implementation can provide 116whatever configuration primitives it might require and the application 117can allow the user/admin (and thus the hardware vendor's support desk 118also) to provide any such input directly to the ENGINE implementation. 119This way, applications do not need to know anything specific to any 120device, they only need to provide the means to carry such user/admin 121input through to the ENGINE in question. Ie. this connects *you* (and 122your helpdesk) to the specific ENGINE implementation (and device), and 123allows application authors to not get buried in hassle supporting 124arbitrary devices they know (and care) nothing about. 125 126A new "openssl" utility, "openssl engine", has been added in that allows 127for testing and examination of ENGINE implementations. Basic usage 128instructions are available by specifying the "-?" command line switch. 129 130DYNAMIC ENGINES 131=============== 132 133The new "dynamic" ENGINE provides a low-overhead way to support ENGINE 134implementations that aren't pre-compiled and linked into OpenSSL-based 135applications. This could be because existing compiled-in implementations 136have known problems and you wish to use a newer version with an existing 137application. It could equally be because the application (or OpenSSL 138library) you are using simply doesn't have support for the ENGINE you 139wish to use, and the ENGINE provider (eg. hardware vendor) is providing 140you with a self-contained implementation in the form of a shared-library. 141The other use-case for "dynamic" is with applications that wish to 142maintain the smallest foot-print possible and so do not link in various 143ENGINE implementations from OpenSSL, but instead leaves you to provide 144them, if you want them, in the form of "dynamic"-loadable 145shared-libraries. It should be possible for hardware vendors to provide 146their own shared-libraries to support arbitrary hardware to work with 147applications based on OpenSSL 0.9.7 or later. If you're using an 148application based on 0.9.7 (or later) and the support you desire is only 149announced for versions later than the one you need, ask the vendor to 150backport their ENGINE to the version you need. 151 152How does "dynamic" work? 153------------------------ 154 155The dynamic ENGINE has a special flag in its implementation such that 156every time application code asks for the 'dynamic' ENGINE, it in fact 157gets its own copy of it. As such, multi-threaded code (or code that 158multiplexes multiple uses of 'dynamic' in a single application in any 159way at all) does not get confused by 'dynamic' being used to do many 160independent things. Other ENGINEs typically don't do this so there is 161only ever 1 ENGINE structure of its type (and reference counts are used 162to keep order). The dynamic ENGINE itself provides absolutely no 163cryptographic functionality, and any attempt to "initialise" the ENGINE 164automatically fails. All it does provide are a few "control commands" 165that can be used to control how it will load an external ENGINE 166implementation from a shared-library. To see these control commands, 167use the command-line; 168 169 openssl engine -vvvv dynamic 170 171The "SO_PATH" control command should be used to identify the 172shared-library that contains the ENGINE implementation, and "NO_VCHECK" 173might possibly be useful if there is a minor version conflict and you 174(or a vendor helpdesk) is convinced you can safely ignore it. 175"ID" is probably only needed if a shared-library implements 176multiple ENGINEs, but if you know the engine id you expect to be using, 177it doesn't hurt to specify it (and this provides a sanity check if 178nothing else). "LIST_ADD" is only required if you actually wish the 179loaded ENGINE to be discoverable by application code later on using the 180ENGINE's "id". For most applications, this isn't necessary - but some 181application authors may have nifty reasons for using it. The "LOAD" 182command is the only one that takes no parameters and is the command 183that uses the settings from any previous commands to actually *load* 184the shared-library ENGINE implementation. If this command succeeds, the 185(copy of the) 'dynamic' ENGINE will magically morph into the ENGINE 186that has been loaded from the shared-library. As such, any control 187commands supported by the loaded ENGINE could then be executed as per 188normal. Eg. if ENGINE "foo" is implemented in the shared-library 189"libfoo.so" and it supports some special control command "CMD_FOO", the 190following code would load and use it (NB: obviously this code has no 191error checking); 192 193 ENGINE *e = ENGINE_by_id("dynamic"); 194 ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0); 195 ENGINE_ctrl_cmd_string(e, "ID", "foo", 0); 196 ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0); 197 ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0); 198 199For testing, the "openssl engine" utility can be useful for this sort 200of thing. For example the above code excerpt would achieve much the 201same result as; 202 203 openssl engine dynamic \ 204 -pre SO_PATH:/lib/libfoo.so \ 205 -pre ID:foo \ 206 -pre LOAD \ 207 -pre "CMD_FOO:some input data" 208 209Or to simply see the list of commands supported by the "foo" ENGINE; 210 211 openssl engine -vvvv dynamic \ 212 -pre SO_PATH:/lib/libfoo.so \ 213 -pre ID:foo \ 214 -pre LOAD 215 216Applications that support the ENGINE API and more specifically, the 217"control commands" mechanism, will provide some way for you to pass 218such commands through to ENGINEs. As such, you would select "dynamic" 219as the ENGINE to use, and the parameters/commands you pass would 220control the *actual* ENGINE used. Each command is actually a name-value 221pair and the value can sometimes be omitted (eg. the "LOAD" command). 222Whilst the syntax demonstrated in "openssl engine" uses a colon to 223separate the command name from the value, applications may provide 224their own syntax for making that separation (eg. a win32 registry 225key-value pair may be used by some applications). The reason for the 226"-pre" syntax in the "openssl engine" utility is that some commands 227might be issued to an ENGINE *after* it has been initialised for use. 228Eg. if an ENGINE implementation requires a smart-card to be inserted 229during initialisation (or a PIN to be typed, or whatever), there may be 230a control command you can issue afterwards to "forget" the smart-card 231so that additional initialisation is no longer possible. In 232applications such as web-servers, where potentially volatile code may 233run on the same host system, this may provide some arguable security 234value. In such a case, the command would be passed to the ENGINE after 235it has been initialised for use, and so the "-post" switch would be 236used instead. Applications may provide a different syntax for 237supporting this distinction, and some may simply not provide it at all 238("-pre" is almost always what you're after, in reality). 239 240How do I build a "dynamic" ENGINE? 241---------------------------------- 242 243This question is trickier - currently OpenSSL bundles various ENGINE 244implementations that are statically built in, and any application that 245calls the "ENGINE_load_builtin_engines()" function will automatically 246have all such ENGINEs available (and occupying memory). Applications 247that don't call that function have no ENGINEs available like that and 248would have to use "dynamic" to load any such ENGINE - but on the other 249hand such applications would only have the memory footprint of any 250ENGINEs explicitly loaded using user/admin provided control commands. 251The main advantage of not statically linking ENGINEs and only using 252"dynamic" for hardware support is that any installation using no 253"external" ENGINE suffers no unnecessary memory footprint from unused 254ENGINEs. Likewise, installations that do require an ENGINE incur the 255overheads from only *that* ENGINE once it has been loaded. 256 257Sounds good? Maybe, but currently building an ENGINE implementation as 258a shared-library that can be loaded by "dynamic" isn't automated in 259OpenSSL's build process. It can be done manually quite easily however. 260Such a shared-library can either be built with any OpenSSL code it 261needs statically linked in, or it can link dynamically against OpenSSL 262if OpenSSL itself is built as a shared library. The instructions are 263the same in each case, but in the former (statically linked any 264dependencies on OpenSSL) you must ensure OpenSSL is built with 265position-independent code ("PIC"). The default OpenSSL compilation may 266already specify the relevant flags to do this, but you should consult 267with your compiler documentation if you are in any doubt. 268 269This example will show building the "atalla" ENGINE in the 270crypto/engine/ directory as a shared-library for use via the "dynamic" 271ENGINE. 272 273 1. "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL 274 source tree. 275 276 2. Recompile at least one source file so you can see all the compiler 277 flags (and syntax) being used to build normally. Eg; 278 279 touch hw_atalla.c ; make 280 281 will rebuild "hw_atalla.o" using all such flags. 282 283 3. Manually enter the same compilation line to compile the 284 "hw_atalla.c" file but with the following two changes; 285 * add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches, 286 * change the output file from "hw_atalla.o" to something new, 287 eg. "tmp_atalla.o" 288 289 4. Link "tmp_atalla.o" into a shared-library using the top-level 290 OpenSSL libraries to resolve any dependencies. The syntax for doing 291 this depends heavily on your system/compiler and is a nightmare 292 known well to anyone who has worked with shared-library portability 293 before. 'gcc' on Linux, for example, would use the following syntax; 294 295 gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto 296 297 5. Test your shared library using "openssl engine" as explained in the 298 previous section. Eg. from the top-level directory, you might try 299 300 apps/openssl engine -vvvv dynamic \ 301 -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD 302 303If the shared-library loads successfully, you will see both "-pre" 304commands marked as "SUCCESS" and the list of control commands 305displayed (because of "-vvvv") will be the control commands for the 306*atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add 307the "-t" switch to the utility if you want it to try and initialise 308the atalla ENGINE for use to test any possible hardware/driver issues. 309 310PROBLEMS 311======== 312 313It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32. 314A quick test done right before the release showed that trying "openssl speed 315-engine cswift" generated errors. If the DSO gets enabled, an attempt is made 316to write at memory address 0x00000002. 317