readme.md
1# asn1crypto
2
3A fast, pure Python library for parsing and serializing ASN.1 structures.
4
5 - [Features](#features)
6 - [Why Another Python ASN.1 Library?](#why-another-python-asn1-library)
7 - [Related Crypto Libraries](#related-crypto-libraries)
8 - [Current Release](#current-release)
9 - [Dependencies](#dependencies)
10 - [Installation](#installation)
11 - [License](#license)
12 - [Documentation](#documentation)
13 - [Continuous Integration](#continuous-integration)
14 - [Testing](#testing)
15 - [Development](#development)
16 - [CI Tasks](#ci-tasks)
17
18[](https://github.com/wbond/asn1crypto/actions?workflow=CI)
19[](https://travis-ci.org/wbond/asn1crypto)
20[](https://ci.appveyor.com/project/wbond/asn1crypto)
21[](https://circleci.com/gh/wbond/asn1crypto)
22[](https://codecov.io/gh/wbond/asn1crypto)
23[](https://pypi.org/project/asn1crypto/)
24
25## Features
26
27In addition to an ASN.1 BER/DER decoder and DER serializer, the project includes
28a bunch of ASN.1 structures for use with various common cryptography standards:
29
30| Standard | Module | Source |
31| ---------------------- | ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
32| X.509 | [`asn1crypto.x509`](asn1crypto/x509.py) | [RFC 5280](https://tools.ietf.org/html/rfc5280) |
33| CRL | [`asn1crypto.crl`](asn1crypto/crl.py) | [RFC 5280](https://tools.ietf.org/html/rfc5280) |
34| CSR | [`asn1crypto.csr`](asn1crypto/csr.py) | [RFC 2986](https://tools.ietf.org/html/rfc2986), [RFC 2985](https://tools.ietf.org/html/rfc2985) |
35| OCSP | [`asn1crypto.ocsp`](asn1crypto/ocsp.py) | [RFC 6960](https://tools.ietf.org/html/rfc6960) |
36| PKCS#12 | [`asn1crypto.pkcs12`](asn1crypto/pkcs12.py) | [RFC 7292](https://tools.ietf.org/html/rfc7292) |
37| PKCS#8 | [`asn1crypto.keys`](asn1crypto/keys.py) | [RFC 5208](https://tools.ietf.org/html/rfc5208) |
38| PKCS#1 v2.1 (RSA keys) | [`asn1crypto.keys`](asn1crypto/keys.py) | [RFC 3447](https://tools.ietf.org/html/rfc3447) |
39| DSA keys | [`asn1crypto.keys`](asn1crypto/keys.py) | [RFC 3279](https://tools.ietf.org/html/rfc3279) |
40| Elliptic curve keys | [`asn1crypto.keys`](asn1crypto/keys.py) | [SECG SEC1 V2](http://www.secg.org/sec1-v2.pdf) |
41| PKCS#3 v1.4 | [`asn1crypto.algos`](asn1crypto/algos.py) | [PKCS#3 v1.4](ftp://ftp.rsasecurity.com/pub/pkcs/ascii/pkcs-3.asc) |
42| PKCS#5 v2.1 | [`asn1crypto.algos`](asn1crypto/algos.py) | [PKCS#5 v2.1](http://www.emc.com/collateral/white-papers/h11302-pkcs5v2-1-password-based-cryptography-standard-wp.pdf) |
43| CMS (and PKCS#7) | [`asn1crypto.cms`](asn1crypto/cms.py) | [RFC 5652](https://tools.ietf.org/html/rfc5652), [RFC 2315](https://tools.ietf.org/html/rfc2315) |
44| TSP | [`asn1crypto.tsp`](asn1crypto/tsp.py) | [RFC 3161](https://tools.ietf.org/html/rfc3161) |
45| PDF signatures | [`asn1crypto.pdf`](asn1crypto/pdf.py) | [PDF 1.7](http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/pdf/pdfs/PDF32000_2008.pdf) |
46
47## Why Another Python ASN.1 Library?
48
49Python has long had the [pyasn1](https://pypi.org/project/pyasn1/) and
50[pyasn1_modules](https://pypi.org/project/pyasn1-modules/) available for
51parsing and serializing ASN.1 structures. While the project does include a
52comprehensive set of tools for parsing and serializing, the performance of the
53library can be very poor, especially when dealing with bit fields and parsing
54large structures such as CRLs.
55
56After spending extensive time using *pyasn1*, the following issues were
57identified:
58
59 1. Poor performance
60 2. Verbose, non-pythonic API
61 3. Out-dated and incomplete definitions in *pyasn1-modules*
62 4. No simple way to map data to native Python data structures
63 5. No mechanism for overridden universal ASN.1 types
64
65The *pyasn1* API is largely method driven, and uses extensive configuration
66objects and lowerCamelCase names. There were no consistent options for
67converting types of native Python data structures. Since the project supports
68out-dated versions of Python, many newer language features are unavailable
69for use.
70
71Time was spent trying to profile issues with the performance, however the
72architecture made it hard to pin down the primary source of the poor
73performance. Attempts were made to improve performance by utilizing unreleased
74patches and delaying parsing using the `Any` type. Even with such changes, the
75performance was still unacceptably slow.
76
77Finally, a number of structures in the cryptographic space use universal data
78types such as `BitString` and `OctetString`, but interpret the data as other
79types. For instance, signatures are really byte strings, but are encoded as
80`BitString`. Elliptic curve keys use both `BitString` and `OctetString` to
81represent integers. Parsing these structures as the base universal types and
82then re-interpreting them wastes computation.
83
84*asn1crypto* uses the following techniques to improve performance, especially
85when extracting one or two fields from large, complex structures:
86
87 - Delayed parsing of byte string values
88 - Persistence of original ASN.1 encoded data until a value is changed
89 - Lazy loading of child fields
90 - Utilization of high-level Python stdlib modules
91
92While there is no extensive performance test suite, the
93`CRLTests.test_parse_crl` test case was used to parse a 21MB CRL file on a
94late 2013 rMBP. *asn1crypto* parsed the certificate serial numbers in just
95under 8 seconds. With *pyasn1*, using definitions from *pyasn1-modules*, the
96same parsing took over 4,100 seconds.
97
98For smaller structures the performance difference can range from a few times
99faster to an order of magnitude or more.
100
101## Related Crypto Libraries
102
103*asn1crypto* is part of the modularcrypto family of Python packages:
104
105 - [asn1crypto](https://github.com/wbond/asn1crypto)
106 - [oscrypto](https://github.com/wbond/oscrypto)
107 - [csrbuilder](https://github.com/wbond/csrbuilder)
108 - [certbuilder](https://github.com/wbond/certbuilder)
109 - [crlbuilder](https://github.com/wbond/crlbuilder)
110 - [ocspbuilder](https://github.com/wbond/ocspbuilder)
111 - [certvalidator](https://github.com/wbond/certvalidator)
112
113## Current Release
114
1151.3.0 - [changelog](changelog.md)
116
117## Dependencies
118
119Python 2.6, 2.7, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8 or pypy. *No third-party
120packages required.*
121
122## Installation
123
124```bash
125pip install asn1crypto
126```
127
128## License
129
130*asn1crypto* is licensed under the terms of the MIT license. See the
131[LICENSE](LICENSE) file for the exact license text.
132
133## Documentation
134
135The documentation for *asn1crypto* is composed of tutorials on basic usage and
136links to the source for the various pre-defined type classes.
137
138### Tutorials
139
140 - [Universal Types with BER/DER Decoder and DER Encoder](docs/universal_types.md)
141 - [PEM Encoder and Decoder](docs/pem.md)
142
143### Reference
144
145 - [Universal types](asn1crypto/core.py), `asn1crypto.core`
146 - [Digest, HMAC, signed digest and encryption algorithms](asn1crypto/algos.py), `asn1crypto.algos`
147 - [Private and public keys](asn1crypto/keys.py), `asn1crypto.keys`
148 - [X509 certificates](asn1crypto/x509.py), `asn1crypto.x509`
149 - [Certificate revocation lists (CRLs)](asn1crypto/crl.py), `asn1crypto.crl`
150 - [Online certificate status protocol (OCSP)](asn1crypto/ocsp.py), `asn1crypto.ocsp`
151 - [Certificate signing requests (CSRs)](asn1crypto/csr.py), `asn1crypto.csr`
152 - [Private key/certificate containers (PKCS#12)](asn1crypto/pkcs12.py), `asn1crypto.pkcs12`
153 - [Cryptographic message syntax (CMS, PKCS#7)](asn1crypto/cms.py), `asn1crypto.cms`
154 - [Time stamp protocol (TSP)](asn1crypto/tsp.py), `asn1crypto.tsp`
155 - [PDF signatures](asn1crypto/pdf.py), `asn1crypto.pdf`
156
157## Continuous Integration
158
159 - [Windows](https://ci.appveyor.com/project/wbond/asn1crypto/history) via AppVeyor
160 - [OS X](https://circleci.com/gh/wbond/asn1crypto) via CircleCI
161 - [Linux](https://travis-ci.org/wbond/asn1crypto/builds) via Travis CI
162 - [Test Coverage](https://codecov.io/gh/wbond/asn1crypto/commits) via Codecov
163
164## Testing
165
166Tests are written using `unittest` and require no third-party packages.
167
168Depending on what type of source is available for the package, the following
169commands can be used to run the test suite.
170
171### Git Repository
172
173When working within a Git working copy, or an archive of the Git repository,
174the full test suite is run via:
175
176```bash
177python run.py tests
178```
179
180To run only some tests, pass a regular expression as a parameter to `tests`.
181
182```bash
183python run.py tests ocsp
184```
185
186### PyPi Source Distribution
187
188When working within an extracted source distribution (aka `.tar.gz`) from
189PyPi, the full test suite is run via:
190
191```bash
192python setup.py test
193```
194
195### Package
196
197When the package has been installed via pip (or another method), the package
198`asn1crypto_tests` may be installed and invoked to run the full test suite:
199
200```bash
201pip install asn1crypto_tests
202python -m asn1crypto_tests
203```
204
205## Development
206
207To install the package used for linting, execute:
208
209```bash
210pip install --user -r requires/lint
211```
212
213The following command will run the linter:
214
215```bash
216python run.py lint
217```
218
219Support for code coverage can be installed via:
220
221```bash
222pip install --user -r requires/coverage
223```
224
225Coverage is measured by running:
226
227```bash
228python run.py coverage
229```
230
231To change the version number of the package, run:
232
233```bash
234python run.py version {pep440_version}
235```
236
237To install the necessary packages for releasing a new version on PyPI, run:
238
239```bash
240pip install --user -r requires/release
241```
242
243Releases are created by:
244
245 - Making a git tag in [PEP 440](https://www.python.org/dev/peps/pep-0440/#examples-of-compliant-version-schemes) format
246 - Running the command:
247
248 ```bash
249 python run.py release
250 ```
251
252Existing releases can be found at https://pypi.org/project/asn1crypto/.
253
254## CI Tasks
255
256A task named `deps` exists to download and stage all necessary testing
257dependencies. On posix platforms, `curl` is used for downloads and on Windows
258PowerShell with `Net.WebClient` is used. This configuration sidesteps issues
259related to getting pip to work properly and messing with `site-packages` for
260the version of Python being used.
261
262The `ci` task runs `lint` (if flake8 is available for the version of Python) and
263`coverage` (or `tests` if coverage is not available for the version of Python).
264If the current directory is a clean git working copy, the coverage data is
265submitted to codecov.io.
266
267```bash
268python run.py deps
269python run.py ci
270```
271
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}