1.. _module-pw_protobuf: 2 3----------- 4pw_protobuf 5----------- 6The protobuf module provides a lightweight interface for encoding and decoding 7the Protocol Buffer wire format. 8 9.. note:: 10 11 The protobuf module is a work in progress. Wire format encoding and decoding 12 is supported, though the APIs are not final. C++ code generation exists for 13 encoding, but not decoding. 14 15Design 16====== 17Unlike other protobuf libraries, which typically provide in-memory data 18structures to represent protobuf messages, ``pw_protobuf`` operates directly on 19the wire format and leaves data storage to the user. This has a few benefits. 20The primary one is that it allows the library to be incredibly small, with the 21encoder and decoder each having a code size of around 1.5K and negligible RAM 22usage. Users can choose the tradeoffs most suitable for their product on top of 23this core implementation. 24 25``pw_protobuf`` also provides zero-overhead C++ code generation which wraps its 26low-level wire format operations with a user-friendly API for processing 27specific protobuf messages. The code generation integrates with Pigweed's GN 28build system. 29 30Configuration 31============= 32``pw_protobuf`` supports the following configuration options. 33 34* ``PW_PROTOBUF_CFG_MAX_VARINT_SIZE``: 35 When encoding nested messages, the number of bytes to reserve for the varint 36 submessage length. Nested messages are limited in size to the maximum value 37 that can be varint-encoded into this reserved space. 38 39 The values that can be set, and their corresponding maximum submessage 40 lengths, are outlined below. 41 42 +-------------------+----------------------------------------+ 43 | MAX_VARINT_SIZE | Maximum submessage length | 44 +===================+========================================+ 45 | 1 byte | 127 | 46 +-------------------+----------------------------------------+ 47 | 2 bytes | 16,383 or < 16KiB | 48 +-------------------+----------------------------------------+ 49 | 3 bytes | 2,097,151 or < 2048KiB | 50 +-------------------+----------------------------------------+ 51 | 4 bytes (default) | 268,435,455 or < 256MiB | 52 +-------------------+----------------------------------------+ 53 | 5 bytes | 4,294,967,295 or < 4GiB (max uint32_t) | 54 +-------------------+----------------------------------------+ 55 56Usage 57===== 58``pw_protobuf`` splits wire format encoding and decoding operations. Links to 59the design and APIs of each are listed in below. 60 61See also :ref:`module-pw_protobuf_compiler` for details on ``pw_protobuf``'s 62build system integration. 63 64**pw_protobuf functionality** 65 66.. toctree:: 67 :maxdepth: 1 68 69 decoding 70 71Comparison with other protobuf libraries 72======================================== 73 74protobuf-lite 75^^^^^^^^^^^^^ 76protobuf-lite is the official reduced-size C++ implementation of protobuf. It 77uses a restricted subset of the protobuf library's features to minimize code 78size. However, is is still around 150K in size and requires dynamic memory 79allocation, making it unsuitable for many embedded systems. 80 81nanopb 82^^^^^^ 83`nanopb <https://github.com/nanopb/nanopb>`_ is a commonly used embedded 84protobuf library with very small code size and full code generation. It provides 85both encoding/decoding functionality and in-memory C structs representing 86protobuf messages. 87 88nanopb works well for many embedded products; however, using its generated code 89can run into RAM usage issues when processing nontrivial protobuf messages due 90to the necessity of defining a struct capable of storing all configurations of 91the message, which can grow incredibly large. In one project, Pigweed developers 92encountered an 11K struct statically allocated for a single message---over twice 93the size of the final encoded output! (This was what prompted the development of 94``pw_protobuf``.) 95 96To avoid this issue, it is possible to use nanopb's low-level encode/decode 97functions to process individual message fields directly, but this loses all of 98the useful semantics of code generation. ``pw_protobuf`` is designed to optimize 99for this use case; it allows for efficient operations on the wire format with an 100intuitive user interface. 101 102Depending on the requirements of a project, either of these libraries could be 103suitable. 104