1.. _module-pw_hdlc: 2 3======= 4pw_hdlc 5======= 6`High-Level Data Link Control (HDLC) 7<https://en.wikipedia.org/wiki/High-Level_Data_Link_Control>`_ is a data link 8layer protocol intended for serial communication between devices. HDLC is 9standardized as `ISO/IEC 13239:2002 <https://www.iso.org/standard/37010.html>`_. 10 11The ``pw_hdlc`` module provides a simple, robust frame-oriented transport that 12uses a subset of the HDLC protocol. ``pw_hdlc`` supports sending between 13embedded devices or the host. It can be used with :ref:`module-pw_rpc` to enable 14remote procedure calls (RPCs) on embedded on devices. 15 16**Why use the pw_hdlc module?** 17 18 * Enables the transmission of RPCs and other data between devices over serial. 19 * Detects corruption and data loss. 20 * Light-weight, simple, and easy to use. 21 * Supports streaming to transport without buffering, since the length is not 22 encoded. 23 24.. admonition:: Try it out! 25 26 For an example of how to use HDLC with :ref:`module-pw_rpc`, see the 27 :ref:`module-pw_hdlc-rpc-example`. 28 29.. toctree:: 30 :maxdepth: 1 31 :hidden: 32 33 rpc_example/docs 34 35-------------------- 36Protocol Description 37-------------------- 38 39Frames 40====== 41The HDLC implementation in ``pw_hdlc`` supports only HDLC unnumbered 42information frames. These frames are encoded as follows: 43 44.. code-block:: text 45 46 _________________________________________ 47 | | | | | | |... 48 | | | | | | |... [More frames] 49 |_|_|_|__________________________|____|_|... 50 F A C Payload FCS F 51 52 F = flag byte (0x7e, the ~ character) 53 A = address field 54 C = control field 55 FCS = frame check sequence (CRC-32) 56 57 58Encoding and sending data 59========================= 60This module first writes an initial frame delimiter byte (0x7E) to indicate the 61beginning of the frame. Before sending any of the payload data through serial, 62the special bytes are escaped: 63 64 +-------------------------+-----------------------+ 65 | Unescaped Special Bytes | Escaped Special Bytes | 66 +=========================+=======================+ 67 | 7E | 7D 5E | 68 +-------------------------+-----------------------+ 69 | 7D | 7D 5D | 70 +-------------------------+-----------------------+ 71 72The bytes of the payload are escaped and written in a single pass. The 73frame check sequence is calculated, escaped, and written after. After this, a 74final frame delimiter byte (0x7E) is written to mark the end of the frame. 75 76Decoding received bytes 77======================= 78Frames may be received in multiple parts, so we need to store the received data 79in a buffer until the ending frame delimiter (0x7E) is read. When the 80``pw_hdlc`` decoder receives data, it unescapes it and adds it to a buffer. 81When the frame is complete, it calculates and verifies the frame check sequence 82and does the following: 83 84* If correctly verified, the decoder returns the decoded frame. 85* If the checksum verification fails, the frame is discarded and an error is 86 reported. 87 88--------- 89API Usage 90--------- 91There are two primary functions of the ``pw_hdlc`` module: 92 93 * **Encoding** data by constructing a frame with the escaped payload bytes and 94 frame check sequence. 95 * **Decoding** data by unescaping the received bytes, verifying the frame 96 check sequence, and returning successfully decoded frames. 97 98Encoder 99======= 100The Encoder API provides a single function that encodes data as an HDLC 101unnumbered information frame. 102 103C++ 104--- 105.. cpp:namespace:: pw 106 107.. cpp:function:: Status hdlc::WriteUIFrame(uint64_t address, ConstByteSpan data, stream::Writer& writer) 108 109 Writes a span of data to a :ref:`pw::stream::Writer <module-pw_stream>` and 110 returns the status. This implementation uses the :ref:`module-pw_checksum` 111 module to compute the CRC-32 frame check sequence. 112 113.. code-block:: cpp 114 115 #include "pw_hdlc/encoder.h" 116 #include "pw_hdlc/sys_io_stream.h" 117 118 int main() { 119 pw::stream::SysIoWriter serial_writer; 120 Status status = WriteUIFrame(123 /* address */, 121 data, 122 serial_writer); 123 if (!status.ok()) { 124 PW_LOG_INFO("Writing frame failed! %s", status.str()); 125 } 126 } 127 128Python 129------ 130.. automodule:: pw_hdlc.encode 131 :members: 132 133.. code-block:: python 134 135 import serial 136 from pw_hdlc import encode 137 138 ser = serial.Serial() 139 address = 123 140 ser.write(encode.ui_frame(address, b'your data here!')) 141 142Typescript 143---------- 144 145Encoder 146======= 147The Encoder class provides a way to build complete, escaped HDLC UI frames. 148 149.. js:method:: Encoder.uiFrame(address, data) 150 151 :param number address: frame address. 152 :param Uint8Array data: frame data. 153 :returns: Uint8Array containing a complete HDLC frame. 154 155Decoder 156======= 157The decoder class unescapes received bytes and adds them to a buffer. Complete, 158valid HDLC frames are yielded as they are received. 159 160.. js:method:: Decoder.process(bytes) 161 162 :param Uint8Array bytes: bytes received from the medium. 163 :yields: Frame complete frames. 164 165C++ 166--- 167.. cpp:class:: pw::hdlc::Decoder 168 169 .. cpp:function:: pw::Result<Frame> Process(std::byte b) 170 171 Parses a single byte of an HDLC stream. Returns a Result with the complete 172 frame if the byte completes a frame. The status is the following: 173 174 - OK - A frame was successfully decoded. The Result contains the Frame, 175 which is invalidated by the next Process call. 176 - UNAVAILABLE - No frame is available. 177 - RESOURCE_EXHAUSTED - A frame completed, but it was too large to fit in 178 the decoder's buffer. 179 - DATA_LOSS - A frame completed, but it was invalid. The frame was 180 incomplete or the frame check sequence verification failed. 181 182 .. cpp:function:: void Process(pw::ConstByteSpan data, F&& callback, Args&&... args) 183 184 Processes a span of data and calls the provided callback with each frame or 185 error. 186 187This example demonstrates reading individual bytes from ``pw::sys_io`` and 188decoding HDLC frames: 189 190.. code-block:: cpp 191 192 #include "pw_hdlc/decoder.h" 193 #include "pw_sys_io/sys_io.h" 194 195 int main() { 196 std::byte data; 197 while (true) { 198 if (!pw::sys_io::ReadByte(&data).ok()) { 199 // Log serial reading error 200 } 201 Result<Frame> decoded_frame = decoder.Process(data); 202 203 if (decoded_frame.ok()) { 204 // Handle the decoded frame 205 } 206 } 207 } 208 209Python 210------ 211.. autoclass:: pw_hdlc.decode.FrameDecoder 212 :members: 213 214Below is an example using the decoder class to decode data read from serial: 215 216.. code-block:: python 217 218 import serial 219 from pw_hdlc import decode 220 221 ser = serial.Serial() 222 decoder = decode.FrameDecoder() 223 224 while True: 225 for frame in decoder.process_valid_frames(ser.read()): 226 # Handle the decoded frame 227 228Typescript 229---------- 230Decodes one or more HDLC frames from a stream of data. 231 232.. js:method:: process(data) 233 234 :param Uint8Array data: bytes to be decoded. 235 :yields: HDLC frames, including corrupt frames. 236 The Frame.ok() method whether the frame is valid. 237 238.. js:method:: processValidFrames(data) 239 240 :param Uint8Array data: bytes to be decoded. 241 :yields: Valid HDLC frames, logging any errors. 242 243Allocating buffers 244------------------ 245Since HDLC's encoding overhead changes with payload size and what data is being 246encoded, this module provides helper functions that are useful for determining 247the size of buffers by providing worst-case sizes of frames given a certain 248payload size and vice-versa. 249 250.. code-block:: cpp 251 252 #include "pw_assert/check.h" 253 #include "pw_bytes/span.h" 254 #include "pw_hdlc/encoder" 255 #include "pw_hdlc/encoded_size.h" 256 #include "pw_status/status.h" 257 258 // The max on-the-wire size in bytes of a single HDLC frame after encoding. 259 constexpr size_t kMtu = 512; 260 constexpr size_t kRpcEncodeBufferSize = pw::hdlc::MaxSafePayloadSize(kMtu); 261 std::array<std::byte, kRpcEncodeBufferSize> rpc_encode_buffer; 262 263 // Any data encoded to this buffer is guaranteed to fit in the MTU after 264 // HDLC encoding. 265 pw::ConstByteSpan GetRpcEncodeBuffer() { 266 return rpc_encode_buffer; 267 } 268 269The HDLC ``Decoder`` has its own helper for allocating a buffer since it doesn't 270need the entire escaped frame in-memory to decode, and therefore has slightly 271lower overhead. 272 273.. code-block:: cpp 274 275 #include "pw_hdlc/decoder.h" 276 277 // The max on-the-wire size in bytes of a single HDLC frame after encoding. 278 constexpr size_t kMtu = 512; 279 280 // Create a decoder given the MTU constraint. 281 constexpr size_t kDecoderBufferSize = 282 pw::hdlc::Decoder::RequiredBufferSizeForFrameSize(kMtu); 283 pw::hdlc::DecoderBuffer<kDecoderBufferSize> decoder; 284 285------------------- 286Additional features 287------------------- 288 289Interleaving unstructured data with HDLC 290======================================== 291It is possible to decode HDLC frames from a stream using different protocols or 292unstructured data. This is not recommended, but may be necessary when 293introducing HDLC to an existing system. 294 295The ``FrameAndNonFrameDecoder`` Python class supports working with raw data and 296HDLC frames in the same stream. 297 298.. autoclass:: pw_hdlc.decode.FrameAndNonFrameDecoder 299 :members: 300 301RpcChannelOutput 302================ 303The ``RpcChannelOutput`` implements pw_rpc's ``pw::rpc::ChannelOutput`` 304interface, simplifying the process of creating an RPC channel over HDLC. A 305``pw::stream::Writer`` must be provided as the underlying transport 306implementation. 307 308If your HDLC routing path has a Maximum Transmission Unit (MTU) limitation, 309using the ``FixedMtuChannelOutput`` is strongly recommended to verify that the 310currently configured max RPC payload size (dictated by pw_rpc's static encode 311buffer) will always fit safely within the limits of the fixed HDLC MTU *after* 312HDLC encoding. 313 314HdlcRpcClient 315============= 316.. autoclass:: pw_hdlc.rpc.HdlcRpcClient 317 :members: 318 319.. autoclass:: pw_hdlc.rpc.HdlcRpcLocalServerAndClient 320 :members: 321 322Example pw::rpc::system_server backend 323====================================== 324This module includes an example implementation of ``pw_rpc``'s ``system_server`` 325facade. This implementation sends HDLC encoded RPC packets via ``pw_sys_io``, 326and has blocking sends/reads, so it is hardly performance-oriented and 327unsuitable for performance-sensitive applications. This mostly servers as a 328simplistic example for quickly bringing up RPC over HDLC on bare-metal targets. 329 330----------- 331Size report 332----------- 333The HDLC module currently optimizes for robustness and flexibility instead of 334binary size or performance. 335 336There are two size reports: the first shows the cost of everything needed to 337use HDLC, including the dependencies on common modules like CRC32 from 338pw_checksum and variable-length integer handling from pw_varint. The other is 339the cost if your application is already linking those functions. pw_varint is 340commonly used since it's necessary for protocol buffer handling, so is often 341already present. 342 343.. include:: size_report 344 345------- 346Roadmap 347------- 348- **Expanded protocol support** - ``pw_hdlc`` currently only supports 349 unnumbered information frames. Support for different frame types and 350 extended control fields may be added in the future. 351 352------------- 353Compatibility 354------------- 355C++17 356 357------ 358Zephyr 359------ 360To enable ``pw_hdlc.pw_rpc`` for Zephyr add ``CONFIG_PIGWEED_HDLC_RPC=y`` to 361the project's configuration. 362 363