1.. _module-pw_stream: 2 3--------- 4pw_stream 5--------- 6 7``pw_stream`` provides a foundational interface for streaming data from one part 8of a system to another. In the simplest use cases, this is basically a memcpy 9behind a reusable interface that can be passed around the system. On the other 10hand, the flexibility of this interface means a ``pw_stream`` could terminate is 11something more complex, like a UART stream or flash memory. 12 13Overview 14======== 15At the most basic level, ``pw_stream``'s interfaces provide very simple handles 16to enabling streaming data from one location in a system to an endpoint. 17 18Example: 19 20.. code-block:: cpp 21 22 void DumpSensorData(pw::stream::Writer& writer) { 23 static char temp[64]; 24 ImuSample imu_sample; 25 imu.GetSample(&info); 26 size_t bytes_written = imu_sample.AsCsv(temp, sizeof(temp)); 27 writer.Write(temp, bytes_written); 28 } 29 30In this example, ``DumpSensorData()`` only cares that it has access to a 31``Writer`` that it can use to stream data to using ``Writer::Write()``. The 32``Writer`` itself can be backed by anything that can act as a data "sink." 33 34 35pw::stream::Writer 36------------------ 37This is the foundational stream ``Writer`` abstract class. Any class that wishes 38to implement the ``Writer`` interface **must** provide a ``DoWrite()`` 39implementation. Note that ``Write()`` itself is **not** virtual, and should not 40be overridden. 41 42Buffering 43^^^^^^^^^ 44If any buffering occurs in a ``Writer`` and data must be flushed before it is 45fully committed to the sink, a ``Writer`` implementation is resposible for any 46``Flush()`` capability. 47 48pw::stream::Reader 49------------------ 50This is the foundational stream ``Reader`` abstract class. Any class that wishes 51to implement the ``Reader`` interface **must** provide a ``DoRead()`` 52implementation. Note that ``Read()`` itself is **not** virtual, and should not 53be overridden. 54 55pw::stream::MemoryWriter 56------------------------ 57The ``MemoryWriter`` class implements the ``Writer`` interface by backing the 58data destination with an **externally-provided** memory buffer. 59``MemoryWriterBuffer`` extends ``MemoryWriter`` to internally provide a memory 60buffer. 61 62pw::stream::MemoryReader 63------------------------ 64The ``MemoryReader`` class implements the ``Reader`` interface by backing the 65data source with an **externally-provided** memory buffer. 66 67pw::stream::NullWriter 68------------------------ 69The ``NullWriter`` class implements the ``Writer`` interface by dropping all 70requested data writes, similar to ``/dev/null``. 71 72Why use pw_stream? 73================== 74 75Standard API 76------------ 77``pw_stream`` provides a standard way for classes to express that they have the 78ability to write data. Writing to one sink versus another sink is a matter of 79just passing a reference to the appropriate ``Writer``. 80 81As an example, imagine dumping sensor data. If written against a random HAL 82or one-off class, there's porting work required to write to a different sink 83(imagine writing over UART vs dumping to flash memory). Building a "dumping" 84implementation against the ``Writer`` interface prevents a dependency from 85forming on an artisainal API that would require porting work. 86 87Similarly, after building a ``Writer`` implementation for a Sink that data 88could be dumped to, that same ``Writer`` can be reused for other contexts that 89already write data to the ``pw::stream::Writer`` interface. 90 91Before: 92 93.. code-block:: cpp 94 95 // Not reusable, depends on `Uart`. 96 void DumpSensorData(Uart& uart) { 97 static char temp[64]; 98 ImuSample imu_sample; 99 imu.GetSample(&info); 100 size_t bytes_written = imu_sample.AsCsv(temp, sizeof(temp)); 101 uart.Transmit(temp, bytes_written, /*timeout_ms=*/ 200); 102 } 103 104After: 105 106.. code-block:: cpp 107 108 // Reusable; no more Uart dependency! 109 void DumpSensorData(Writer& writer) { 110 static char temp[64]; 111 ImuSample imu_sample; 112 imu.GetSample(&info); 113 size_t bytes_written = imu_sample.AsCsv(temp, sizeof(temp)); 114 writer.Write(temp, bytes_written); 115 } 116 117Reduce intermediate buffers 118--------------------------- 119Often functions that write larger blobs of data request a buffer is passed as 120the destination that data should be written to. This *requires* a buffer is 121allocated, even if the data only exists in that buffer for a very short period 122of time before it's written somewhere else. 123 124In situations where data read from somewhere will immediately be written 125somewhere else, a ``Writer`` interface can cut out the middleman buffer. 126 127Before: 128 129.. code-block:: cpp 130 131 // Requires an intermediate buffer to write the data as CSV. 132 void DumpSensorData(Uart* uart) { 133 char temp[64]; 134 ImuSample imu_sample; 135 imu.GetSample(&info); 136 size_t bytes_written = imu_sample.AsCsv(temp, sizeof(temp)); 137 uart.Transmit(temp, bytes_written, /*timeout_ms=*/ 200); 138 } 139 140After: 141 142.. code-block:: cpp 143 144 // Both DumpSensorData() and RawSample::AsCsv() use a Writer, eliminating the 145 // need for an intermediate buffer. 146 void DumpSensorData(Writer* writer) { 147 RawSample imu_sample; 148 imu.GetSample(&info); 149 imu_sample.AsCsv(writer); 150 } 151 152Prevent buffer overflow 153----------------------- 154When copying data from one buffer to another, there must be checks to ensure the 155copy does not overflow the destination buffer. As this sort of logic is 156duplicated throughout a codebase, there's more opportunities for bound-checking 157bugs to sneak in. ``Writers`` manage this logic internally rather than pushing 158the bounds checking to the code that is moving or writing the data. 159 160Similarly, since only the ``Writer`` has access to any underlying buffers, it's 161harder for functions that share a ``Writer`` to accidentally clobber data 162written by others using the same buffer. 163 164Before: 165 166.. code-block:: cpp 167 168 Status BuildPacket(Id dest, span<const std::byte> payload, 169 span<std::byte> dest) { 170 Header header; 171 if (dest.size_bytes() + payload.size_bytes() < sizeof(Header)) { 172 return Status::RESOURCE_EXHAUSTED; 173 } 174 header.dest = dest; 175 header.src = DeviceId(); 176 header.payload_size = payload.size_bytes(); 177 178 memcpy(dest.data(), &header, sizeof(header)); 179 // Forgetting this line would clobber buffer contents. Also, using 180 // a temporary span instead could leave `dest` to be misused elsewhere in 181 // the function. 182 dest = dest.subspan(sizeof(header)); 183 memcpy(dest.data(), payload.data(), payload.size_bytes()); 184 } 185 186After: 187 188.. code-block:: cpp 189 190 Status BuildPacket(Id dest, span<const std::byte> payload, Writer& writer) { 191 Header header; 192 header.dest = dest; 193 header.src = DeviceId(); 194 header.payload_size = payload.size_bytes(); 195 196 writer.Write(header); 197 return writer.Write(payload); 198 } 199 200Why NOT pw_stream? 201================== 202pw_stream provides a virtual interface. This inherently has more overhead than 203a regular function call. In extremely performance-sensitive contexts, a virtual 204interface might not provide enough utility to justify the performance cost. 205 206Dependencies 207============ 208 * ``pw_assert`` module 209 * ``pw_preprocessor`` module 210 * ``pw_status`` module 211 * ``pw_span`` module 212