1[/ 2 Copyright (c) 2016-2019 Vinnie Falco (vinnie dot falco at gmail dot com) 3 4 Distributed under the Boost Software License, Version 1.0. (See accompanying 5 file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) 6 7 Official repository: https://github.com/boostorg/beast 8] 9 10[section Chunked Encoding] 11 12For message payloads whose size is not known ahead of time, HTTP 13version 1.1 defines the 14[@https://tools.ietf.org/html/rfc7230#section-4.1 ['chunked]] 15transfer coding. This coding consists of zero or more 16[@https://tools.ietf.org/html/rfc7230#section-4.1 ['chunked bodies]], 17followed by a 18[@https://tools.ietf.org/html/rfc7230#section-4.1 ['last chunk]]. 19Each chunked body may contain optional application-defined, connection-specific 20[@https://tools.ietf.org/html/rfc7230#section-4.1.1 ['chunk-extensions]]. 21The last chunk may contain additional HTTP field values in a section 22of the last chunk called a 23[@https://tools.ietf.org/html/rfc7230#section-4.1.2 ['chunk-trailer]]. 24The field values are "promised" in the header as a comma delimited list 25of field names in the 26[@https://tools.ietf.org/html/rfc7230#section-4.4 [*Trailer]] 27field value. Clients indicate their willingness to accept trailers 28by including the "trailers" token in the 29[@https://tools.ietf.org/html/rfc7230#section-4.3 [*TE]] 30field value. 31 32[heading Serializing Chunks] 33 34The __serializer__ automatically applies the chunked transfer encoding 35when a message returns `true` from 36[link beast.ref.boost__beast__http__message.chunked.overload1 `message::chunked`]. 37The boundaries between chunks emitted by the serializer are implementation 38defined. Chunk extensions and trailers are omitted. Applications which 39need precise control over the chunk boundaries, extensions, and trailers 40may use a set of helper classes which enable manual emission of message 41payloads using chunk encoding. 42 43To use these helper classes, first serialize the header portion of the 44message using the standard interface. Then prepare the buffers, chunk 45extensions, and desired trailers, and use them with these helpers: 46 47[table Chunking Helpers 48[[Name][Description]] 49[ 50 [[link beast.ref.boost__beast__http__chunk_body `chunk_body`]] 51 [ 52 A buffer sequence representing a complete chunk body. 53 ] 54][ 55 [[link beast.ref.boost__beast__http__chunk_crlf `chunk_crlf`]] 56 [ 57 A buffer sequence representing the CRLF (`"\r\n"`) delimiter. 58 This class is used when the caller desires to emit the 59 chunk body in two or more individual stream operations. 60 ] 61][ 62 [ 63 [link beast.ref.boost__beast__http__chunk_extensions `chunk_extensions`] 64 65 66 [link beast.ref.boost__beast__http__basic_chunk_extensions `basic_chunk_extensions`] 67 ] 68 [ 69 This is a simple, allocating container which lets callers 70 easily build up a set of chunk extensions. 71 ] 72][ 73 [[link beast.ref.boost__beast__http__chunk_header `chunk_header`]] 74 [ 75 A buffer sequence representing a hex-encoded chunk size, 76 followed by an optional set of chunk extensions, including 77 the terminating CRLF (`"\r\n"`) delimiter which precedes 78 the chunk body. This class is used when the caller desires 79 to emit the chunk body in two or more individual stream 80 operations. 81 ] 82][ 83 [[link beast.ref.boost__beast__http__chunk_last `chunk_last`]] 84 [ 85 A buffer sequence representing a last chunk. The last chunk 86 indicates the end of the chunked message payload, and may 87 contain optional trailer fields. 88 ] 89][ 90 [ 91 [link beast.ref.boost__beast__http__make_chunk `make_chunk`] 92 93 [link beast.ref.boost__beast__http__make_chunk_last `make_chunk_last`] 94 ] 95 [ 96 These helper functions are used to construct a chunk 97 or last chunk directly at call sites. 98 ] 99]] 100 101We demonstrate the use of these objects first by declaring a function 102which returns the next buffer sequence to use as a chunk body: 103 104[http_snippet_17] 105 106This example demonstrates sending a complete chunked message payload 107manually. No chunk extensions or trailers are emitted: 108 109[http_snippet_18] 110 111The following code sends additional chunks, and sets chunk extensions 112using the helper container. The container automatically quotes values 113in the serialized output when necessary: 114 115[http_snippet_19] 116 117Callers can take over the generation and management of the extensions 118buffer by passing a non-owning string. Note that this requires the 119string contents to adhere to the correct syntax for chunk extensions, 120including the needed double quotes for values which contain spaces: 121 122[http_snippet_20] 123 124The next code sample emits a chunked response which promises two 125trailer fields and delivers them in the last chunk. The implementation 126allocates memory using the default or a passed-in allocator to hold 127the state information required to serialize the trailer: 128 129[http_snippet_21] 130 131Using a custom allocator to serialize the last chunk: 132 133[http_snippet_22] 134 135Alternatively, callers can take over the generation and lifetime 136management of the serialized trailer fields by passing in a non-owning 137string: 138 139[http_snippet_23] 140 141For the ultimate level of control, a caller can manually compose the 142chunk itself by first emitting a header with the correct chunk body 143size, and then by emitting the chunk body in multiple calls to the 144stream write function. In this case the caller is responsible for 145also emitting the terminating CRLF (`"\r\n"`): 146 147[http_snippet_24] 148 149[heading Parsing Chunks] 150 151The __parser__ automatically removes the chunked transfer coding when 152it is the last encoding in the list. However, it also discards the 153chunk extensions and does not provide a way to determine the boundaries 154between chunks. Advanced applications which need to access the chunk 155extensions or read complete individual chunks may use a callback 156interface provided by __parser__: 157 158[table Chunking Parse Callbacks 159[[Name][Description]] 160[ 161 [[link beast.ref.boost__beast__http__parser.on_chunk_header `on_chunk_header`]] 162 [ 163 Set a callback to be invoked on each chunk header. 164 165 The callback will be invoked once for every chunk in the message 166 payload, as well as once for the last chunk. The invocation 167 happens after the chunk header is available but before any body 168 octets have been parsed. 169 170 The extensions are provided in raw, validated form, use 171 [link beast.ref.boost__beast__http__basic_chunk_extensions.parse `chunk_extensions::parse`] 172 to parse the extensions into a structured container for easier access. 173 The implementation type-erases the callback without requiring 174 a dynamic allocation. For this reason, the callback object is 175 passed by a non-constant reference. 176 177 The function object will be called with this equivalent signature: 178 ``` 179 void 180 callback( 181 std::uint64_t size, // Size of the chunk, zero for the last chunk 182 string_view extensions, // The chunk-extensions in raw form 183 error_code& ec); // May be set by the callback to indicate an error 184 ``` 185 ] 186][ 187 [[link beast.ref.boost__beast__http__parser.on_chunk_body `on_chunk_body`]] 188 [ 189 Set a callback to be invoked on chunk body data. 190 191 The callback will be invoked one or more times to provide 192 buffers corresponding to the chunk body for the current chunk. 193 The callback receives the number of octets remaining in this 194 chunk body including the octets in the buffer provided. 195 196 The callback must return the number of octets actually consumed. 197 Any octets not consumed will be presented again in a subsequent 198 invocation of the callback. 199 The implementation type-erases the callback without requiring 200 a dynamic allocation. For this reason, the callback object is 201 passed by a non-constant reference. 202 203 The function object will be called with this equivalent signature: 204 ``` 205 std::size_t 206 callback( 207 std::uint64_t remain, // Octets remaining in this chunk, includes `body` 208 string_view body, // A buffer holding some or all of the remainder of the chunk body 209 error_code& ec); // May be set by the callback to indicate an error 210 ``` 211 ] 212]] 213 214This example will read a message header from the stream, and then manually 215read each chunk. It recognizes the chunk boundaries and outputs the contents 216of each chunk as it comes in. Any chunk extensions are printed, each extension 217on its own line. Finally, any trailers promised in the header are printed. 218 219[example_chunk_parsing] 220 221Given the HTTP response as input on the left, the output of the function shown 222above is shown on the right: 223 224[table Chunk Parsing Example Output 225[[Input][Output]] 226[ 227 [ 228 ``` 229 HTTP/1.1 200 OK\r\n 230 Server: test\r\n 231 Trailer: Expires, Content-MD5\r\n 232 Transfer-Encoding: chunked\r\n 233 \r\n 234 5\r\n 235 First\r\n 236 d;quality=1.0\r\n 237 Hello, world!\r\n 238 e;file=abc.txt;quality=0.7\r\n 239 The Next Chunk\r\n 240 8;last\r\n 241 Last one\r\n 242 0\r\n 243 Expires: never\r\n 244 Content-MD5: f4a5c16584f03d90\r\n 245 \r\n 246 ``` 247 ] 248 [ 249 ``` 250 Chunk Body: First 251 Extension: quality = 1.0 252 Chunk Body: Hello, world! 253 Extension: file = abc.txt 254 Extension: quality = 0.7 255 Chunk Body: The Next Chunk 256 Extension: last 257 Chunk Body: Last one 258 Expires: never 259 Content-MD5: f4a5c16584f03d90 260 ``` 261 ] 262]] 263 264[endsect] 265