1<?xml version="1.0" encoding="utf-8"?> 2<?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?> 3 4<!DOCTYPE rfc SYSTEM "rfc2629.dtd"> 5<?rfc toc="yes"?> 6<rfc ipr="full2026" docname="draft-libpcap-dump-format-00.txt"> 7 <front> 8 <title>PCAP New Generation Dump File Format</title> 9 <author initials="L." surname="Degioanni" fullname="Loris Degioanni"> 10 <organization>Politecnico di Torino</organization> 11 <address> 12 <postal> 13 <street>Corso Duca degli Abruzzi, 24</street> 14 <city>Torino</city> 15 <code>10129</code> 16 <country>Italy</country> 17 </postal> 18 <phone>+39 011 564 7008</phone> 19 <email>loris.degioanni@polito.it</email> 20 <uri>http://netgroup.polito.it/loris/</uri> 21 </address> 22 </author> 23 <author initials="F." surname="Risso" fullname="Fulvio Risso"> 24 <organization>Politecnico di Torino</organization> 25 <address> 26 <postal> 27 <street>Corso Duca degli Abruzzi, 24</street> 28 <city>Torino</city> 29 <code>10129</code> 30 <country>Italy</country> 31 </postal> 32 <phone>+39 011 564 7008</phone> 33 <email>fulvio.risso@polito.it</email> 34 <uri>http://netgroup.polito.it/fulvio.risso/</uri> 35 </address> 36 </author> 37 38 <!-- Other authors go here --> 39 40 <date month="March" year="2004"/> 41 <area>General</area> 42<!-- 43 <workgroup> 44--> 45 <keyword>Internet-Draft</keyword> 46 <keyword>Libpcap, dump file format</keyword> 47 <abstract> 48<t>This document describes a format to dump captured packets on a file. This format is extensible and it is currently proposed for implementation in the libpcap/WinPcap packet capture library.</t> 49 </abstract> 50<!-- 51 <note ...> 52--> 53 </front> 54 <middle> 55 56<section title="Objectives"> 57<t>The problem of exchanging packet traces becomes more and more critical every day; unfortunately, no standard solutions exist for this task right now. One of the most accepted packet interchange formats is the one defined by libpcap, which is rather old and does not fit for some of the nowadays applications especially in terms of extensibility.</t> 58<t>This document proposes a new format for dumping packet traces. The following goals are being pursued:</t> 59<list style="symbols"> 60<t>Extensibility: aside of some common functionalities, third parties should be able to enrich the information embedded in the file with proprietary extensions, which will be ignored by tools that are not able to understand them.</t> 61<t>Portability: a capture trace must contain all the information needed to read data independently from network, hardware and operating system of the machine that made the capture.</t> 62<t>Merge/Append data: it should be possible to add data at the end of a given file, and the resulting file must still be readable.</t> 63</list> 64 65</section> 66 67 68<section title="General File Structure"> 69 70<section anchor="sectionblock" title="General Block Structure"> 71<t>A capture file is organized in blocks, that are appended one to another to form the file. All the blocks share a common format, which is shown in <xref target="formatblock"/>.</t> 72 73<figure anchor="formatblock" title="Basic block structure."> 74<artwork> 75 0 1 2 3 76 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 77 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 78 | Block Type | 79 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 80 | Block Total Length | 81 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 82 / Block Body / 83 / /* variable length, aligned to 32 bits */ / 84 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 85 | Block Total Length | 86 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 87</artwork> 88</figure> 89 90<t>The fields have the following meaning:</t> 91 92<list style="symbols"> 93<t>Block Type (32 bits): unique value that identifies the block. Values whose Most Significant Bit (MSB) is equal to 1 are reserved for local use. They allow to save private data to the file and to extend the file format.</t> 94<t>Block Total Length: total size of this block, in bytes. For instance, a block that does not have a body has a length of 12 bytes.</t> 95<t>Block Body: content of the block.</t> 96<t>Block Total Length: total size of this block, in bytes. This field is duplicated for permitting backward file navigation.</t> 97</list> 98 99<t>This structure, shared among all blocks, makes easy to process a file and to skip unneeded or unknown blocks. Blocks can be nested one inside the others (NOTE: needed?). Some of the blocks are mandatory, i.e. a dump file is not valid if they are not present, other are optional.</t> 100<t>The structure of the blocks allows to define other blocks if needed. A parser that does non understand them can simply ignore their content.</t> 101</section> 102 103<section title="Block Types"> 104<t>The currently defined blocks are the following:</t> 105<list style="numbers"> 106<t>Section Header Block: it defines the most important characteristics of the capture file.</t> 107<t>Interface Description Block: it defines the most important characteristics of the interface(s) used for capturing traffic.</t> 108<t>Packet Block: it contains a single captured packet, or a portion of it.</t> 109<t>Simple Packet Block: it contains a single captured packet, or a portion of it, with only a minimal set of information about it.</t> 110<t>Name Resolution Block: it defines the mapping from numeric addresses present in the packet dump and the canonical name counterpart.</t> 111<t>Capture Statistics Block: it defines how to store some statistical data (e.g. packet dropped, etc) which can be useful to undestand the conditions in which the capture has been made.</t> 112<t>Compression Marker Block: TODO</t> 113<t>Encryption Marker Block: TODO</t> 114<t>Fixed Length Marker Block: TODO</t> 115</list> 116 117<t>The following blocks instead are considered interesting but the authors believe that they deserve more in-depth discussion before being defined:</t> 118<list style="numbers"> 119<t>Further Packet Blocks</t> 120<t>Directory Block</t> 121<t>Traffic Statistics and Monitoring Blocks</t> 122<t>Alert and Security Blocks</t> 123</list> 124 125<t>TODO Currently standardized Block Type codes are specified in Appendix 1.</t> 126 127</section> 128 129<section title="Block Hierarchy and Precedence"> 130<t>The file must begin with a Section Header Block. However, more than one Section Header Block can be present on the dump, each one covering the data following it till the next one (or the end of file). A Section includes the data delimited by two Section Header Blocks (or by a Section Header Block and the end of the file), including the first Section Header Block.</t> 131<t>In case an application cannot read a Section because of different version number, it must skip everything until the next Section Header Block. Note that, in order to properly skip the blocks until the next section, all blocks must have the fields Type and Length at the beginning. This is a mandatory requirement that must be maintained in future versions of the block format.</t> 132<t><xref target="fssample-SHB"/> shows two valid files: the first has a typical configuration, with a single Section Header that covers the whole file. The second one contains three headers, and is normally the result of file concatenation. An application that understands only version 1.0 of the file format skips the intermediate section and restart processing the packets after the third Section Header.</t> 133 134<figure anchor="fssample-SHB" title="File structure example: the Section Header Block."> 135<artwork> 136 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 137 | SHB v1.0 | Data | 138 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 139 Typical configuration with a single Section Header Block 140 141 142 |-- 1st Section --|-- 2nd Section --|-- 3rd Section --| 143 | | 144 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 145 | SHB v1.0 | Data | SHB V1.1 | Data | SHB V1.0 | Data | 146 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 147 Configuration with three different Section Header Blocks 148</artwork> 149</figure> 150 151<t>NOTE: TO BE COMPLETED with some examples of other blocks</t> 152 153</section> 154 155<section title="Data format"> 156<t>Data contained in each section will always be saved according to the characteristics (little endian / big endian) of the dumping machine. This refers to all fields that are saved as numbers and that span over two or more bytes.</t> 157<t>The approach of having each section saved in the native format of the generating host is more efficient because it avoids translation of data when reading / writing on the host itself, which is the most common case when generating/processing capture dumps.</t> 158<t>TODO Probably we have to specify something more here. Is what we're saying enough to avoid any kind of ambiguity?.</t> 159</section> 160 161</section> 162 163 164 165 166<section title="Block Definition"> 167<t>This section details the format of the body of the blocks currently defined.</t> 168 169<section anchor="sectionshb" title="Section Header Block (mandatory)"> 170<t>The Section Header Block is mandatory. It identifies the beginning of a section of the capture dump file. Its format is shown in <xref target="formatSHB"/>.</t> 171<figure anchor="formatSHB" title="Section Header Block format."> 172<artwork> 173 0 1 2 3 174 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 175 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 176 | Magic | 177 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 178 | Major | Minor | 179 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 180 / / 181 / Options (variable) / 182 / / 183 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 184</artwork> 185</figure> 186 187<t>The meaning of the fields is:</t> 188<list style="symbols"> 189<t>Magic: magic number, whose value is the hexadecimal number 0x1A2B3C4D. This number can be used to distinguish section that have been saved on little-endian machines from the one saved on big-endian machines.</t> 190<t>Major: number of the current mayor version of the format. Current value is 1.</t> 191<t>Minor: number of the current minor version of the format. Current value is 0.</t> 192<t>Options: optionally, a list of options (formatted according to the rules defined in <xref target="sectionopt"/>) can be present.</t> 193</list> 194 195<t>Aside form the options defined in <xref target="sectionopt"/>, the following options are valid within this block:</t> 196 197<texttable anchor="InterfaceOptions1"> 198 <ttcol>Name</ttcol> 199 <ttcol>Code</ttcol> 200 <ttcol>Length</ttcol> 201 <ttcol>Description</ttcol> 202 203 <c>Hardware</c> 204 <c>2</c> 205 <c>variable</c> 206 <c>An ascii string containing the description of the hardware used to create this section.</c> 207 208 <c>Operating System</c> 209 <c>3</c> 210 <c>variable</c> 211 <c>An ascii string containing the name of the operating system used to create this section.</c> 212 213 <c>User Application</c> 214 <c>3</c> 215 <c>variable</c> 216 <c>An ascii string containing the name of the application used to create this section.</c> 217</texttable> 218 219 220<t>The Section Header Block does not contain data but it rather identifies a list of blocks (interfaces, packets) that are logically correlated. This block does not contain any reference to the size of the section it is currently delimiting, therefore the reader cannot skip a whole section at once. In case a section must be skipped, the user has to repeatedly skip all the blocks contained within it; this makes the parsing of the file slower but it permits to append several capture dumps at the same file.</t> 221</section> 222 223<section anchor="sectionidb" title="Interface Description Block (mandatory)"> 224<t>The Interface Description Block is mandatory. This block is needed to specify the characteristics of the network interface on which the capture has been made. In order to properly associate the captured data to the corresponding interface, the Interface Description Block must be defined before any other block that uses it; therefore, this block is usually placed immediately after the Section Header Block.</t> 225 226<t>An Interface Description Block is valid only inside the section which it belongs to. The structure of a Interface Description Block is shown in <xref target="formatidb"/>.</t> 227 228<figure anchor="formatidb" title="Interface Description Block format."> 229<artwork> 230 0 1 2 3 231 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 232 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 233 | Interface ID | LinkType | 234 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 235 | SnapLen | 236 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 237 / / 238 / Options (variable) / 239 / / 240 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 241 </artwork> 242</figure> 243 244<t>The meaning of the fields is:</t> 245<list style="symbols"> 246<t>Interface ID: a progressive number that identifies uniquely any interface inside current section. Two Interface Description Blocks can have the same Interface ID only if they are in different sections of the file. The Interface ID is referenced by the packet blocks.</t> 247<t>LinkType: a value that defines the link layer type of this interface.</t> 248<t>SnapLen: maximum number of bytes dumped from each packet. The portion of each packet that exceeds this value will not be stored in the file.</t> 249<t>Options: optionally, a list of options (formatted according to the rules defined in <xref target="sectionopt"/>) can be present.</t> 250</list> 251 252<t>In addition to the options defined in <xref target="sectionopt"/>, the following options are valid within this block:</t> 253 254<texttable anchor="InterfaceOptions2"> 255 <ttcol>Name</ttcol> 256 <ttcol>Code</ttcol> 257 <ttcol>Length</ttcol> 258 <ttcol>Description</ttcol> 259 260 <c>if_name</c> 261 <c>2</c> 262 <c>Variable</c> 263 <c>Name of the device used to capture data.</c> 264 265 <c>if_IPv4addr</c> 266 <c>3</c> 267 <c>8</c> 268 <c>Interface network address and netmask.</c> 269 270 <c>if_IPv6addr</c> 271 <c>4</c> 272 <c>17</c> 273 <c>Interface network address and prefix length (stored in the last byte).</c> 274 275 <c>if_MACaddr</c> 276 <c>5</c> 277 <c>6</c> 278 <c>Interface Hardware MAC address (48 bits).</c> 279 280 <c>if_EUIaddr</c> 281 <c>6</c> 282 <c>8</c> 283 <c>Interface Hardware EUI address (64 bits), if available.</c> 284 285 <c>if_speed</c> 286 <c>7</c> 287 <c>8</c> 288 <c>Interface speed (in bps).</c> 289 290 <c>if_tsaccur</c> 291 <c>8</c> 292 <c>1</c> 293 <c>Precision of timestamps. If the Most Significant Bit is equal to zero, the remaining bits indicates the accuracy as as a negative power of 10 (e.g. 6 means microsecond accuracy). If the Most Significant Bit is equal to zero, the remaining bits indicates the accuracy as as negative power of 2 (e.g. 10 means 1/1024 of second). If this option is not present, a precision of 10^-6 is assumed.</c> 294 295 <c>if_tzone</c> 296 <c>9</c> 297 <c>4</c> 298 <c>Time zone for GMT support (TODO: specify better).</c> 299 300 <c>if_flags</c> 301 <c>10</c> 302 <c>4</c> 303 <c>Interface flags. (TODO: specify better. Possible flags: promiscuous, inbound/outbound, traffic filtered during capture).</c> 304 305 <c>if_filter</c> 306 <c>11</c> 307 <c>variable</c> 308 <c>The filter (e.g. "capture only TCP traffic") used to capture traffic. The first byte of the Option Data keeps a code of the filter used (e.g. if this is a libpcap string, or BPF bytecode, and more). More details about this format will be presented in Appendix XXX (TODO).</c> 309 310 <c>if_opersystem</c> 311 <c>12</c> 312 <c>variable</c> 313 <c>An ascii string containing the name of the operating system of the machine that hosts this interface. This can be different from the same information that can be contained by the Section Header Block (<xref target="sectionshb"/>) because the capture can have been done on a remote machine.</c> 314 315</texttable> 316 317</section> 318 319 320 321<section anchor="sectionpb" title="Packet Block (optional)"> 322<t>A Packet Block is the standard container for storing the packets coming from the network. The Packet Block is optional because packets can be stored either by means of this block or the Simple Packet Block, which can be used to speed up dump generation. The format of a packet block is shown in <xref target="formatpb"/>.</t> 323 324<figure anchor="formatpb" title="Packet Block format."> 325<artwork> 326 0 1 2 3 327 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 328 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 329 | Interface ID | Drops Count | 330 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 331 | Timestamp (High) | 332 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 333 | Timestamp (Low) | 334 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 335 | Captured Len | 336 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 337 | Packet Len | 338 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 339 | | 340 | Packet Data | 341 | | 342 | /* variable length, byte-aligned */ | 343 | | 344 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 345 / / 346 / Options (variable) / 347 / / 348 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 349</artwork> 350</figure> 351 352<t>The Packet Block has the following fields:</t> 353 354<list style="symbols"> 355<t>Interface ID: Specifies the interface this packet comes from, and corresponds to the ID of one of the Interface Description Blocks present in this section of the file (see <xref target="formatidb"/>).</t> 356<t>Drops Count: a local drop counter. It specified the number of packets lost (by the interface and the operating system) between this packet and the preceding one. The value xFFFF (in hexadecimal) is reserved for those systems in which this information is not available.</t> 357<t>Timestamp (High): the most significative part of the timestamp. in standard Unix format, i.e. from 1/1/1970.</t> 358<t>Timestamp (Low): the less significative part of the timestamp. The way to interpret this field is specified by the 'ts_accur' option (see <xref target="formatidb"/>) of the Interface Description block referenced by this packet. If the Interface Description block does not contain a 'ts_accur' option, then this field is expressed in microseconds.</t> 359<t>Captured Len: number of bytes captured from the packet (i.e. the length of the Packet Data field). It will be the minimum value among the actual Packet Length and the snapshot length (defined in <xref target="formatidb"/>).</t> 360<t>Packet Len: actual length of the packet when it was transmitted on the network. Can be different from Captured Len if the user wants only a snapshot of the packet.</t> 361<t>Packet Data: the data coming from the network, including link-layer headers. The length of this field is Captured Len. The format of the link-layer headers depends on the LinkType field specified in the Interface Description Block (see <xref target="sectionidb"/>) and it is specified in Appendix XXX (TODO).</t> 362<t>Options: optionally, a list of options (formatted according to the rules defined in <xref target="sectionopt"/>) can be present.</t> 363</list> 364 365<t></t> 366</section> 367 368 369<section title="Simple Packet Block (optional)"> 370<t>The Simple Packet Block is a lightweight container for storing the packets coming from the network. Its presence is optional.</t> 371<t>A Simple Packet Block is similar to a Packet Block (see <xref target="sectionpb"/>), but it is smaller, simpler to process and contains only a minimal set of information. This block is preferred to the standard Packet Block when performance or space occupation are critical factors, such as in sustained traffic dump applications. A capture file can contain both Packet Blocks and Simple Packet Blocks: for example, a capture tool could switch from Packet Blocks to Simple Packet Blocks when the hardware resources become critical.</t> 372<t>The Simple Packet Block does not contain the Interface ID field. Therefore, it must be assumed that all the Simple Packet Blocks have been captured on the interface previously specified in the Interface Description Block.</t> 373<t><xref target="formatpbs"/> shows the format of the Simple Packet Block.</t> 374 375<figure anchor="formatpbs" title="Simple Packet Block format."> 376<artwork> 377 0 1 2 3 378 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 379 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 380 | Packet Len | 381 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 382 | | 383 | Packet Data | 384 | | 385 | /* variable length, byte-aligned */ | 386 | | 387 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 388 </artwork> 389</figure> 390 391<t>The Packet Block has the following fields:</t> 392<list style="symbols"> 393<t>Packet Len: actual length of the packet when it was transmitted on the network. Can be different from captured len if the packet has been truncated.</t> 394<t>Packet data: the data coming from the network, including link-layers headers. The length of this field can be derived from the field Block Total Length, present in the Block Header.</t> 395</list> 396 397<t>The Simple Packet Block does not contain the timestamp because this is one of the most costly operations on PCs. Additionally, there are applications that do not require it; e.g. an Intrusion Detection System is interested in packets, not in their timestamp.</t> 398 399<t>The Simple Packet Block is very efficient in term of disk space: a snapshot of length 100 bytes requires only 16 bytes of overhead, which corresponds to an efficiency of more than 86%.</t> 400 401</section> 402 403 404 405<section title="Name Resolution Block (optional)"> 406<t>The Name Resolution Block is used to support the correlation of numeric addresses (present in the captured packets) and their corresponding canonical names and it is optional. Having the literal names saved in the file, this prevents the need of a name resolution in a delayed time, when the association between names and addresses can be different from the one in use at capture time. Moreover, The Name Resolution Block avoids the need of issuing a lot of DNS requests every time the trace capture is opened, and allows to have name resolution also when reading the capture with a machine not connected to the network.</t> 407<t>The format of the Name Resolution Block is shown in <xref target="formatnrb"/>.</t> 408 409<figure anchor="formatnrb" title="Name Resolution Block format."> 410<artwork> 411 0 1 2 3 412 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 413 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 414 | Record Type | Record Length | 415 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 416 | Record Value | 417 | /* variable length, byte-aligned */ | 418 | + + + + + + + + + + + + + + + + + + + + + + + + + 419 | | | | | 420 +-+-+-+-+-+-+-+-+ + + + + + + + + + + + + + + + + + + + + + + + + 421 . . . other records . . . 422 | Record Type == end_of_recs | Record Length == 00 | 423 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 424 / / 425 / Options (variable) / 426 / / 427 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 428 </artwork> 429</figure> 430 431<t>A Name Resolution Block is a zero-terminated list of records (in the TLV format), each of which contains an association between a network address and a name. There are three possible types of records:</t> 432 433<texttable anchor="nrrecords"> 434 <ttcol>Name</ttcol> 435 <ttcol>Code</ttcol> 436 <ttcol>Length</ttcol> 437 <ttcol>Description</ttcol> 438 439 <c>end_of_recs</c> 440 <c>0</c> 441 <c>0</c> 442 <c>End of records</c> 443 444 <c>ip4_rec</c> 445 <c>1</c> 446 <c>Variable</c> 447 <c>Specifies an IPv4 address (contained in the first 4 bytes), followed by one or more zero-terminated strings containing the DNS entries for that address.</c> 448 449 <c>ip6_rec</c> 450 <c>1</c> 451 <c>Variable</c> 452 <c>Specifies an IPv6 address (contained in the first 16 bytes), followed by one or more zero-terminated strings containing the DNS entries for that address.</c> 453</texttable> 454 455<t>After the list or Name Resolution Records, optionally, a list of options (formatted according to the rules defined in <xref target="sectionopt"/>) can be present.</t> 456 457<t>A Name Resolution Block is normally placed at the beginning of the file, but no assumptions can be taken about its position. Name Resolution Blocks can be added in a second time by tools that process the file, like network analyzers.</t> 458 459<t>In addiction to the options defined in <xref target="sectionopt"/>, the following options are valid within this block:</t> 460 461<texttable> 462 <ttcol>Name</ttcol> 463 <ttcol>Code</ttcol> 464 <ttcol>Length</ttcol> 465 <ttcol>Description</ttcol> 466 467 <c>ns_dnsname</c> 468 <c>2</c> 469 <c>Variable</c> 470 <c>An ascii string containing the name of the machine (DNS server) used to perform the name resolution.</c> 471</texttable> 472 473</section> 474 475 476<section title="Interface Statistics Block (optional)"> 477<t>The Interface Statistics Block contains the capture statistics for a given interface and it is optional. The statistics are referred to the interface defined in the current Section identified by the Interface ID field.</t> 478<t>The format of the Interface Statistics Block is shown in <xref target="formatisb"/>.</t> 479 480<figure anchor="formatisb" title="Interface Statistics Block format."> 481<artwork> 482 0 1 2 3 483 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 484 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 485 | IfRecv | 486 | (high + low) | 487 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 488 | IfDrop | 489 | (high + low) | 490 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 491 | FilterAccept | 492 | (high + low) | 493 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 494 | OSDrop | 495 | (high + low) | 496 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 497 | UsrDelivered | 498 | (high + low) | 499 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 500 | Interface ID | Reserved | 501 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 502 / / 503 / Options (variable) / 504 / / 505 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 506 </artwork> 507</figure> 508 509<t>The fields have the following meaning:</t> 510 511<list style="symbols"> 512<t>IfRecv: number of packets received from the interface during the capture. This number is reported as a 64 bits value, in which the most significat bits are located in the first four bytes of the field.</t> 513<t>IfDrop: number of packets dropped by the interface during the capture due to lack of resources.</t> 514<t>FilterAccept: number of packets accepeted by filter during current capture.</t> 515<t>OSDrop: number of packets dropped by the operating system during the capture.</t> 516<t>UsrDelivered: number of packets delivered to the user. UsrDelivered can be different from the value 'FilterAccept - OSDropped' because some packets could still lay in the OS buffers when the capture ended.</t> 517<t>Interface ID: reference to an Interface Description Block.</t> 518<t>Reserved: Reserved to future use.</t> 519<t>Options: optionally, a list of options (formatted according to the rules defined in <xref target="sectionopt"/>) can be present.</t> 520</list> 521 522<t>In addiction to the options defined in <xref target="sectionopt"/>, the following options are valid within this block:</t> 523 524<texttable> 525 <ttcol>Name</ttcol> 526 <ttcol>Code</ttcol> 527 <ttcol>Length</ttcol> 528 <ttcol>Description</ttcol> 529 530 <c>isb_starttime</c> 531 <c>2</c> 532 <c>8</c> 533 <c>Time in which the capture started; time will be stored in two blocks of four bytes each, containing the timestamp in seconds and nanoseconds.</c> 534 535 <c>isb_endtime</c> 536 <c>3</c> 537 <c>8</c> 538 <c>Time in which the capture started; time will be stored in two blocks of four bytes each, containing the timestamp in seconds and nanoseconds.</c> 539</texttable> 540 541</section> 542</section> 543 544 545 546<section anchor="sectionopt" title="Options"> 547<t>Almost all blocks have the possibility to embed optional fields. Optional fields can be used to insert some information that may be useful when reading data, but that it is not really needed for packet processing. Therefore, each tool can be either read the content of the optional fields (if any), or skip them at once.</t> 548<t>Skipping all the optional fields at once is straightforward because most of the blocks have a fixed length, therefore the field Block Length (present in the General Block Structure, see <xref target="sectionblock"/>) can be used to skip everything till the next block.</t> 549 550<t>Options are a list of Type - Length - Value fields, each one containing a single value:</t> 551 552<list style="symbols"> 553<t>Option Type (2 bytes): it contains the code that specifies the type of the current TLV record. Option types whose Most Significant Bit is equal to one are reserved for local use; therefore, there is no guarantee that the code used is unique among all capture files (generated by other applications). In case of vendor-specific extensions that have to be identified uniquely, vendors must request an Option Code whose MSB is equal to zero.</t> 554<t>Option Length (2 bytes): it contains the length of the following 'Option Value' field.</t> 555<t>Option Value (variable length): it contains the value of the given option. The length of this field as been specified by the Option Length field.</t> 556</list> 557 558<t>Options may be repeated several times (e.g. an interface that has several IP addresses associated to it). The option list is terminated by a special code which is the 'End of Option'.</t> 559 560<t>The format of the optional fields is shown in <xref target="formatopt"/>.</t> 561 562<figure anchor="formatopt" title="Options format."> 563<artwork> 564 0 1 2 3 565 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 566 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 567 | Option Code | Option Length | 568 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 569 | Option Value | 570 | /* variable length, byte-aligned */ | 571 | + + + + + + + + + + + + + + + + + + + + + + + + + 572 | / / / | 573 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 574 / / 575 / . . . other options . . . / 576 / / 577 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 578 | Option Code == opt_endofopt | Option Length == 0 | 579 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 580 </artwork> 581</figure> 582 583<t>The following codes can always be present in any optional field:</t> 584 585<texttable> 586 <ttcol>Name</ttcol> 587 <ttcol>Code</ttcol> 588 <ttcol>Length</ttcol> 589 <ttcol>Description</ttcol> 590 591 <c>opt_endofopt</c> 592 <c>0</c> 593 <c>0</c> 594 <c>End of options: it is used to delimit the end of the optional fields. This block cannot be repeated within a given list of options.</c> 595 596 <c>opt_comment</c> 597 <c>1</c> 598 <c>variable</c> 599 <c>Comment: it is an ascii string containing a comment that is associated to the current block.</c> 600</texttable> 601 602</section> 603 604 605 606 607<section title="Experimental Blocks (deserved to a further investigation)"> 608 609<section title="Other Packet Blocks (experimental)"> 610<t>Can some other packet blocks (besides the two described in the previous paragraphs) be useful?</t> 611</section> 612 613<section title="Compression Block (experimental)"> 614<t>The Compression Block is optional. A file can contain an arbitrary number of these blocks. A Compression Block, as the name says, is used to store compressed data. Its format is shown in <xref target="formatcb"/>.</t> 615 616<figure anchor="formatcb" title="Compression Block format."> 617<artwork> 618 0 1 2 3 619 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 620 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 621 | Compr. Type | | 622 +-+-+-+-+-+-+-+-+ | 623 | | 624 | Compressed Data | 625 | | 626 | /* variable length, byte-aligned */ | 627 | | 628 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 629 </artwork> 630</figure> 631 632<t>The fields have the following meaning:</t> 633 634<list style="symbols"> 635<t>Compression Type: specifies the compression algorithm. Possible values for this field are 0 (uncompressed), 1 (Lempel Ziv), 2 (Gzip), other?? Probably some kind of dumb and fast compression algorithm could be effective with some types of traffic (for example web), but which?</t> 636<t>Compressed Data: data of this block. Once decompressed, it is made of other blocks.</t> 637</list> 638 639</section> 640 641 642<section title="Encryption Block (experimental)"> 643<t>The Encryption Block is optional. A file can contain an arbitrary number of these blocks. An Encryption Block is used to sotre encrypted data. Its format is shown in <xref target="formateb"/>.</t> 644 645<figure anchor="formateb" title="Encryption Block format."> 646<artwork> 647 0 1 2 3 648 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 649 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 650 | Encr. Type | | 651 +-+-+-+-+-+-+-+-+ | 652 | | 653 | Compressed Data | 654 | | 655 | /* variable length, byte-aligned */ | 656 | | 657 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 658 </artwork> 659</figure> 660 661<t>The fields have the following meaning:</t> 662<list style="symbols"> 663<t>Compression Type: specifies the encryption algorithm. Possible values for this field are ??? NOTE: this block should probably contain other fields, depending on the encryption algorithm. To be define precisely.</t> 664<t>Encrypted Data: data of this block. Once decripted, it consists of other blocks.</t> 665</list> 666 667</section> 668 669 670<section title="Fixed Length Block (experimental)"> 671<t>The Fixed Length Block is optional. A file can contain an arbitrary number of these blocks. A Fixed Length Block can be used to optimize the access to the file. Its format is shown in <xref target="formatflm"/>. 672A Fixed Length Block stores records with constant size. It contains a set of Blocks (normally Packet Blocks or Simple Packet Blocks), of wihich it specifies the size. Knowing this size a priori helps to scan the file and to load some portions of it without truncating a block, and is particularly useful with cell-based networks like ATM.</t> 673 674<figure anchor="formatflm" title="Fixed Length Block format."> 675<artwork> 676 0 1 2 3 677 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 678 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 679 | Cell Size | | 680 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 681 | | 682 | Fixed Size Data | 683 | | 684 | /* variable length, byte-aligned */ | 685 | | 686 | | 687 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 688 </artwork> 689</figure> 690 691<t>The fields have the following meaning:</t> 692<list style="symbols"> 693<t>Cell size: the size of the blocks contained in the data field.</t> 694<t>Fixed Size Data: data of this block.</t> 695</list> 696 697</section> 698 699<section title="Directory Block (experimental)"> 700<t>If present, this block contains the following information:</t> 701<list style="symbols"> 702<t>number of indexed packets (N)</t> 703<t>table with position and length of any indexed packet (N entries)</t> 704</list> 705 706<t>A directory block must be followed by at least N packets, otherwise it must be considered invalid. It can be used to efficiently load portions of the file to memory and to support operations on memory mapped files. This block can be added by tools like network analyzers as a consequence of file processing.</t> 707</section> 708 709<section title="Traffic Statistics and Monitoring Blocks (experimental)"> 710<t>One or more blocks could be defined to contain network statistics or traffic monitoring information. They could be use to store data collected from RMON or Netflow probes, or from other network monitoring tools.</t> 711</section> 712 713<section title="Event/Security Block (experimental)"> 714<t>This block could be used to store events. Events could contain generic information (for example network load over 50%, server down...) or security alerts. An event could be:</t> 715 716<list style="symbols"> 717<t>skipped, if the application doesn't know how to do with it</t> 718<t>processed independently by the packets. In other words, the applications skips the packets and processes only the alerts</t> 719<t>processed in relation to packets: for example, a security tool could load only the packets of the file that are near a security alert; a monitorg tool could skip the packets captured while the server was down.</t> 720</list> 721 722</section> 723 724</section> 725 726 727 728 729<section title="Conclusions"> 730<t>The file format proposed in this document should be very versatile and satisfy a wide range of applications. 731In the simplest case, it can contain a raw dump of the network data, made of a series of Simple Packet Blocks. 732In the most complex case, it can be used as a repository for heterogeneous information. 733In every case, the file remains easy to parse and an application can always skip the data it is not interested in; at the same time, different applications can share the file, and each of them can benfit of the information produced by the others. 734Two or more files can be concatenated obtaining another valid file.</t> 735</section> 736 737 738<section title="Most important open issues"> 739<list style="symbols"> 740<t>Data, in the file, must be byte or word aligned? Currently, the structure of this document is not consistent with respect to this point.</t> 741</list> 742</section> 743 744</middle> 745 746</rfc> 747