1# Annotating FlatBuffers 2 3This provides a way to annotate flatbuffer binary data, byte-by-byte, with a 4schema. It is useful for development purposes and understanding the details of 5the internal format. 6 7## Annotating 8 9Given a `schema`, as either a plain-text (`.fbs`) or a binary schema (`.bfbs`), 10and `binary` file(s) that were created by the `schema`. You can annotate them 11using: 12 13```sh 14flatc --annotate SCHEMA -- BINARY_FILES... 15``` 16 17This will produce a set of annotated files (`.afb` Annotated FlatBuffer) 18corresponding to the input binary files. 19 20### Example 21 22Taken from the [tests/annotated_binary](https://github.com/google/flatbuffers/tree/master/tests/annotated_binary). 23 24```sh 25cd tests/annotated_binary 26../../flatc --annotate annotated_binary.fbs -- annotated_binary.bin 27``` 28 29Which will produce a `annotated_binary.afb` file in the current directory. 30 31The `annotated_binary.bin` is the flatbufer binary of the data contained within 32`annotated_binary.json`, which was made by the following command: 33 34```sh 35..\..\flatc -b annotated_binary.fbs annotated_binary.json 36``` 37 38## .afb Text Format 39 40Currently there is a built-in text-based format for outputting the annotations. 41A full example is shown here: 42[`annotated_binary.afb`](https://github.com/google/flatbuffers/blob/master/tests/annotated_binary/annotated_binary.afb) 43 44The data is organized as a table with fixed [columns](#columns) grouped into 45Binary [sections](#binary-sections) and [regions](#binary-regions), starting 46from the beginning of the binary (offset `0`). 47 48### Columns 49 50The columns are as follows: 51 521. The offset from the start of the binary, expressed in hexadecimal format 53 (e.g. `+0x003c`). 54 55 The prefix `+` is added to make searching for the offset (compared to some 56 random value) a bit easier. 57 582. The raw binary data, expressed in hexadecimal format. 59 60 This is in the little endian format the buffer uses internally and what you 61 would see with a normal binary text viewer. 62 633. The type of the data. 64 65 This may be the type specified in the schema or some internally defined 66 types: 67 68 69 | Internal Type | Purpose | 70 |---------------|----------------------------------------------------| 71 | `VOffset16` | Virtual table offset, relative to the table offset | 72 | `UOffset32` | Unsigned offset, relative to the current offset | 73 | `SOffset32` | Signed offset, relative to the current offset | 74 75 764. The value of the data. 77 78 This is shown in big endian format that is generally written for humans to 79 consume (e.g. `0x0013`). As well as the "casted" value (e.g. `0x0013 `is 80 `19` in decimal) in parentheses. 81 825. Notes about the particular data. 83 84 This describes what the data is about, either some internal usage, or tied 85 to the schema. 86 87### Binary Sections 88 89The file is broken up into Binary Sections, which are comprised of contiguous 90[binary regions](#binary-regions) that are logically grouped together. For 91example, a binary section may be a single instance of a flatbuffer `Table` or 92its `vtable`. The sections may be labelled with the name of the associated type, 93as defined in the input schema. 94 95An example of a `vtable` Binary Section that is associated with the user-defined 96`AnnotateBinary.Bar` table. 97 98``` 99vtable (AnnotatedBinary.Bar): 100 +0x00A0 | 08 00 | uint16_t | 0x0008 (8) | size of this vtable 101 +0x00A2 | 13 00 | uint16_t | 0x0013 (19) | size of referring table 102 +0x00A4 | 08 00 | VOffset16 | 0x0008 (8) | offset to field `a` (id: 0) 103 +0x00A6 | 04 00 | VOffset16 | 0x0004 (4) | offset to field `b` (id: 1) 104``` 105 106These are purely annotative, there is no embedded information about these 107regions in the flatbuffer itself. 108 109### Binary Regions 110 111Binary regions are contiguous bytes regions that are grouped together to form 112some sort of value, e.g. a `scalar` or an array of scalars. A binary region may 113be split up over multiple text lines, if the size of the region is large. 114 115#### Annotation Example 116 117Looking at an example binary region: 118 119``` 120vtable (AnnotatedBinary.Bar): 121 +0x00A0 | 08 00 | uint16_t | 0x0008 (8) | size of this vtable 122``` 123 124The first column (`+0x00A0`) is the offset to this region from the beginning of 125the buffer. 126 127The second column are the raw bytes (hexadecimal) that make up this region. 128These are expressed in the little-endian format that flatbuffers uses for the 129wire format. 130 131The third column is the type to interpret the bytes as. For the above example, 132the type is `uint16_t` which is a 16-bit unsigned integer type. 133 134The fourth column shows the raw bytes as a compacted, big-endian value. The raw 135bytes are duplicated in this fashion since it is more intuitive to read the data 136in the big-endian format (e.g., `0x0008`). This value is followed by the decimal 137representation of the value (e.g., `(8)`). For strings, the raw string value is 138shown instead. 139 140The fifth column is a textual comment on what the value is. As much metadata as 141known is provided. 142 143### Offsets 144 145If the type in the 3rd column is of an absolute offset (`SOffet32` or 146`Offset32`), the fourth column also shows an `Loc: +0x025A` value which shows 147where in the binary this region is pointing to. These values are absolute from 148the beginning of the file, their calculation from the raw value in the 4th 149column depends on the context.