• Home
Name Date Size #Lines LOC

..--

parse/03-May-2024-255157

verify/03-May-2024-844600

.clang-formatD03-May-2024239

README.mdD03-May-20244.6 KiB10891

README.md

1# `libjsonpbparse`
2
3This library provides functions to parse a JSON file to a structured Protobuf
4message.
5
6At this time of writing, `libprotobuf-cpp-full` is at version 3.0.0-beta, and
7unknown fields in a JSON file cannot be ignored. Do **NOT** use this library in
8vendor / recovery until `libprotobuf-cpp-full` is updated.
9
10## Using `libjsoncpp` in parser code
11
12Since `libjsonpbparse` cannot be used in vendor / recovery processes yet,
13`libjsoncpp` is used instead. However, there are notable differences in the
14logic of `libjsoncpp` and `libprotobuf` when parsing JSON files.
15
16- There are no implicit string to integer conversion in `libjsoncpp`. Hence:
17  - If the Protobuf schema uses 64-bit integers (`(s|fixed|u|)int64`):
18    - The JSON file must use strings (to pass tests in `libjsonpbverify`)
19    - Parser code (that uses `libjsoncpp`) must explicitly convert strings to
20      integers. Example:
21      ```c++
22      strtoull(value.asString(), 0, 10)
23      ```
24  - If the Protobuf schema uses special floating point values:
25    - The JSON file must use strings (e.g. `"NaN"`, `"Infinity"`, `"-Infinity"`)
26    - Parser code must explicitly handle these cases. Example:
27      ```c++
28      double d;
29      if (value.isNumeric()) {
30        d = value.asDouble();
31      } else {
32        auto&& s = value.asString();
33        if (s == "NaN") d = std::numeric_limits<double>::quiet_NaN();
34        else if (s == "Infinity") d = std::numeric_limits<double>::infinity();
35        else if (s == "-Infinity") d = -std::numeric_limits<double>::infinity();
36      }
37      ```
38- `libprotobuf` accepts either `lowerCamelCase` (or `json_name` option if it is
39  defined) or the original field name as keys in the input JSON file.
40  The test in `libjsonpbverify` explicitly check this case to avoid ambiguity;
41  only the original field name (or `json_name` option if it is defined) can be
42  used.
43
44Once `libprotobuf` in the source tree is updated to a higher version and
45`libjsonpbparse` is updated to ignore unknown fields in JSON files, all parsing
46code must be converted to use `libjsonpbparse` for consistency.
47
48# `libjsonpbverify`
49
50This library provides functions and tests to examine a JSON file and validate
51it against a Protobuf message definition.
52
53In addition to a validity check that `libprotobuf` can convert the JSON file to a
54Protobuf message (using `libjsonpbparse`), it also checks the following:
55
56- Whether there are fields unknown to the schema. All fields in the JSON file
57  must be well defined in the schema.
58- Whether the Protobuf file defines JSON keys clearly. The JSON keys must be
59  the `json_name` option of a Protobuf field, or name of a Protobuf field if
60  `json_name` is not defined. `lowerCamelCase` supported by `libprotobuf` is
61  explicitly disallowed (unless explicitly used in `json_name`). For example,
62  in the following Protobuf file, only keys `foo_bar` and `barBaz` are allowed
63  in the JSON file:
64  ```
65  message Foo {
66      string foo_bar = 1;
67      string bar_baz = 2 [json_name = "barBaz"];
68  }
69  ```
70- Whether `json == convert_to_json(convert_to_pb(json))`, using `libprotobuf`.
71  This imposes additional restrictions including:
72  - Enum values must be present as names (not integer values) in the JSON file.
73  - 64-bit integers and special floating point values (infinity, NaN) must
74    always be strings.
75
76## Defining a JSON schema using Protobuf
77
78Check [JSON Mapping](https://developers.google.com/protocol-buffers/docs/proto3#json)
79before defining a Protobuf object as a JSON schema. In general:
80
81- **Use proto3**. `libjsonverify` does not support proto2.
82- JSON booleans should be `bool`.
83- JSON numbers should be `(s|fixed|u|)int32`, `float`, or `double` in the schema
84- JSON strings are generally `string`s, but if you want to impose more
85  restrictions on the string, you can also use `Timestamp`, `bytes`,
86  **`float`** or **`double`** (if NaN and infinity are valid values),
87  enumerations, etc.
88  - If a custom enumeration is used, parser code should **NOT** error when the
89    enumeration value name is unknown, as enumeration definitions may be
90    extended in the future.
91- JSON arrays should be repeated fields.
92- JSON objects should be a well-defined `message`, unless you have a good reason
93  to use `map<string, T>`.
94- Don't use `Any`; it defeats the purpose of having the schema.
95
96## Validating a JSON file against a Protobuf definition
97
98Example:
99```c++
100#include <jsonpb/verify.h>
101using namespace ::android::jsonpb;
102std::unique_ptr<JsonSchemaTestConfig> CreateCgroupsParam() {
103
104}
105INSTANTIATE_TEST_SUITE_P(LibProcessgroupProto, JsonSchemaTest,
106                         ::testing::Values(MakeTestParam<Cgroups>("cgroups.json")));
107```
108