1### Protobuf Editions for Schema Producers 2 3**Author:** [@fowles](https://github.com/fowles) 4 5Explains the expected use of and interaction with editions for schema providers 6and their customers. 7 8## Background 9 10The [Protobuf Editions](what-are-protobuf-editions.md) project uses "editions" 11to allow Protobuf to safely evolve over time. This is primarily accomplished 12through ["features"](protobuf-editions-design-features.md). The first edition 13(colloquially known as "Edition Zero") will use features to unify proto2 and 14proto3 ([Edition Zero Features](edition-zero-features.md)). This document will 15use definitions from [Protobuf Editions: Rollout](protobuf-editions-rollout.md) 16but focus primarily on the use case of **Schema Producers** and **Schema 17Consumers**: 18 19> **Schema Producers** are teams that produce `.proto` files for the consumption 20> of other teams. 21 22As a reminder, features will generally not change the wire format of messages 23and thus changing the edition for a `.proto` will not change the wire format of 24message. 25 26## Initial Release 27 28There will be a large period of time during which `protoc` is able to consume 29`proto3`, `proto2`, and editions files. Once all of the supported `protoc` 30releases handle editions, schema producers should upgrade their published 31`.proto` files to edition zero. The protobuf team will provide a tool that 32upgrades `proto2` and `proto3` files to edition zero in a fully compatible way. 33 34### Edition Zero Features 35 36In order to unify proto2 and proto3, "Edition Zero" is taking an opinionated 37stance on which choices are good and bad, by choosing "good" defaults and 38requiring explicit requests for the "bad" semantics 39([Edition Zero Features](edition-zero-features.md)). Schema producers that are 40simply upgrading existing `.proto` files should publish these files as produced 41by the upgrade tool. This will ensure wire compatibility. Newly published 42`.proto` files should use the default values from this first edition. 43 44## Steady State Flow 45 46For the most part, editions should not disturb the general pattern for schema 47producers. Any schema producer should already specify what versions of protobuf 48they support and should not support versions of protobuf that are themselves 49unsupported. Schema producers should generally publish all of their `.proto` 50files with a consistent edition for the simplicity of their users. When updating 51the edition for their `.proto` files, producers should target an edition 52supported by all of the versions of protobuf in their support matrix. **A good 53rule of thumb is to target the newest edition supported by the oldest release of 54protobuf in the support matrix.** 55 56### Best Practices 57 58#### Publish `.proto` files 59 60Schema producers should publish `.proto` files and not generated sources. This 61is already the case and editions do not change it. Publishing generated sources 62can lead to mismatches between the compiler and runtime environment used. 63Protobuf does not support mixed generation/runtime configurations and sometimes 64security patches require updating both. 65 66#### Minimize use of features 67 68Codegenerator specific [features](protobuf-editions-design-features.md) (like 69`features.(pb.cpp).string_field_type`) should only be applied within the context 70of a single code base. **Consumers** of published schemas may wish to add 71generator specific features (either by hand or with an automated `.proto` 72refactoring tool), but **producers** should not force that onto users. 73 74## Client Usage Patterns 75 76Consumers’ usage is heavily constrained by their build system. Language agnostic 77build systems, like Bazel, can run `protoc` as one of the build steps. Language 78specific build systems, like Maven or Go, make running `protoc` more difficult 79and so consumers often avoid it. Languages like Python that traditionally lack a 80build system are more extreme. 81 82### Running `protoc` Directly 83 84Because language-specific features will not change the wire format of messages, 85clients will be able to update to newer editions or specify specific features 86appropriate to their environment while still connecting to external endpoints. 87 88In particular, protobuf will provide two distinct mechanisms for supporting 89these users. First, we will provide tools for automating updates to `.proto` 90files in a safe way. These tools will apply semantic patches to `.proto` files 91that they can then commit into source control. Second, we will provide 92primitives in `protoc` to compile a `.proto` file and a semantic patch as a set 93of inputs so that users never have to materialize the modified `.proto` file. 94Protobuf team will investigate adding support for semantic patches when it 95addresses Bazel rules. 96 97In the long term, we want a Bazel rule (and possibly similar for other build 98systems) that seamlessly packages changes like: 99 100```proto 101proto_library( 102 name = "cloud_spanner_proto", 103 modifications = ["cloud_spanner.change_spec"], 104 deps = ["@com_google_cloud//spanner"], 105) 106``` 107 108### Publishing Generated Libraries 109 110As a reminder, publishing generated code is not a good idea. It frequently runs 111afoul of runtime/generation time mismatches and is an active source of confusion 112where users are unable to reason about what version they are on. 113 114Teams determined to do this anyway should adhere to the following best 115practices. 116 117* Publish generated libraries using semver. 118* Publish both the generated code and the protobuf runtime. 119* Pin the protobuf runtime library to the exact version used to generate the 120 library. 121* When upgrading the `protoc` generator for a major, minor, or micro release, 122 increment the corresponding number in the published library’s version. 123* When updating the edition of the `.proto` file, increment the major number 124 of the published library’s version. 125 126It is worth note that only the last bullet point is new, everything else is a 127restatement of current best practice. 128