diff --git a/README.md b/README.md index b2b3558..6c081f7 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,207 @@ protoplugin [![GoDoc](https://pkg.go.dev/badge/github.com/bufbuild/protoplugin.svg)](https://pkg.go.dev/github.com/bufbuild/protoplugin) [![Slack](https://img.shields.io/badge/slack-buf-%23e01563)](https://buf.build/links/slack) -This is a simple library to assist in writing `protoc` plugins in go. +The premise of this library is pretty simple: writing your [protoc](https://github.com/protocolbuffers/protobuf) +plugins is a powerful way to make Protobuf even more useful, but to do so, we all have to write the same +boilerplate scaffolding over and over again. Additionally, you want to make sure that you have a +[`CodeGeneratorRequest`](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/plugin.proto) +that had sensibile validations applied to it, and that you produce a valid +[CodeGeneratorResponse](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/plugin.proto). +It's really easy to produce a `CodeGeneratorResponse` that `protoc` or [buf](https://github.com/bufbuild/buf) will happily consume, but is actually invalid and will result in unexpected generated code. + +`Protoplugin` takes care of all of this for you, and nothing more. `Protoplugin` makes authoring `protoc` plugins +in Golang dead-simple. `Protoplugin` will: + +- Deal with all of the boilerplate of consuming `CodeGeneratorRequests` and `CodeGeneratorResponses` for you, + providing you with a simple [Handler](https://pkg.go.dev/github.com/bufbuild/protoplugin#Handler) interface + to implement. +- Validate that the `CodeGeneratorRequest` consumed matches basic expectations. +- Help you create a `CodeGeneratorResponse` that is valid, or give you an error otherwise. + +The validation performed takes into account years of experience we've had here at [Buf](https://buf.build) to +handle edge cases you've never thought about - the same code backs the execution of plugins within the +`buf` compiler, and has dealt with handling plugins that misbehave in ways we never would have expected. + +`Protoplugin` is also ready for [Protobuf Editions](https://protobuf.dev/editions) from day one, helping you +navigate this new Protobuf functionality with ease. + +`Protoplugin` has a single non-test dependency, on +`google.golang.org/protobuf` - all other dependencies in [`go.mod`](go.mod) are for `protoplugin`'s own +tests, and will not result in additional dependencies in your code. + +If you are authoring `protoc` plugins in Golang that do anything other than produce `.go` files, +you should use `protoplugin`. It's the foundational library you need, and it doesn't include anything +you don't want. + +If you are authoring `protoc` plugins that produce `.go` files, you +should use [protogen](https://pkg.go.dev/google.golang.org/protobuf/compiler/protogen), as it has +Golang-specific helpers, such as dealing with Golang import paths, and handling the standard Golang +`protoc` plugin flags like (`paths=source_relative`). However, `protogen` is very Golang-specific - +the interface exposed doesn't really make sense outside of generating `.go` files, and you specifically +do not want most plugins to expose the standard Golang `protoc` plugin flags. If you'd like to use `protogen` +but also take advantage of `protoplugin`'s hardening, it's very easy to wrap `protogen` with +`protoplugin` - see the [protoc-gen-protogen-simple](internal/examples/protoc-gen-protogen-simple/main.go) +example in this repository. + +## Protoplugin Handlers + +Implementing a plugin with `protoplugin` is as simple as implementing a `Handler`. The `Handler` +is then typically invoked via `protoplugin.Main`, or if doing testing, via `protoplugin.Run` (which +gives you control over `args, stdin, stdout, stderr`). + +Here's a simple plugin that just prints the top-level message names for all files in `file_to_generate`: + +```go +// Package main implements a very simple plugin that just outputs text files +// with the names of the top-level messages in each file. +// +// Example: if a/b.proto had top-level messages C, D, the file "a/b.proto.txt" would be +// outputted, containing "C\nD\n". +package main + +import ( + "context" + "strings" + + "github.com/bufbuild/protoplugin" +) + +func main() { + protoplugin.Main(context.Background(), protoplugin.HandlerFunc(handle)) +} + +func handle( + _ context.Context, + responseWriter *protoplugin.ResponseWriter, + request *protoplugin.Request, +) error { + // Set the flag indicating that we support proto3 optionals. We don't even use them in this + // plugin, but protoc will error if it encounters a proto3 file with an optional but the + // plugin has not indicated it will support it. + responseWriter.AddFeatureProto3Optional() + + fileDescriptorProtos, err := request.GenerateFileDescriptorProtos() + if err != nil { + return err + } + for _, fileDescriptorProto := range fileDescriptorProtos { + topLevelMessageNames := make([]string, len(fileDescriptorProto.GetMessageType())) + for i, descriptorProto := range fileDescriptorProto.GetMessageType() { + topLevelMessageNames[i] = descriptorProto.GetName() + } + // Add the response file to the response. + responseWriter.AddFile( + fileDescriptorProto.GetName()+".txt", + strings.Join(topLevelMessageNames, "\n")+"\n", + ) + } + + return nil +} +``` + +A `Handler` takes a [`Request`](https://pkg.go.dev/github.com/bufbuild/protoplugin#Request), and expects a response +to be written to the [`ResponseWriter`](https://pkg.go.dev/github.com/bufbuild/protoplugin#ResponseWriter). + +### Requests + +A `Request` wraps a `CodeGeneratorRequest`, but performs common-sense validation. The `Handler` can assume +all of the following is true: + +- The `CodeGeneratorRequest` given to the plugin was not nil. +- `file_to_generate and `proto_file were not empty. +- Each `FileDescriptorProto` in `proto_file` will have a valid path (see below) as the `name` field. +- Each value of `file_to_generate` will be a valid path. +- Each value of `file_to_generate` will have a corresponding value in `proto_file`. +- (For editions) if `source_file_descriptors` is not empty, each `FileDescriptorProto` in + `source_file_descriptors` will have a valid path as the name field. +- (For editions) if `source_file_descriptors` is not empty, the values of `file_to_generate` will + have a 1-1 mapping to the names in `source_file_descriptors`. + +Paths are considered valid if they are non-empty, relative, use '/' as the path separator, do not jump context (`..`), +and have `.proto` as the file extension. + +If any of these validations fail, the plugin will exit with a non-zero exit code. + +This is all per the spec of `CodeGeneratorRequest`, but you'd be surprised what producers of +`CodeGeneratorRequests` (including `protoc` and `buf`) can do - compilers are not immune to bugs! + +A Request exposes two ways to get the file information off of the `CodeGeneratorRequest`: + +- Via the rich [protoreflect API](https://pkg.go.dev/google.golang.org/protobuf@v1.32.0/reflect/protoreflect) + by the types [`protoreflect.FileDescriptor`](https://pkg.go.dev/google.golang.org/protobuf@v1.32.0/reflect/protoreflect#FileDescriptor) + and [`*protoregistry.Files`](https://pkg.go.dev/google.golang.org/protobuf@v1.32.0/reflect/protoregistry#Files) +- Directly via the `FileDescriptorProtos`. + +The methods `GenerateFileDescriptors` and `GenerateFileDescriptorProtos` will provide file information +only for those files specified in `file_to_generate`, while `AllFiles` and `AllFileDescriptorProtos` +will provide file information for all files in `proto_file`. + +See [protoc-gen-protoreflect-simple](internal/examples/protoc-gen-protoreflect-simple/main.go) for a simple +example using the `protoreflect` API, and [protoc-gen-simple](internal/examples/protoc-gen-simple/main.go) +for a simple example using the `FileDescriptorProtos` directly. + +Additionally, if the option `WithSourceRetentionOptions` is specified, any of these methods will return the files +with source-retention options automatically. This is a new Editions feature that most plugin authors do not +need to be concerned with yet. + +A `Request` also exposes the `Parameters` and `CompilerVersion` specified on the `CodeGeneratorRequest`, +the latter with validation the the version is valid. Additionally, if you need low-level access, a +`CodeGeneratorRequest` method is provided to expose the underlying `CodeGeneratorRequest` + +### ResponseWriters + +A `ResponseWriter` builds `CodeGeneratorRequests` for you. The most common methods you will use: + +- `AddFiles`: Add a new file with content. +- `AddError`: Add to the error message that will be propagated to the compiler. +- `AddFeatureProto3Optional`: Denote that your plugin handles `optional` in `proto3` (all new plugins should set this). +- `AddFeatureSupportsEditions`: Denote that you support editions (most plugins will not yet). + +A `ResponseWriter` also provide low-level access for advanced plugins that need to build the `CodeGeneratorRequest` +more directly: + +- `AddCodeGeneratorResponseFiles`: Add `CodeGeneratorResponse.File`s directly. May be needed when using i.e. + insertion points. +- `AddSupportedFeatures`: Set supported features directly. +- `SetMinimumEdition/SetMaximumEdition`: directly set the minimum and maximum Edition supported. + +For most authors, however, you should use the common methods. + +`ResponseWriters` will also perform validation for you. The following must be true: + +- All files added must have non-empty, relative paths that use '/' as the path separator and do not jump context (`..`). +- The minimum Edition must be less than or equal to the maximum Edition. + +If any of these validations fail, the plugin will exit with a non-zero exit code. + +A warning will also be produced if: + +- Files do not have unique names. `protoc` will continue on without erroring if this happens, but will just + silently drop all occurrences of the file after the name has already been seen. In almost all cases, + a duplicate name is plugin authoring issue, and here at Buf, we've seen a lot of plugins have this issue! +- Any file path is not cleaned. + +By default, warnings are produced to `stderr`, however this can be overridden via the `WithWarningHandler` +option to `Main` or `Run`. + +## What this library is not + +This library is not a full-fledged plugin authoring framework with language-specific interfaces, +and doesn't intend to be. The only language-specific framework in wide use that we are aware of: + +- [`protogen`](https://pkg.go.dev/google.golang.org/protobuf/compiler/protogen): As mentioned in the introduction, + `protogen` is the standard way to write plugins that produce `.go` files, however it is specific to this purpose - + if you are writing a plugin in Golang that produces Ruby, Python, YAML, etc, you are better-served without the + Golang-specific interfaces, and the Golang-specific `protoc` plugin flags that all `protogen`-authored plugins + have added. +- [`bufbuild/protobuf-es/protoplugin`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoplugin): + framework for writing JavaScript/TypeScript plugins that we also authored. It's great! And it backs + [`protobuf-es`](https://github.com/bufbuild/protobuf-es) and [`connect-es`](https://github.com/connectrpc/connect-es). + +It would be great if there were other language-specific frameworks out there, and perhaps we will get to it +some day. However, `protoplugin` is meant to be the foundational layer that every Golang-authored plugin wants: +it gives you the basics, so you don't have to write them again and again. ## Status: Alpha @@ -14,6 +214,4 @@ This module is still being developed and may change. Proper documentation will c ## Legal -Offered under the [Apache 2 license][license]. - -[license]: https://github.com/bufbuild/protoplugin/blob/main/LICENSE +Offered under the [Apache 2 license](https://github.com/bufbuild/protoplugin/blob/main/LICENSE). diff --git a/handler.go b/handler.go index 8030733..b4993ff 100644 --- a/handler.go +++ b/handler.go @@ -29,6 +29,9 @@ type Handler interface { // - Each FileDescriptorProto in ProtoFile will have a valid path as the name field. // - Each value of FileToGenerate will be a valid path. // - Each value of FileToGenerate will have a corresponding value in ProtoFile. + // + // If SourceFileDescriptors is not empty: + // // - Each FileDescriptorProto in SourceFileDescriptors will have a valid path as the name field. // - The values of FileToGenerate will have a 1-1 mapping to the names in SourceFileDescriptors. //