Add an intermediate representation for the new compiler #419
Conversation
| "github.com/bufbuild/protocompile/internal/iter" | ||
| ) | ||
|
|
||
| var _ iter.Seq[int] // Mark iter as used. |
There was a problem hiding this comment.
I know this is the simplest way to do this. But it would be nice to just emit only the imports that are actually needed. You could emit:
import (
"fmt"And then loop through methods and if/when you see "all" emit
"github.com/bufbuild/protocompile/internal/iter"
And emit the closing ) after that.
Another option might be not emit this line but still unconditionally emit the imports, and then always run make lintfix after go generate in the Makefile, which should fix up formatting, including organizing imports.
But I think just emitting the correct imports is fine for now.
Another option is, if the All method isn't used in very many places, just make them hand-written instead of generated.
There was a problem hiding this comment.
I don't think it's really worth the complexity to get the imports right. No one is going to read this file (or should, at least).
experimental/ir/presence/kind.go
Outdated
|
|
||
| // The field is singular and presence is equivalent to the field having its | ||
| // zero value. This corresponds to a non-message field not marked "optional" | ||
| // in proto3. |
There was a problem hiding this comment.
Yes, but this comment is mostly meant to be giving an example, not be exhaustive.
| // Type is the element type of this value. For arrays, this is the type of the | ||
| // element; Protobuf does not have a concept of an "array type". |
There was a problem hiding this comment.
It may behoove us to provide an "array" type, even though Protobuf descriptors do not. That could enable simpler code that inspects and works with types, to treat them more uniformly. Though I think doing so requires adding another byte to Type, to put an isArray flag there 🤷 (which may not be desirable since, for alignment, it means likely adding 4 or 8 bytes to the Type struct in practice...).
There was a problem hiding this comment.
I am slightly skeptical of that. My gut instinct would be to not add that for now, and see how painful it makes things.
experimental/ir/option.go
Outdated
| // Unknown means this is a message type. | ||
| // Enum types are predeclared.Int32. | ||
| ty predeclared.Name |
There was a problem hiding this comment.
These seems unfortunate, that you can't inspect the enum type of an enum value. Maybe instead encode enums/messages/groups as negative values here and map to a separate enum for NamedType?
There was a problem hiding this comment.
Well, one thing that you could do is say that predeclared.Unknown represents "is a user type", and determining whether it's a message or enum requires looking at the type.
experimental/ir/option.go
Outdated
| // 4. Repeated fields. This holds an arena.Pointer[[]rawValueBits], where each | ||
| // value is interpreted as a non-array with this value's type. | ||
| // This exploits the fact that arrays cannot contain other arrays. |
There was a problem hiding this comment.
Possibly worth noting that map values will be represented as repeated message fields, where each message is a key+value entry.
| } | ||
|
|
||
| type rawMessageValue struct { | ||
| ty ref[rawType] |
There was a problem hiding this comment.
Per above comment: how about typ (here and throughout)?
|
|
||
| type rawMessageValueEntry struct { | ||
| key intern.ID | ||
| field int32 // Index of the field within rawMessageValue.ty; -1 if unknown. |
There was a problem hiding this comment.
We need more here to handle extensions. I think you could instead include a ref[rawField] here, instead of this field, and then interpret its ref.file field as an index into fields of rawMessageValue.typ if it is negative but not -1. (And in such cases, ref.ptr is ignored.)
There was a problem hiding this comment.
I think the representation for an expanded Any will be kind of funky, but there's really no avoiding that, and I think this can handle that correctly. We just have to resolve a type URL "field name" to a ref[rawType] and associate that with the field's value.
There was a problem hiding this comment.
Yeah, I think that for any we need to have both a ref[rawField] and ref[rawType]. Which is unfortunate, since most messages will only contain an int32.
Perhaps we'll wind up having different kinds of entries... we'll see. I'm kind of trying to punt on extensions in message literals for now, since it's a niche feature and I want to prioritize getting things working first.
This PR is the first draft of the intermediate representation for symbol resolution, type inference (of option types) and eventually, generation of
FileDescriptorProtos.This PR is essentially just data structure boilerplate: it does not contain everything (for example, there is no support for services yet) nor does it include code for constructing from an AST or an imported FDP. The purpose of this PR is to get the first draft checked in so that followup PRs can iterate on the API as symbol resolution, type checking, and option resolution are implemented.