Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write a completely custom JSON decoder/encoder #1262

Closed
Tracked by #1248
alecthomas opened this issue Apr 15, 2024 · 6 comments · Fixed by #1441
Closed
Tracked by #1248

Write a completely custom JSON decoder/encoder #1262

alecthomas opened this issue Apr 15, 2024 · 6 comments · Fixed by #1441
Assignees
Labels
intermediate A task requiring an intermediate level of knowledge

Comments

@alecthomas
Copy link
Collaborator

Currently we delegate to the encoding/json package under certain circumstances, which can do the wrong thing. We should also add support for ,omitempty including for any zero value (not just nil, as with the stdlib).

@github-actions github-actions bot added the triage Issue needs triaging label Apr 15, 2024
@alecthomas alecthomas added intermediate A task requiring an intermediate level of knowledge and removed triage Issue needs triaging labels Apr 15, 2024
deniseli added a commit that referenced this issue May 7, 2024
This PR:
* Rips out the existing textMarshaler and jsonMarshaler usage from
encoding.go. We may want to add those back for
#1296 down the road, but we
will need to be thoughtful about how we do that. Removing it for now
keeps the logic much more predictable.
* Moves the (un)marshaling logic for `ftl.Option` out of `option.go` and
into `encoding.go`.
* Special-cases both `time.Time` (the only stdlib type we currently
support) and `ftl.Option`. Also `json.RawMessage` for _just_ encoding to
preserve the existing `omitempty` behavior.
* Fixes some existing issues where the Pointer unmarshaling wasn't
actually working correctly
* [eww] Adds a rather grotesque alternative to `Peek()` in
`isNextTokenNull()` because json Decoder does not support Peek.
* [eww] Makes the ftl.Option struct fields public so that they are
settable by `encoding.go`.

Suggestions welcome for both counts of [eww] above :)

Fixes #1247.
Addresses most of #1262, except
`omitempty` is only working for json.RawMessage for now.
@deniseli
Copy link
Contributor

deniseli commented May 7, 2024

The work left to do on this ticket is the remaining omitempty support for encodeValue().

@alecthomas
Copy link
Collaborator Author

I can't recall if this was specifically referring to ingress, or to the backend encoder, or both, but it feels like we could/should use it for both.

@deniseli
Copy link
Contributor

deniseli commented May 7, 2024

Now that we've taken out the json marshaler logic in the custom encoder, we are nearly doing that already. That said, referring back to the title, the fact that we're still using json.Decoder means we don't really have a "completely custom JSON decoder/encoder". I added #1432 to track the idea of replacing json.Decoder with a better, custom implementation. So I think we can close this issue after the omitempty handling is done (which I'm working on now)... but please let me know if we missed anything

@alecthomas
Copy link
Collaborator Author

I think I got a bit mixed up before - we should separate the two concerns. The ingress JSON encoding/decoding shouldn't be tightly coupled to the backend encoding package. I think we'll want completely separate encoders/decoders for both.

@deniseli
Copy link
Contributor

deniseli commented May 8, 2024

Ahhh got it, thanks for clarifying! I'll make a note to touch base with Lizzy on this when she's back. In the meantime, I did a quick review of #1432 and that doesn't talk about where the encoder is used, except that it's defined in go-runtime, so I think we're ok. Please let me know if I missed something, though - there were a lot of loose ends on this one

@deniseli
Copy link
Contributor

deniseli commented May 8, 2024

I also just added #1439 to track the removal of json.RawMessage so that we can remove the special case. So I believe all the outstanding work related to this ticket is now tracked between #1439 and #1432.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
intermediate A task requiring an intermediate level of knowledge
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants