Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add schema serde utilities #73

Merged
merged 9 commits into from
Jul 9, 2024
Merged

Add schema serde utilities #73

merged 9 commits into from
Jul 9, 2024

Conversation

lovromazgon
Copy link
Member

@lovromazgon lovromazgon commented Jul 5, 2024

Description

Adds serde (serialization/deserialization) utilities for schemas. Most of the code already existed in conduit and was moved here. Notable changes:

  • Introduced the field id on the schema (protobuf and Go type), as it's needed in Conduit to create the Confluent wire format, which has the ID prepended in the header.
    • Note that this is a breaking change for the buf linter, but since that proto file was not released yet we can ignore that.
  • A global cache was added for serde objects. The global cache is used so the schema bytes don't have to be parsed every time we process a new record.
  • The previous schema.AvroBuilder was moved into the avro package and became avro.Builder.
  • This repository contains a stricter linter config, so the code needed to be adjusted in some places (e.g. introduced static error variables, error wrapping, exhaustive switch checks etc.).

Closes #53, closes #51.

Quick checks:

  • I have followed the Code Guidelines.
  • There is no other pull request for the same update/change.
  • I have written unit tests.
  • I have made sure that the PR is of reasonable size and can be easily reviewed.

@lovromazgon lovromazgon marked this pull request as ready for review July 8, 2024 12:30
@lovromazgon lovromazgon requested a review from a team as a code owner July 8, 2024 12:30
schema/avro/union.go Outdated Show resolved Hide resolved
schema/avro/serde.go Outdated Show resolved Hide resolved
schema/schema.go Outdated Show resolved Hide resolved
Copy link
Contributor

@hariso hariso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@lovromazgon lovromazgon enabled auto-merge (squash) July 9, 2024 14:21
@lovromazgon lovromazgon merged commit d973cba into main Jul 9, 2024
4 of 5 checks passed
@lovromazgon lovromazgon deleted the lovro/schema-serde branch July 9, 2024 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Schemas] Encode/decode an object using a schema [Schemas] Implement schema auto-generation
2 participants