-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-36760: [Go] Adding avro ocf reader - schema converter #36796
GH-36760: [Go] Adding avro ocf reader - schema converter #36796
Conversation
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a few nitpick questions, otherwise this looks good to me
to match apacheGH-36760: [Go] Adding avro ocf reader - schema converter apache#36796
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM i hope rebasing this into the reader PR isn't gonna be a nightmare for you, haha
@loicalleyne looks like you've got some build/test failures. Other than fixing those issues, everything else looks good to me |
github.com/hamba/avro/v2 requires go 1.19 so the builds for go 1.17 and go 1.18 will always fail. How many go releases worth of backwards compatibility does arrow need to maintain? |
to match apacheGH-36760: [Go] Adding avro ocf reader - schema converter apache#36796
@loicalleyne Hmmm, i didn't realize that. I think i'm fine with us bumping the minimum to go1.19 and dropping the go1.17/go1.18 builds. I'll put together a PR for that. Let's see if we can dig in and figure out the bug in the |
@loicalleyne should this be dropped/closed in favor of #37115 ? It seems like this hasn't been updated or rebased |
@zeroshade it's up to you, I submitted the schema converter and the reader in separate PRs to keep the scope small. |
let's close this one and just use #37115 to track the whole thing. Please rebase that one and have a look at my comments there and close this one. It's likely easier to just keep it all in there than go back and forth between these and have you having to maintain both. |
ae1e08a
to
3697bcd
Compare
to match apacheGH-36760: [Go] Adding avro ocf reader - schema converter apache#36796
to match apacheGH-36760: [Go] Adding avro ocf reader - schema converter apache#36796
Rationale for this change
First step in implementing an Avro reader.
What changes are included in this PR?
Added Avro schema to Arrow schema converter.
Are these changes tested?
yes
Are there any user-facing changes?
One exported functions:
ArrowSchemaFromAvro returns a new Arrow schema from an Avro schema JSON.
ArrowSchemaFromAvro(avroSchema []byte, includeTopLevel bool) (*arrow.Schema, error)