Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add s3tables catalog #807

Merged
merged 16 commits into from
Dec 27, 2024
Merged

feat: add s3tables catalog #807

merged 16 commits into from
Dec 27, 2024

Conversation

flaneur2020
Copy link
Contributor

@flaneur2020 flaneur2020 commented Dec 16, 2024

fixes #754

this PR adds the implementation of s3tables catalog. i've tested the CRUD of namespaces/tables in my local laptop with a real s3tables bucket.

need to add a mocked test suites in ci in another pr.

@flaneur2020 flaneur2020 marked this pull request as draft December 16, 2024 14:54
@Xuanwo
Copy link
Member

Xuanwo commented Dec 20, 2024

Hi @flaneur2020, I suggest splitting this PR into multiple ones to make it easier to review and accelerate the iteration speed.

@flaneur2020
Copy link
Contributor Author

flaneur2020 commented Dec 20, 2024

@Xuanwo i believe the missing part of this pr is adding tests, i've created a real s3tables bucket to test it and it looks work fine, can you give some suggestions about the test part?

i found the glue catalog is using a mock service from moto, but the mock service for s3tables still not available in moto yet.

@flaneur2020 flaneur2020 marked this pull request as ready for review December 20, 2024 10:04
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this PR, really great! Only some small suggestions.

crates/catalog/s3tables/src/catalog.rs Show resolved Hide resolved
crates/catalog/s3tables/src/catalog.rs Show resolved Hide resolved
crates/catalog/s3tables/src/catalog.rs Show resolved Hide resolved
@flaneur2020 flaneur2020 requested a review from Xuanwo December 27, 2024 10:55
@flaneur2020
Copy link
Contributor Author

@Xuanwo added comments on all the public apis, PTAL

Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice work, thank you!

@Xuanwo Xuanwo merged commit 9f286a0 into apache:main Dec 27, 2024
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add s3tables catalog support
2 participants