Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: More frequent updates to iceberg version and other dependencies #10079

Open
fuzing opened this issue Dec 11, 2024 · 4 comments
Open

Comments

@fuzing
Copy link

fuzing commented Dec 11, 2024

Description

It looks like the latest builds of nessie are using iceberg 1.5.0 which is now 9 months old (current version 1.7.1). Is it possible that the project could update dependencies on a more frequent basis.
Thanks for a great project!

Expected Use Cases

Any/All

Requested Changes in public API

N/A

@dimas-b
Copy link
Member

dimas-b commented Dec 11, 2024

@fuzing : why is it a concern?

AFAIK, Nessie's behaviour should not be affected by the version of its Iceberg dependencies.

@fuzing
Copy link
Author

fuzing commented Dec 11, 2024

Hi @dimas-b -

By way of one example, Nessie supports the iceberg REST catalog implementation. By extension my presumption is that Nessie (through its Iceberg jar dependency) handles atomic metadata/snapshot file updates along with a whole host of other functionality (e.g. the GC, Admin and CLI tools for various ops that may depend on some of the iceberg functionality). From V1.5.0 through V1.7.1 there have been numerous bug fixes, performance improvements and several enhancements that some folks might need (enumerated here) - e.g. new table data types etc.

If I'm mistaken and the iceberg dependency is only used for extrinsic functions (e.g. unit testing) then I apologize.

While this ticket relates to a specific upgrade of a specific dependency (i.e. 1.5.0 to 1.7.1 of iceberg version), it would be helpful to also understand what general policies the Nessie maintainers have in place to support dependency upgrades. This matters a LOT for anyone concerned about timely patches for such things as CVEs (etc) and new features (e.g. the upcoming table V3 releases [unknown timeframe and deps]). Clearly, if the Nessie maintainers aren't committed to some form of dependency update schedule then this may factor into the decision to use/not-use Nessie for some organizations. It would just be great to know if there's a clear set of policies relating to this, so that users can make informed decisions as to the suitability of the project.

Again - love the project, but would like a little more information about the frequency of dependency upgrades.

@snazy
Copy link
Member

snazy commented Dec 12, 2024

@fuzing dependencies are always kept up to date automatically - the recent Nessie releases have/had Apache Iceberg 1.7.1/1.7.0/1.6.x/...

@dimas-b
Copy link
Member

dimas-b commented Dec 13, 2024

Iceberg artifacts are mostly not used in the REST API services runtime. While Nessie supports the REST API spec of Iceberg 1.7.0 (if I'm not mistaken), this support cannot be assumed or derived from the Iceberg artifact versions.

@fuzing : Do you have any suggestions for what would be usable from your POV for declaring API compatibility with Iceberg REST?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants