Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the Java TPCDS connector generate maintenance data sets #23135

Open
yingsu00 opened this issue Jul 4, 2024 · 1 comment
Open

Make the Java TPCDS connector generate maintenance data sets #23135

yingsu00 opened this issue Jul 4, 2024 · 1 comment
Assignees
Labels
feature request iceberg Apache Iceberg related

Comments

@yingsu00
Copy link
Contributor

yingsu00 commented Jul 4, 2024

The current TPCDS connector only generates base tables. It needs to generate maintenance data sets. The dsdgen option for this is --update.

First we need to find out how to add the new tables for these. Then we need to select the right rows for TINY scale factor, since the dsdgen can only generate 1GB as the minimum SF. For that we will need to see how the TINY SF were chosen on the base tables, then apply the same strategy to the maintenance tables.

@yingsu00
Copy link
Contributor Author

yingsu00 commented Jul 4, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request iceberg Apache Iceberg related
Projects
Status: 🆕 Unprioritized
Development

No branches or pull requests

2 participants