Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure ingest record batch type and partition file writer type against table #1799

Closed
patchwork01 opened this issue Feb 6, 2024 · 1 comment
Labels
duplicate This issue or pull request already exists enhancement New feature or request

Comments

@patchwork01
Copy link
Collaborator

patchwork01 commented Feb 6, 2024

Background

We have instance properties sleeper.ingest.record.batch.type and sleeper.ingest.partition.file.writer.type. These configure how ingest happens in a standard ingest task.

There's a system test IngestIT which tests we can ingest records with the different batch and writer types in a real standard ingest task.

This was split from the following issue:

Description

We'd like to configure the ingest record batch type and file writer type in table properties.

Analysis

We can use this in the system test IngestIT.

@patchwork01 patchwork01 added the enhancement New feature or request label Feb 6, 2024
@patchwork01
Copy link
Collaborator Author

This was duplicated and completed in:

@patchwork01 patchwork01 added the duplicate This issue or pull request already exists label Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant