Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE]: Provide option to write output and quarantine data to tables as defined in the config #167

Open
1 task done
mwojtyczka opened this issue Feb 12, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@mwojtyczka
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Problem statement

We are currently not utilizing the output_table and quarantine_table specified in the config, to actually write any data out. It's up to the user to persist the data after quality checking. We only use quarantine_table in the dashboard but it's up to the user to write the data out.

Note: This is only relevant if DQX is installed in the workspace and not used purely as a library.

Proposed Solution

Provide an option to write the data out to tables as per output_table and quarantine_table field in the config after quality checking.

Additional Context

No response

@mwojtyczka mwojtyczka added the enhancement New feature or request label Feb 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant