Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support upsert for ReplacingMergeTree engine in Clickhouse sink #17270

Closed
hzxa21 opened this issue Jun 17, 2024 · 1 comment
Closed

Support upsert for ReplacingMergeTree engine in Clickhouse sink #17270

hzxa21 opened this issue Jun 17, 2024 · 1 comment
Assignees
Milestone

Comments

@hzxa21
Copy link
Collaborator

hzxa21 commented Jun 17, 2024

Is your feature request related to a problem? Please describe.

From community slack: https://risingwave-community.slack.com/archives/C03BW71523T/p1718428447943059

When using ClickHouse Sink with the ReplacingMergeTree, I think you can support CDC DELETE OPs by leveraging their newer (optional) is_delete addition. Basically, when you're creating the ReplacingMergeTree you can optionally provide a UInt8 column to signify when a record was deleted, and thus will not show up during a query which uses FINAL in its scan. This might be a good tradeoff from their CollapsingMergeTree being a little cumbersome and inefficient performance wise.
From a usability perspective, I think perhaps an optional clickhouse.delete.column (or similar) can be provided for ReplacingMergeTree that would allow upsert (or at least propagating DELETE OPs). You'd keep that column set to 0 until you want to record it deleted, in which case you'd set it to 1

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

@github-actions github-actions bot added this to the release-1.10 milestone Jun 17, 2024
@hzxa21
Copy link
Collaborator Author

hzxa21 commented Jun 18, 2024

#17283

@xxhZs xxhZs closed this as completed Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants