Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add callback functions into crud #46

Open
robcxyz opened this issue Sep 28, 2022 · 0 comments
Open

Add callback functions into crud #46

robcxyz opened this issue Sep 28, 2022 · 0 comments

Comments

@robcxyz
Copy link
Contributor

robcxyz commented Sep 28, 2022

Would be really nice to have the ability to configure a callback for a given crud object. In this way we can then configure de-duplication of primary keys as a configurable property of the crud object.

  • Right now we are only doing upserts but this will later be configurable within the crud struct
  • Right now we only have one topic that needs ack so it is simple but future of crud package will have numerous

Order of operations

  • Should be able to run the callback before / after the Tx
    • TODO: Look into patterns with channels to receive callback from db tx
      • Almost certainly we need to mod the channel input but worth investigating patterns

Uses

  • dedupe
  • ack callback
    • Would need to also include the offset / partition / topic in the message
    • Ack needs to happen after a successful transaction with DB
      • Would require creating a new channel to be able to handle this struct -> needs to have the offset data
      • or
      • Look into reading msg before ack into some kind of buffer
  • Enrichment post dedup
    • TODO: List additional use cases / patterns where this is needed -> can think of several but should be formalized
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant