-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: consistent pipeline table names and schemas #587
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, a couple of small questions.
With this change we use bytes instead of string for the state column. Casting to a string is easy and straightforward.
pipeline/DESIGN.md
Outdated
graph LR; | ||
raw_events; | ||
streams; | ||
chain_proofs; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not sure about this name for this table.
First, is the table actually going to include the proofs? Or just the timestamp conclusions from the proofs? The actual proofs are in the raw_events
table right? And then wouldn't this table be derived from raw_events
? Or do you actually plan on splitting the raw data for time events to be in a different table than init and data events?
Also I kind of prefer time_proofs
to chain_proofs
, but that's a minor preference.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The intent of this table is to store the ChainInclusionProof structs. This way the conclusion_events table can be built by joining the raw_events table with this table to make time based conclusions. Should we call it chain_inclusion_proofs? It is not the proof contained within the raw events but rather the timestamp and root cid that we know are included in a chain.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah that struct is kind of named weird cause it's not actually storing the proof. It's storing the derived conclusions that come from validating the proof.
How about... time_conclusions
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Works for me.
Co-authored-by: Spencer T Brody <[email protected]>
Co-authored-by: Spencer T Brody <[email protected]>
…-ceramic into docs/pipeline-design
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a few small comments but otherwise LGTM
|
||
| name | code | description | | ||
| ---- | ---- | ----------- | | ||
| Data | 0x00 | An event containing data for the stream | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this "Event Types" table only meant to apply to conclusion events? It's not nested under the conclusion events section of this document. This document also describes the raw events table as well, and in that table init events do have their own event
Co-authored-by: Spencer T Brody <[email protected]>
Co-authored-by: Spencer T Brody <[email protected]>
Co-authored-by: Spencer T Brody <[email protected]>
Co-authored-by: Spencer T Brody <[email protected]>
…-ceramic into docs/pipeline-design
No description provided.