Add an --until option to stop syncing at a certain chain point. #187
Conversation
I would have naturally opted for an inclusive range although in practice it doesn't really matter as we are on a discrete sequence of event here. So we can always just take the next one or previous one. Still, I would go for inclusive because other similar parameters such as |
There was a problem hiding this comment.
Nothing major except some minor tweaks. I'll see to add an integration test (actually even have one already that would benefit from this 😅 ...) and cut a release.
Thanks @Quantumplation
Sometimes it is useful to sync only up to a specific point. For example, suppose you want to get a snapshot of the wallet balances for some token at a single point in time, such as to tally an on-chain vote or poll. You could index without --prune-utxo, but this may be a very expensive syncing operation. However, if you index *with* --prune-utxo, then as soon as kupo passes the snapshot point, you run the risk of pruning UTxOs that were spent after the snapshot, which would distort the snapshot you obtain. This commit adds an --until option. Kupo will index as normal, until it reaches the specified point, after which it will stop consuming blocks and continue to answer http queries about the data. To do this, I pulled the `rollForward` function out, so the consumer could be given some behavior, and then provided two options: - rollForward, which ehaves as normal - rollForwardUntil, which executes rollForward on all blocks before the point Note: `--until` is currently exclusive: it will roll *up to* but *not include* the point you specify. It is unclear what the most intuitive semantics should be, so I'm happy to change this. It doesn't particularly matter for my use case.
I would have naturally opted for an inclusive range although in practice it doesn't really matter as we are on a discrete sequence of event here. So we can always just take the next one or previous one. Still, an inclusive seems more appropriate because other similar parameters such as created_after, created_before, spend_after and spend_before are already inclusive. So to avoid surprises, it seems better to align.
It's more convenient for end-user to provide a slot; although technically less precise if one if after a precise point. For old enough slots, it doesn't matter; but for recent points, it can be necessary to also control the header hash in case the node is following a fork.
The dishes are clean and I helped. |
Add an '--until' option
Sometimes it is useful to sync only up to a specific point. For example, suppose you want to get a snapshot of the wallet balances for some token at a single point in time, such as to tally an on-chain vote or poll.
You could index without --prune-utxo, but this may be a very expensive syncing operation.
However, if you index with --prune-utxo, then as soon as kupo passes the snapshot point, you run the risk of pruning UTxOs that were spent after the snapshot, which would distort the snapshot you obtain.
This commit adds an --until option.
Kupo will index as normal, until it reaches the specified point, after which it will stop consuming blocks and continue to answer http queries about the data.
To do this, I pulled the
rollForwardfunction out, so the consumer could be given some behavior, and then provided two options:Note:
--untilis currently exclusive: it will roll up to but not include the point you specify. It is unclear what the most intuitive semantics should be, so I'm happy to change this. It doesn't particularly matter for my use case.Also, this is basically the extent of my ability to do Haskell, so apologies that I didn't spend time to write a test.
I also wasn't able to test it in practice, because it's now 4:30 am and I need to get some sleep 😅
I'll report back in the next day or two with a real world test, but I wanted to get the PR open now :)