You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Dec 20, 2024. It is now read-only.
After PR #71, which fixed the issue with the backend database change, it seems to me that we might sync data with signficant delay, as the syncing, I believe, now depends on both prod and staging having at least one settlement every few hours. The reason for that, I think, is the min operator in this line:
I would change it to max and remove the - REORG_THRESHOLD.
With the new query we look at blocks which have already successfully been processed (since these is an entry in settlement_observations). What can then go wrong is that maybe there are staging and production settlements on adjacent blocks and the later settlement is already processed but not the earlier one. Then the earlier one will not end up on dune. Not sure if that is actually happening sometimes. For that purpose we could have some delay, so something like - MAX_PROCESSING_DELAY with MAX_PROCESSING_DELAY = 1 or 2 or whatever seems fit. The old query had a delay of 65 which might included implicitly a MAX_PROCESSING_DELAY = 1 since the reorg threshold should be 64.
This PR addresses issue #72. At the moment the block range is computed
using the minimum of the latest finalized block from production and
staging and subtracting `REORG_THRESHOLD = 65`. This was done as a hot fix in #71.
One problem with this approach is that having few settlements in staging
blocks upload of production data to dune.
This PR changes the behavior to look at the maximum block of the latest
finalized blocks from production and staging subtracting
`MAX_PROCESSING_DELAY = 1`. This should leave enough time for the autopilot
to process settlements after finalization in staging and production.
After PR #71, which fixed the issue with the backend database change, it seems to me that we might sync data with signficant delay, as the syncing, I believe, now depends on both prod and staging having at least one settlement every few hours. The reason for that, I think, is the
min
operator in this line:dune-sync/src/fetch/orderbook.py
Line 72 in d6f34e0
I would propose that we actually change it to
max
now. Let me know what you think @fhennekeThe text was updated successfully, but these errors were encountered: