Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Block Enqueue race condition while waiting for latest block to be written #494

Merged
merged 1 commit into from
Oct 10, 2023

Conversation

pharr117
Copy link
Collaborator

@pharr117 pharr117 commented Oct 9, 2023

Force block enqueue to fully wait for block to be available by moving to < latestBlock check.

#487 RCA:

I was able to reproduce the issue from #487 in the following manner:

  1. Setup a local node using Osmosis' LocalOsmosis localnet Docker setup
  2. Use the following configs (which define a really fast set of RPC workers):
    • start-block = 1
    • end-block = -1
    • wait-for-chain = false
    • throttling = 0
    • rpc-workers = 4
  3. Let the local indexer catch up to the LatestHeight

I was able to reliably reproduce the state height loading error:

image

I think this is happening for the following reason: The Node is reporting LatestHeight from the Status RPC request before the height is fully available at the GetTxsEvent. This leads fast indexer configs to catch up to the chain really quickly and then requesting the Txs at the latest height way too fast.

This solution

This solution forces the block_enqueue to stay 1 block behind the value reported by latest height. This will prevent ever requesting transactions at a block height that is unavailable.

Closes #487

@pharr117 pharr117 requested a review from danbryan October 9, 2023 20:51
@danbryan danbryan merged commit 827a37d into main Oct 10, 2023
3 checks passed
@danbryan danbryan deleted the patch/487-block-enqueue-race-conditions branch October 10, 2023 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Block enqueue method is passing blocks that are not available
2 participants