We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Validator was working fine and suddenly got this error and can't reconect
v1.0.0-rc.5
6:53PM INF finalized block block_app_hash=22745F10514607098155B7455FD06A7AABDF3991976E71413FC65351AFC41686 height=334799 module=state num_txs_res=85 num_val_updates=0 6:53PM INF executed block app_hash=22745F10514607098155B7455FD06A7AABDF3991976E71413FC65351AFC41686 height=334799 module=state 6:53PM INF committed state block_app_hash=86E7D343111F522EBE9D46B390997EFB3C75F345C4458E5B2F5CB4BC36111487 height=334799 module=state 6:53PM INF indexed block events height=334799 module=txindex 6:53PM INF Timed out dur=4689.863219 height=334800 module=consensus round=0 step=RoundStepNewHeight 6:53PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0 6:53PM INF Timed out dur=3000 height=334800 module=consensus round=0 step=RoundStepPropose 6:53PM INF received proposal module=consensus proposal="Proposal{334800/0 (1EB28D2C376252A632D3CE4B54BBC64CE59FDF7A31711F1F9A36F09B422C9DF7:2:6088828F7549, -1) B763E99DF294 @ 2025-02-19T18:53:49.412865591Z}" proposer=9401159CD6024B55F3B4E2E154D7D7E54D63F0E9 6:53PM INF received complete proposal block hash=1EB28D2C376252A632D3CE4B54BBC64CE59FDF7A31711F1F9A36F09B422C9DF7 height=334800 module=consensus 6:53PM ERR panic recovered in ExtendVote hash=1EB28D2C376252A632D3CE4B54BBC64CE59FDF7A31711F1F9A36F09B422C9DF7 height=334800 module=server panic=null 6:53PM ERR CONSENSUS FAILURE!!! err="ExtendVote call failed: recovered application panic in ExtendVote: failed to get validator address: validator address does not exist with BLS public key ��\rG\x7f�%���G2��NR��f\a��\x17\vOW"�:fˠmOY1���aL��\f��7�\x0fH\x01�o�z�����O\x13��>P����{�2�ǵ\x01P��\x1b~\x1f��\x14�W�\x0f1Ef��u: Validator address does not exist [cosmossdk.io/[email protected]/errors.go:155]" module=consensus stack="goroutine 676 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:26 +0x5e\ngithub.com/cometbft/cometbft/consensus.(*State).receiveRoutine.func2()\n\tgithub.com/cometbft/[email protected]/consensus/state.go:801 +0x46\npanic({0x337a8c0?, 0xc007ec6300?})\n\truntime/panic.go:785 +0x132\ngithub.com/cometbft/cometbft/state.(*BlockExecutor).ExtendVote(, {, _}, _, _, {{{0xb, 0x0}, {0xc00437a2d0, 0x7}}, {0xc00437a2e0, ...}, ...})\n\tgithub.com/cometbft/[email protected]/state/execution.go:351 +0x4f3\ngithub.com/cometbft/cometbft/consensus.(*State).signVote(0xc00157aa88, 0x2, {0xc00b768540, 0x20, 0x20}, {0xc07928?, {0xc00b768560?, 0xc00df2542c?, 0x160?}}, 0xc0010ea000)\n\tgithub.com/cometbft/[email protected]/consensus/state.go:2397 +0x4dd\ngithub.com/cometbft/cometbft/consensus.(*State).signAddVote(0xc00157aa88, 0x2, {0xc00b768540, 0x20, 0x20}, {0x1?, {0xc00b768560?, 0xc008e36b60?, 0x20?}}, 0xc0010ea000)\n\tgithub.com/cometbft/[email protected]/consensus/state.go:2458 +0x215\ngithub.com/cometbft/cometbft/consensus.(*State).enterPrecommit(0xc00157aa88, 0x51bd0, 0x0)\n\tgithub.com/cometbft/[email protected]/consensus/state.go:1537 +0x12d7\ngithub.com/cometbft/cometbft/consensus.(*State).addVote(0xc00157aa88, 0xc007d40d00, {0xc009f5c1e0, 0x28})\n\tgithub.com/cometbft/[email protected]/consensus/state.go:2315 +0x19ef\ngithub.com/cometbft/cometbft/consensus.(*State).tryAddVote(0xc00157aa88, 0xc007d40d00, {0xc009f5c1e0?, 0x0?})\n\tgithub.com/cometbft/[email protected]/consensus/state.go:2067 +0x26\ngithub.com/cometbft/cometbft/consensus.(*State).handleMsg(0xc00157aa88, {{0x51ab160, 0xc008b004c8}, {0xc009f5c1e0, 0x28}})\n\tgithub.com/cometbft/[email protected]/consensus/state.go:929 +0x38b\ngithub.com/cometbft/cometbft/consensus.(*State).receiveRoutine(0xc00157aa88, 0x0)\n\tgithub.com/cometbft/[email protected]/consensus/state.go:836 +0x3f1\ncreated by github.com/cometbft/cometbft/consensus.(*State).OnStart in goroutine 442\n\tgithub.com/cometbft/[email protected]/consensus/state.go:398 +0x10c\n" 6:53PM INF service stop impl=baseWAL module=consensus msg="Stopping baseWAL service" wal=/root/.babylond/data/cs.wal/wal 6:53PM INF service stop impl=Group module=consensus msg="Stopping Group service" wal=/root/.babylond/data/cs.wal/wal 6:53PM INF Timed out dur=1000 height=334800 module=consensus round=0 step=RoundStepPrevoteWait 6:54PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0 6:54PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0 6:55PM ERR Connection failed @ sendRoutine conn=MConn{46.4.91.76:30556} err="pong timeout" module=p2p [email protected]:30556 6:55PM INF service stop impl=MConn{46.4.91.76:30556} module=p2p msg="Stopping MConnection service" [email protected]:30556 6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{46.4.91.76:30556} 5e83cc27be007d8944b5424e2141bca51dd9121a out}" 6:55PM INF service stop impl="Peer{MConn{46.4.91.76:30556} 5e83cc27be007d8944b5424e2141bca51dd9121a out}" module=p2p msg="Stopping Peer service" [email protected]:30556 6:55PM ERR Connection failed @ sendRoutine conn=MConn{168.119.31.198:55706} err="pong timeout" module=p2p [email protected]:55706 6:55PM INF service stop impl=MConn{168.119.31.198:55706} module=p2p msg="Stopping MConnection service" [email protected]:55706 6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{168.119.31.198:55706} 35eb06875d3d99fd387f959f836ff8efc1912f83 out}" 6:55PM INF service stop impl="Peer{MConn{168.119.31.198:55706} 35eb06875d3d99fd387f959f836ff8efc1912f83 out}" module=p2p msg="Stopping Peer service" [email protected]:55706 6:55PM ERR Connection failed @ sendRoutine conn=MConn{5.180.82.73:26656} err="pong timeout" module=p2p [email protected]:26656 6:55PM INF service stop impl=MConn{5.180.82.73:26656} module=p2p msg="Stopping MConnection service" [email protected]:26656 6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{5.180.82.73:26656} dffe8fde153d29169bacc4e11911279540d867d7 out}" 6:55PM INF service stop impl="Peer{MConn{5.180.82.73:26656} dffe8fde153d29169bacc4e11911279540d867d7 out}" module=p2p msg="Stopping Peer service" [email protected]:26656 6:55PM ERR Connection failed @ sendRoutine conn=MConn{212.90.120.2:26656} err="pong timeout" module=p2p [email protected]:26656 6:55PM INF service stop impl=MConn{212.90.120.2:26656} module=p2p msg="Stopping MConnection service" [email protected]:26656 6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{212.90.120.2:26656} 84e8f4412034116e4326d22efcaa2d23bb0c7fa5 out}" 6:55PM INF service stop impl="Peer{MConn{212.90.120.2:26656} 84e8f4412034116e4326d22efcaa2d23bb0c7fa5 out}" module=p2p msg="Stopping Peer service" [email protected]:26656 6:55PM ERR Connection failed @ sendRoutine conn=MConn{165.154.224.218:26656} err="pong timeout" module=p2p [email protected]:26656 6:55PM INF service stop impl=MConn{165.154.224.218:26656} module=p2p msg="Stopping MConnection service" [email protected]:26656 6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{165.154.224.218:26656} fbc9c4415c0086954221fd760f421d8d6a14b535 out}" 6:55PM INF service stop impl="Peer{MConn{165.154.224.218:26656} fbc9c4415c0086954221fd760f421d8d6a14b535 out}" module=p2p msg="Stopping Peer service" [email protected]:26656 6:55PM ERR Connection failed @ sendRoutine conn=MConn{65.109.49.115:16156} err="pong timeout" module=p2p [email protected]:16156 6:55PM INF service stop impl=MConn{65.109.49.115:16156} module=p2p msg="Stopping MConnection service" [email protected]:16156 6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{65.109.49.115:16156} 6b13a5c219a60f652ed6566e7c82db22696541c6 out}" 6:55PM INF service stop impl="Peer{MConn{65.109.49.115:16156} 6b13a5c219a60f652ed6566e7c82db22696541c6 out}" module=p2p msg="Stopping Peer service" [email protected]:16156 6:55PM INF Reconnecting to peer [email protected]:16156 module=p2p 6:55PM INF service start impl="Peer{MConn{65.109.49.115:16156} 6b13a5c219a60f652ed6566e7c82db22696541c6 out}" module=p2p msg="Starting Peer service" [email protected]:16156 6:55PM INF service start impl=MConn{65.109.49.115:16156} module=p2p msg="Starting MConnection service" [email protected]:16156 6:55PM ERR Connection failed @ sendRoutine conn=MConn{51.89.7.79:26656} err="pong timeout" module=p2p [email protected]:26656 6:55PM INF service stop impl=MConn{51.89.7.79:26656} module=p2p msg="Stopping MConnection service" [email protected]:26656 6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{51.89.7.79:26656} dac672e0f4d02c5c43c0ef62500ac00f212c89b6 out}" 6:55PM INF service stop impl="Peer{MConn{51.89.7.79:26656} dac672e0f4d02c5c43c0ef62500ac00f212c89b6 out}" module=p2p msg="Stopping Peer service" [email protected]:26656 6:55PM INF Reconnecting to peer [email protected]:26656 module=p2p 6:55PM INF service start impl="Peer{MConn{51.89.7.79:26656} dac672e0f4d02c5c43c0ef62500ac00f212c89b6 out}" module=p2p msg="Starting Peer service" [email protected]:26656 6:55PM INF service start impl=MConn{51.89.7.79:26656} module=p2p msg="Starting MConnection service" [email protected]:26656 6:55PM ERR Connection failed @ sendRoutine conn=MConn{65.108.230.161:26656} err="pong timeout" module=p2p [email protected]:26656
The text was updated successfully, but these errors were encountered:
Hey, did you follow the correct steps in https://github.com/babylonlabs-io/networks/blob/main/bbn-test-5/upgrades/v1rc5.md?
Sorry, something went wrong.
Yes. Checked twice
No branches or pull requests
Summary of Bug
Validator was working fine and suddenly got this error and can't reconect
Version
v1.0.0-rc.5
Steps to Reproduce
6:53PM INF finalized block block_app_hash=22745F10514607098155B7455FD06A7AABDF3991976E71413FC65351AFC41686 height=334799 module=state num_txs_res=85 num_val_updates=0
6:53PM INF executed block app_hash=22745F10514607098155B7455FD06A7AABDF3991976E71413FC65351AFC41686 height=334799 module=state
6:53PM INF committed state block_app_hash=86E7D343111F522EBE9D46B390997EFB3C75F345C4458E5B2F5CB4BC36111487 height=334799 module=state
6:53PM INF indexed block events height=334799 module=txindex
6:53PM INF Timed out dur=4689.863219 height=334800 module=consensus round=0 step=RoundStepNewHeight
6:53PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
6:53PM INF Timed out dur=3000 height=334800 module=consensus round=0 step=RoundStepPropose
6:53PM INF received proposal module=consensus proposal="Proposal{334800/0 (1EB28D2C376252A632D3CE4B54BBC64CE59FDF7A31711F1F9A36F09B422C9DF7:2:6088828F7549, -1) B763E99DF294 @ 2025-02-19T18:53:49.412865591Z}" proposer=9401159CD6024B55F3B4E2E154D7D7E54D63F0E9
6:53PM INF received complete proposal block hash=1EB28D2C376252A632D3CE4B54BBC64CE59FDF7A31711F1F9A36F09B422C9DF7 height=334800 module=consensus
6:53PM ERR panic recovered in ExtendVote hash=1EB28D2C376252A632D3CE4B54BBC64CE59FDF7A31711F1F9A36F09B422C9DF7 height=334800 module=server panic=null
6:53PM ERR CONSENSUS FAILURE!!! err="ExtendVote call failed: recovered application panic in ExtendVote: failed to get validator address: validator address does not exist with BLS public key ��\rG\x7f�%���G2��NR��f\a��\x17\vOW"�:fˠmOY1���aL��\f��7�\x0fH\x01�o�z�����O\x13��>P����{�2�ǵ\x01P��\x1b~\x1f��\x14�W�\x0f1Ef��u: Validator address does not exist [cosmossdk.io/[email protected]/errors.go:155]" module=consensus stack="goroutine 676 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:26 +0x5e\ngithub.com/cometbft/cometbft/consensus.(*State).receiveRoutine.func2()\n\tgithub.com/cometbft/[email protected]/consensus/state.go:801 +0x46\npanic({0x337a8c0?, 0xc007ec6300?})\n\truntime/panic.go:785 +0x132\ngithub.com/cometbft/cometbft/state.(*BlockExecutor).ExtendVote(, {, _}, _, _, {{{0xb, 0x0}, {0xc00437a2d0, 0x7}}, {0xc00437a2e0, ...}, ...})\n\tgithub.com/cometbft/[email protected]/state/execution.go:351 +0x4f3\ngithub.com/cometbft/cometbft/consensus.(*State).signVote(0xc00157aa88, 0x2, {0xc00b768540, 0x20, 0x20}, {0xc07928?, {0xc00b768560?, 0xc00df2542c?, 0x160?}}, 0xc0010ea000)\n\tgithub.com/cometbft/[email protected]/consensus/state.go:2397 +0x4dd\ngithub.com/cometbft/cometbft/consensus.(*State).signAddVote(0xc00157aa88, 0x2, {0xc00b768540, 0x20, 0x20}, {0x1?, {0xc00b768560?, 0xc008e36b60?, 0x20?}}, 0xc0010ea000)\n\tgithub.com/cometbft/[email protected]/consensus/state.go:2458 +0x215\ngithub.com/cometbft/cometbft/consensus.(*State).enterPrecommit(0xc00157aa88, 0x51bd0, 0x0)\n\tgithub.com/cometbft/[email protected]/consensus/state.go:1537 +0x12d7\ngithub.com/cometbft/cometbft/consensus.(*State).addVote(0xc00157aa88, 0xc007d40d00, {0xc009f5c1e0, 0x28})\n\tgithub.com/cometbft/[email protected]/consensus/state.go:2315 +0x19ef\ngithub.com/cometbft/cometbft/consensus.(*State).tryAddVote(0xc00157aa88, 0xc007d40d00, {0xc009f5c1e0?, 0x0?})\n\tgithub.com/cometbft/[email protected]/consensus/state.go:2067 +0x26\ngithub.com/cometbft/cometbft/consensus.(*State).handleMsg(0xc00157aa88, {{0x51ab160, 0xc008b004c8}, {0xc009f5c1e0, 0x28}})\n\tgithub.com/cometbft/[email protected]/consensus/state.go:929 +0x38b\ngithub.com/cometbft/cometbft/consensus.(*State).receiveRoutine(0xc00157aa88, 0x0)\n\tgithub.com/cometbft/[email protected]/consensus/state.go:836 +0x3f1\ncreated by github.com/cometbft/cometbft/consensus.(*State).OnStart in goroutine 442\n\tgithub.com/cometbft/[email protected]/consensus/state.go:398 +0x10c\n"
6:53PM INF service stop impl=baseWAL module=consensus msg="Stopping baseWAL service" wal=/root/.babylond/data/cs.wal/wal
6:53PM INF service stop impl=Group module=consensus msg="Stopping Group service" wal=/root/.babylond/data/cs.wal/wal
6:53PM INF Timed out dur=1000 height=334800 module=consensus round=0 step=RoundStepPrevoteWait
6:54PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
6:54PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=10 numToDial=0
6:55PM ERR Connection failed @ sendRoutine conn=MConn{46.4.91.76:30556} err="pong timeout" module=p2p [email protected]:30556
6:55PM INF service stop impl=MConn{46.4.91.76:30556} module=p2p msg="Stopping MConnection service" [email protected]:30556
6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{46.4.91.76:30556} 5e83cc27be007d8944b5424e2141bca51dd9121a out}"
6:55PM INF service stop impl="Peer{MConn{46.4.91.76:30556} 5e83cc27be007d8944b5424e2141bca51dd9121a out}" module=p2p msg="Stopping Peer service" [email protected]:30556
6:55PM ERR Connection failed @ sendRoutine conn=MConn{168.119.31.198:55706} err="pong timeout" module=p2p [email protected]:55706
6:55PM INF service stop impl=MConn{168.119.31.198:55706} module=p2p msg="Stopping MConnection service" [email protected]:55706
6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{168.119.31.198:55706} 35eb06875d3d99fd387f959f836ff8efc1912f83 out}"
6:55PM INF service stop impl="Peer{MConn{168.119.31.198:55706} 35eb06875d3d99fd387f959f836ff8efc1912f83 out}" module=p2p msg="Stopping Peer service" [email protected]:55706
6:55PM ERR Connection failed @ sendRoutine conn=MConn{5.180.82.73:26656} err="pong timeout" module=p2p [email protected]:26656
6:55PM INF service stop impl=MConn{5.180.82.73:26656} module=p2p msg="Stopping MConnection service" [email protected]:26656
6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{5.180.82.73:26656} dffe8fde153d29169bacc4e11911279540d867d7 out}"
6:55PM INF service stop impl="Peer{MConn{5.180.82.73:26656} dffe8fde153d29169bacc4e11911279540d867d7 out}" module=p2p msg="Stopping Peer service" [email protected]:26656
6:55PM ERR Connection failed @ sendRoutine conn=MConn{212.90.120.2:26656} err="pong timeout" module=p2p [email protected]:26656
6:55PM INF service stop impl=MConn{212.90.120.2:26656} module=p2p msg="Stopping MConnection service" [email protected]:26656
6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{212.90.120.2:26656} 84e8f4412034116e4326d22efcaa2d23bb0c7fa5 out}"
6:55PM INF service stop impl="Peer{MConn{212.90.120.2:26656} 84e8f4412034116e4326d22efcaa2d23bb0c7fa5 out}" module=p2p msg="Stopping Peer service" [email protected]:26656
6:55PM ERR Connection failed @ sendRoutine conn=MConn{165.154.224.218:26656} err="pong timeout" module=p2p [email protected]:26656
6:55PM INF service stop impl=MConn{165.154.224.218:26656} module=p2p msg="Stopping MConnection service" [email protected]:26656
6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{165.154.224.218:26656} fbc9c4415c0086954221fd760f421d8d6a14b535 out}"
6:55PM INF service stop impl="Peer{MConn{165.154.224.218:26656} fbc9c4415c0086954221fd760f421d8d6a14b535 out}" module=p2p msg="Stopping Peer service" [email protected]:26656
6:55PM ERR Connection failed @ sendRoutine conn=MConn{65.109.49.115:16156} err="pong timeout" module=p2p [email protected]:16156
6:55PM INF service stop impl=MConn{65.109.49.115:16156} module=p2p msg="Stopping MConnection service" [email protected]:16156
6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{65.109.49.115:16156} 6b13a5c219a60f652ed6566e7c82db22696541c6 out}"
6:55PM INF service stop impl="Peer{MConn{65.109.49.115:16156} 6b13a5c219a60f652ed6566e7c82db22696541c6 out}" module=p2p msg="Stopping Peer service" [email protected]:16156
6:55PM INF Reconnecting to peer [email protected]:16156 module=p2p
6:55PM INF service start impl="Peer{MConn{65.109.49.115:16156} 6b13a5c219a60f652ed6566e7c82db22696541c6 out}" module=p2p msg="Starting Peer service" [email protected]:16156
6:55PM INF service start impl=MConn{65.109.49.115:16156} module=p2p msg="Starting MConnection service" [email protected]:16156
6:55PM ERR Connection failed @ sendRoutine conn=MConn{51.89.7.79:26656} err="pong timeout" module=p2p [email protected]:26656
6:55PM INF service stop impl=MConn{51.89.7.79:26656} module=p2p msg="Stopping MConnection service" [email protected]:26656
6:55PM ERR Stopping peer for error err="pong timeout" module=p2p peer="Peer{MConn{51.89.7.79:26656} dac672e0f4d02c5c43c0ef62500ac00f212c89b6 out}"
6:55PM INF service stop impl="Peer{MConn{51.89.7.79:26656} dac672e0f4d02c5c43c0ef62500ac00f212c89b6 out}" module=p2p msg="Stopping Peer service" [email protected]:26656
6:55PM INF Reconnecting to peer [email protected]:26656 module=p2p
6:55PM INF service start impl="Peer{MConn{51.89.7.79:26656} dac672e0f4d02c5c43c0ef62500ac00f212c89b6 out}" module=p2p msg="Starting Peer service" [email protected]:26656
6:55PM INF service start impl=MConn{51.89.7.79:26656} module=p2p msg="Starting MConnection service" [email protected]:26656
6:55PM ERR Connection failed @ sendRoutine conn=MConn{65.108.230.161:26656} err="pong timeout" module=p2p [email protected]:26656
The text was updated successfully, but these errors were encountered: