在 mainnet-beta 环境异常退出

[249198000]

dmesg 输出以下错误
[10263.326006] solBstoreProc02[15654]: segfault at 1 ip 00007ff829683dcd sp 00007f9e03988368 error 4 in libc.so.6[7ff8294fd000+195000]
[10263.326025] Code: 00 00 00 00 00 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 f3 0f 1e fa 48 89 f8 48 83 fa 20 72 33 <62> e1 fe 28 6f 06 48 83 fa 40 0f 87 a3 00 00 00 62 e1 fe 28 6f 4c

gdb 输出的调用堆栈
#0 __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:317
#1 0x00007f0106986542 in core::intrinsics::copy_nonoverlapping (src=, dst=,
count=) at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/intrinsics.rs:2702
#2 core::ptr::const_ptr::{impl#0}::copy_to_nonoverlapping (self=, dest=,
count=)
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ptr/const_ptr.rs:1316
#3 alloc::slice::hack::{impl#1}::to_vec<u8, alloc::alloc::Global> (s=..., alloc=...)
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/slice.rs:167
#4 0x00007f010667140c in alloc::slice::hack::to_vec<u8, alloc::alloc::Global> (s=...)
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/slice.rs:111
#5 alloc::slice::{impl#0}::to_vec_in<u8, alloc::alloc::Global> (self=...)
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/slice.rs:441
#6 alloc::slice::{impl#0}::to_vec (self=...)
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/slice.rs:416
#7 0x00007f0105f59edc in solana_geyser_plugin_postgres::postgres_client::{impl#1}::new::{closure#0}<solana_geyser_plugin_interface::geyser_plugin_interface::ReplicaAccountInfoV3> (v=...) at src/postgres_client.rs:122
#8 0x00007f0105f42fe2 in core::option::Option<&[u8]>::map<&[u8], alloc::vec::Vec<u8, alloc::alloc::Global>, solana_geyser_plugin_postgres::postgres_client::{impl#1}::new::{closure_env#0}<solana_geyser_plugin_interface::geyser_plugin_interface::ReplicaAccountInfoV3>> (self=..., f=...)
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/option.rs:1072
#9 0x00007f0105f59db2 in solana_geyser_plugin_postgres::postgres_client::DbAccountInfo::new<solana_geyser_plugin_interface::geyser_plugin_interface::ReplicaAccountInfoV3> (account=0x7ea70087be90, slot=255435149)
at src/postgres_client.rs:122
#10 0x00007f0105f74aa1 in solana_geyser_plugin_postgres::postgres_client::ParallelPostgresClient::update_account
(self=0x7f014ea2b1d0, account=0x7ea70087be90, slot=255435149, is_startup=false)
at src/postgres_client.rs:1176
#11 0x00007f0105f4b74a in solana_geyser_plugin_postgres::geyser_plugin_postgres::{impl#1}::update_account (
self=0x7f014ea2b1c0, account=..., slot=255435149, is_startup=false) at src/geyser_plugin_postgres.rs:283

错误都是发生在snapshot restore完成之后，
[2024-03-22T12:14:17.904657011Z INFO solana_metrics::metrics] datapoint: geyser_plugin_notify_account_restore_fro
m_snapshot_summary flush_slots-us=22882884i flush-slots-counts=47102i
[2024-03-22T12:14:17.995162658Z INFO solana_geyser_plugin_postgres::postgres_client] Done with notifying the end
of startup
再过几秒钟之后就直接异常退出了
plugin使用的debug版本
命令行：
solana-validator
--identity /root/sol/validator-keypair.json
--trusted-validator 7Np41oeYqPefeNQEHSv1UDhYrehxin3NStELsSKCT4K2
--trusted-validator GdnSyH3YtwcxFvQrVVJMm1JhTS4QVX7MFsX56uJLUfiZ
--trusted-validator DE1bawNcRJB9rVm3buyMVfr8mBEoyyu73NBovf2oXJsJ
--trusted-validator CakcnaRDHka2gXyfbEd2d3xsvkJkqsLw2akB3zsN1D2S
--no-untrusted-rpc
--ledger /mnt/ledger
--accounts /mnt/accounts
--rpc-port 8899
--private-rpc
--no-port-check
--no-voting
--dynamic-port-range 8000-8110
--entrypoint entrypoint.mainnet-beta.solana.com:8001
--entrypoint entrypoint2.mainnet-beta.solana.com:8001
--entrypoint entrypoint3.mainnet-beta.solana.com:8001
--entrypoint entrypoint4.mainnet-beta.solana.com:8001
--entrypoint entrypoint5.mainnet-beta.solana.com:8001
--expected-genesis-hash 5eykt4UsFv8P8NJdTREpY1vzqKqZKvdpKuc147dw2N9d
--wal-recovery-mode skip_any_corrupted_record
--limit-ledger-size 536870912000
--log /root/sol/solana-validator.log
--geyser-plugin-config /root/sol/plugin_debug.cfg
--no-snapshot-fetch

服务器 内存512G，数据库服务器 在另外一台服务器

[249198000]

Important: Solana's plugin interface requires the build environment of the Solana validator and this plugin to be identical.

This includes the Solana version and Rust compiler version. Loading a plugin targeting wrong versions will result in memory corruption and crashes.