Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(da-clients): add EigenDA client #3155

Merged
merged 24 commits into from
Oct 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -215,9 +215,12 @@ subxt-signer = { version = "0.34", default-features = false }
celestia-types = "0.6.1"
bech32 = "0.11.0"
ripemd = "0.1.3"
tonic = "0.11.0"
tonic = { version = "0.11.0", default-features = false }
pbjson-types = "0.6.0"

# Eigen
tokio-stream = "0.1.16"

# Here and below:
# We *always* pin the latest version of protocol to disallow accidental changes in the execution logic.
# However, for the historical version of protocol crates, we have lax requirements. Otherwise,
Expand Down
8 changes: 6 additions & 2 deletions core/bin/zksync_server/src/node_builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ use zksync_node_framework::{
consensus::MainNodeConsensusLayer,
contract_verification_api::ContractVerificationApiLayer,
da_clients::{
avail::AvailWiringLayer, celestia::CelestiaWiringLayer, no_da::NoDAClientWiringLayer,
object_store::ObjectStorageClientWiringLayer,
avail::AvailWiringLayer, celestia::CelestiaWiringLayer, eigen::EigenWiringLayer,
no_da::NoDAClientWiringLayer, object_store::ObjectStorageClientWiringLayer,
},
da_dispatcher::DataAvailabilityDispatcherLayer,
eth_sender::{EthTxAggregatorLayer, EthTxManagerLayer},
Expand Down Expand Up @@ -517,6 +517,10 @@ impl MainNodeBuilder {
.add_layer(CelestiaWiringLayer::new(config, secret));
}

(DAClientConfig::Eigen(config), DataAvailabilitySecrets::Eigen(secret)) => {
self.node.add_layer(EigenWiringLayer::new(config, secret));
}

(DAClientConfig::ObjectStore(config), _) => {
self.node
.add_layer(ObjectStorageClientWiringLayer::new(config));
Expand Down
13 changes: 13 additions & 0 deletions core/lib/config/src/configs/da_client/eigen.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
use serde::Deserialize;
use zksync_basic_types::secrets::PrivateKey;

#[derive(Clone, Debug, Default, PartialEq, Deserialize)]
pub struct EigenConfig {
pub rpc_node_url: String,
pub inclusion_polling_interval_ms: u64,
}

#[derive(Clone, Debug, PartialEq)]
pub struct EigenSecrets {
pub private_key: PrivateKey,
}
5 changes: 4 additions & 1 deletion core/lib/config/src/configs/da_client/mod.rs
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
use crate::{AvailConfig, CelestiaConfig, ObjectStoreConfig};
use crate::{AvailConfig, CelestiaConfig, EigenConfig, ObjectStoreConfig};

pub mod avail;
pub mod celestia;
pub mod eigen;

pub const AVAIL_CLIENT_CONFIG_NAME: &str = "Avail";
pub const CELESTIA_CLIENT_CONFIG_NAME: &str = "Celestia";
pub const EIGEN_CLIENT_CONFIG_NAME: &str = "Eigen";
pub const OBJECT_STORE_CLIENT_CONFIG_NAME: &str = "ObjectStore";

#[derive(Debug, Clone, PartialEq)]
pub enum DAClientConfig {
Avail(AvailConfig),
Celestia(CelestiaConfig),
Eigen(EigenConfig),
ObjectStore(ObjectStoreConfig),
}
2 changes: 1 addition & 1 deletion core/lib/config/src/configs/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ pub use self::{
commitment_generator::CommitmentGeneratorConfig,
contract_verifier::ContractVerifierConfig,
contracts::{ContractsConfig, EcosystemContracts},
da_client::{avail::AvailConfig, celestia::CelestiaConfig, DAClientConfig},
da_client::{avail::AvailConfig, celestia::CelestiaConfig, eigen::EigenConfig, DAClientConfig},
da_dispatcher::DADispatcherConfig,
database::{DBConfig, PostgresConfig},
eth_sender::{EthConfig, GasAdjusterConfig},
Expand Down
3 changes: 2 additions & 1 deletion core/lib/config/src/configs/secrets.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use zksync_basic_types::url::SensitiveUrl;

use crate::configs::{
consensus::ConsensusSecrets,
da_client::{avail::AvailSecrets, celestia::CelestiaSecrets},
da_client::{avail::AvailSecrets, celestia::CelestiaSecrets, eigen::EigenSecrets},
};

#[derive(Debug, Clone, PartialEq)]
Expand All @@ -22,6 +22,7 @@ pub struct L1Secrets {
pub enum DataAvailabilitySecrets {
Avail(AvailSecrets),
Celestia(CelestiaSecrets),
Eigen(EigenSecrets),
}

#[derive(Debug, Clone, PartialEq)]
Expand Down
6 changes: 3 additions & 3 deletions core/lib/config/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

pub use crate::configs::{
ApiConfig, AvailConfig, BaseTokenAdjusterConfig, CelestiaConfig, ContractVerifierConfig,
ContractsConfig, DAClientConfig, DADispatcherConfig, DBConfig, EthConfig, EthWatchConfig,
ExternalProofIntegrationApiConfig, GasAdjusterConfig, GenesisConfig, ObjectStoreConfig,
PostgresConfig, SnapshotsCreatorConfig,
ContractsConfig, DAClientConfig, DADispatcherConfig, DBConfig, EigenConfig, EthConfig,
EthWatchConfig, ExternalProofIntegrationApiConfig, GasAdjusterConfig, GenesisConfig,
ObjectStoreConfig, PostgresConfig, SnapshotsCreatorConfig,
};

pub mod configs;
Expand Down
37 changes: 33 additions & 4 deletions core/lib/env_config/src/da_client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ use zksync_config::configs::{
AvailClientConfig, AvailSecrets, AVAIL_FULL_CLIENT_NAME, AVAIL_GAS_RELAY_CLIENT_NAME,
},
celestia::CelestiaSecrets,
eigen::EigenSecrets,
DAClientConfig, AVAIL_CLIENT_CONFIG_NAME, CELESTIA_CLIENT_CONFIG_NAME,
OBJECT_STORE_CLIENT_CONFIG_NAME,
EIGEN_CLIENT_CONFIG_NAME, OBJECT_STORE_CLIENT_CONFIG_NAME,
},
secrets::DataAvailabilitySecrets,
AvailConfig,
Expand All @@ -33,6 +34,7 @@ impl FromEnv for DAClientConfig {
},
}),
CELESTIA_CLIENT_CONFIG_NAME => Self::Celestia(envy_load("da_celestia_config", "DA_")?),
EIGEN_CLIENT_CONFIG_NAME => Self::Eigen(envy_load("da_eigen_config", "DA_")?),
OBJECT_STORE_CLIENT_CONFIG_NAME => {
Self::ObjectStore(envy_load("da_object_store", "DA_")?)
}
Expand Down Expand Up @@ -66,11 +68,18 @@ impl FromEnv for DataAvailabilitySecrets {
}
CELESTIA_CLIENT_CONFIG_NAME => {
let private_key = env::var("DA_SECRETS_PRIVATE_KEY")
.map_err(|e| anyhow::format_err!("private key not found: {}", e))?
.map_err(|e| anyhow::format_err!("Celestia private key not found: {}", e))?
.parse()
.map_err(|e| anyhow::format_err!("failed to parse the auth token: {}", e))?;
.map_err(|e| anyhow::format_err!("failed to parse the private key: {}", e))?;
Self::Celestia(CelestiaSecrets { private_key })
}
EIGEN_CLIENT_CONFIG_NAME => {
let private_key = env::var("DA_SECRETS_PRIVATE_KEY")
.map_err(|e| anyhow::format_err!("Eigen private key not found: {}", e))?
.parse()
.map_err(|e| anyhow::format_err!("failed to parse the private key: {}", e))?;
Self::Eigen(EigenSecrets { private_key })
}

_ => anyhow::bail!("Unknown DA client name: {}", client_tag),
};
Expand All @@ -89,7 +98,7 @@ mod tests {
},
object_store::ObjectStoreMode::GCS,
},
AvailConfig, CelestiaConfig, ObjectStoreConfig,
AvailConfig, CelestiaConfig, EigenConfig, ObjectStoreConfig,
};

use super::*;
Expand Down Expand Up @@ -234,6 +243,26 @@ mod tests {
);
}

#[test]
fn from_env_eigen_client() {
let mut lock = MUTEX.lock();
let config = r#"
DA_CLIENT="Eigen"
DA_RPC_NODE_URL="localhost:12345"
DA_INCLUSION_POLLING_INTERVAL_MS="1000"
"#;
lock.set_env(config);

let actual = DAClientConfig::from_env().unwrap();
assert_eq!(
actual,
DAClientConfig::Eigen(EigenConfig {
rpc_node_url: "localhost:12345".to_string(),
inclusion_polling_interval_ms: 1000,
})
);
}

#[test]
fn from_env_celestia_secrets() {
let mut lock = MUTEX.lock();
Expand Down
15 changes: 13 additions & 2 deletions core/lib/protobuf_config/src/da_client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ use zksync_config::configs::{
da_client::{
avail::{AvailClientConfig, AvailConfig, AvailDefaultConfig, AvailGasRelayConfig},
celestia::CelestiaConfig,
DAClientConfig::{Avail, Celestia, ObjectStore},
eigen::EigenConfig,
DAClientConfig::{Avail, Celestia, Eigen, ObjectStore},
},
};
use zksync_protobuf::{required, ProtoRepr};
Expand Down Expand Up @@ -51,6 +52,13 @@ impl ProtoRepr for proto::DataAvailabilityClient {
chain_id: required(&conf.chain_id).context("chain_id")?.clone(),
timeout_ms: *required(&conf.timeout_ms).context("timeout_ms")?,
}),
proto::data_availability_client::Config::Eigen(conf) => Eigen(EigenConfig {
rpc_node_url: required(&conf.rpc_node_url)
.context("rpc_node_url")?
.clone(),
inclusion_polling_interval_ms: *required(&conf.inclusion_polling_interval_ms)
.context("inclusion_polling_interval_ms")?,
}),
proto::data_availability_client::Config::ObjectStore(conf) => {
ObjectStore(object_store_proto::ObjectStore::read(conf)?)
}
Expand Down Expand Up @@ -79,7 +87,6 @@ impl ProtoRepr for proto::DataAvailabilityClient {
),
},
}),

Celestia(config) => {
proto::data_availability_client::Config::Celestia(proto::CelestiaConfig {
api_node_url: Some(config.api_node_url.clone()),
Expand All @@ -88,6 +95,10 @@ impl ProtoRepr for proto::DataAvailabilityClient {
timeout_ms: Some(config.timeout_ms),
})
}
Eigen(config) => proto::data_availability_client::Config::Eigen(proto::EigenConfig {
rpc_node_url: Some(config.rpc_node_url.clone()),
inclusion_polling_interval_ms: Some(config.inclusion_polling_interval_ms),
}),
ObjectStore(config) => proto::data_availability_client::Config::ObjectStore(
object_store_proto::ObjectStore::build(config),
),
Expand Down
6 changes: 6 additions & 0 deletions core/lib/protobuf_config/src/proto/config/da_client.proto
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,17 @@ message CelestiaConfig {
optional uint64 timeout_ms = 4;
}

message EigenConfig {
optional string rpc_node_url = 1;
optional uint64 inclusion_polling_interval_ms = 2;
}

message DataAvailabilityClient {
// oneof in protobuf allows for None
oneof config {
AvailConfig avail = 1;
object_store.ObjectStore object_store = 2;
CelestiaConfig celestia = 3;
EigenConfig eigen = 4;
}
}
5 changes: 5 additions & 0 deletions core/lib/protobuf_config/src/proto/config/secrets.proto
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,15 @@ message CelestiaSecret {
optional string private_key = 1;
}

message EigenSecret {
optional string private_key = 1;
}

message DataAvailabilitySecrets {
oneof da_secrets {
AvailSecret avail = 1;
CelestiaSecret celestia = 2;
EigenSecret eigen = 3;
}
}

Expand Down
10 changes: 9 additions & 1 deletion core/lib/protobuf_config/src/secrets.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ use zksync_basic_types::{
};
use zksync_config::configs::{
consensus::{AttesterSecretKey, ConsensusSecrets, NodeSecretKey, ValidatorSecretKey},
da_client::{avail::AvailSecrets, celestia::CelestiaSecrets},
da_client::{avail::AvailSecrets, celestia::CelestiaSecrets, eigen::EigenSecrets},
secrets::{DataAvailabilitySecrets, Secrets},
DatabaseSecrets, L1Secrets,
};
Expand Down Expand Up @@ -133,6 +133,11 @@ impl ProtoRepr for proto::DataAvailabilitySecrets {
required(&celestia.private_key).context("private_key")?,
)?,
}),
DaSecrets::Eigen(eigen) => DataAvailabilitySecrets::Eigen(EigenSecrets {
private_key: PrivateKey::from_str(
required(&eigen.private_key).context("private_key")?,
)?,
}),
};

Ok(client)
Expand Down Expand Up @@ -179,6 +184,9 @@ impl ProtoRepr for proto::DataAvailabilitySecrets {
private_key: Some(config.private_key.0.expose_secret().to_string()),
}))
}
DataAvailabilitySecrets::Eigen(config) => Some(DaSecrets::Eigen(proto::EigenSecret {
private_key: Some(config.private_key.0.expose_secret().to_string()),
})),
};

Self {
Expand Down
6 changes: 5 additions & 1 deletion core/node/da_clients/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ zksync_types.workspace = true
zksync_object_store.workspace = true
zksync_da_client.workspace = true
zksync_env_config.workspace = true
zksync_basic_types.workspace = true
futures.workspace = true

# Avail dependencies
Expand All @@ -49,5 +50,8 @@ sha2.workspace = true
prost.workspace = true
bech32.workspace = true
ripemd.workspace = true
tonic.workspace = true
tonic = { workspace = true, features = ["tls", "default"] }
pbjson-types.workspace = true

# Eigen dependencies
tokio-stream.workspace = true
2 changes: 2 additions & 0 deletions core/node/da_clients/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,5 @@ Currently, the following DataAvailability clients are implemented:
utilizing the DA framework.
- `Object Store client` that stores the pubdata in the Object Store(GCS).
- `Avail` that sends the pubdata to the Avail DA layer.
- `Celestia` that sends the pubdata to the Celestia DA layer.
- `Eigen` that sends the pubdata to the Eigen DA layer.
35 changes: 35 additions & 0 deletions core/node/da_clients/src/eigen/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# EigenDA client

---

This is an implementation of the EigenDA client capable of sending the blobs to DA layer. It uses authenticated
requests, though the auth headers are kind of mocked in the current API implementation.

The generated files are received by compiling the `.proto` files from EigenDA repo using the following function:

```rust
pub fn compile_protos() {
let fds = protox::compile(
[
"proto/common.proto",
"proto/disperser.proto",
],
["."],
)
.expect("protox failed to build");

tonic_build::configure()
.build_client(true)
.build_server(false)
.skip_protoc_run()
.out_dir("generated")
.compile_fds(fds)
.unwrap();
}
```

proto files are not included here to not create confusion in case they are not updated in time, so the EigenDA
[repo](https://github.com/Layr-Labs/eigenda/tree/master/api/proto) has to be a source of truth for the proto files.

The generated folder here is considered a temporary solution until the EigenDA has a library with either a protogen, or
preferably a full Rust client implementation.
Loading
Loading