Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Da nomos core #390

Merged
merged 8 commits into from
Sep 11, 2023
Merged

Da nomos core #390

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ec98fa2
Refactor da modules
danielSanchezQ Sep 11, 2023
3d42ba9
Include attestation trait
danielSanchezQ Sep 11, 2023
9f9aa82
Added initial approach for Da protocol
danielSanchezQ Sep 11, 2023
bb15638
Added empty certificate trait
danielSanchezQ Sep 11, 2023
2baa92b
Added certificate dispersal method
danielSanchezQ Sep 11, 2023
6521278
Rename validate method to validate attestation
danielSanchezQ Sep 11, 2023
e97ec14
Clippy happy
danielSanchezQ Sep 11, 2023
20c9148
Add validate certificate method
danielSanchezQ Sep 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ members = [
"nomos-services/mempool",
"nomos-services/http",
"nomos-services/data-availability",
"nomos-da-core/reed-solomon",
"nomos-da-core/kzg",
"nomos-da/reed-solomon",
"nomos-da/kzg",
"nodes/nomos-node",
"simulations",
"consensus-engine",
Expand Down
8 changes: 8 additions & 0 deletions nomos-core/src/da/attestation/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
use crate::da::blob::Blob;
use bytes::Bytes;

pub trait Attestation {
type Blob: Blob;
fn blob(&self) -> <Self::Blob as Blob>::Hash;
fn as_bytes(&self) -> Bytes;
}
0 nomos-core/src/blob/mod.rs → nomos-core/src/da/blob/mod.rs
View file
Open in desktop
File renamed without changes.
1 change: 1 addition & 0 deletions nomos-core/src/da/certificate/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pub trait Certificate {}
30 changes: 30 additions & 0 deletions nomos-core/src/da/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
// std
use std::error::Error;
// crates
use bytes::Bytes;
use futures::Stream;
// internal
use crate::da::attestation::Attestation;
use crate::da::blob::Blob;
use crate::da::certificate::Certificate;

pub mod attestation;
pub mod blob;
pub mod certificate;

pub trait DaProtocol {
type Blob: Blob;
type Attestation: Attestation;
type Certificate: Certificate;

fn encode<T: AsRef<[u8]>>(&self, data: T) -> Box<dyn Stream<Item = Self::Blob>>;
fn decode<S: Stream<Item = Self::Blob>>(&self, s: S) -> Result<Bytes, Box<dyn Error>>;
fn validate_attestation(&self, blob: &Self::Blob, attestation: &Self::Attestation) -> bool;

fn certificate_dispersal<S: Stream<Item = Self::Attestation>>(
&self,
attestations: S,
) -> Self::Certificate;

fn validate_certificate(certificate: &Self::Certificate) -> bool;
Copy link
Contributor

@zeegomo zeegomo Sep 11, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this have the blob in input?

EDIT: I think it's actually ok for now since we don't know much yet

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure, actually. Maybe we need to validate the certificate over the original data?
In this case certificate is the aggregation of attestations that proofs that data is distributed. Either we can check it stand alone or for all blobs (?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess having something Certificate::blob is sufficient, you would then check that the certificate itself is valid on chain and you have the blob id to retrieve it if necessary

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are calling blob different stuff here (it may be my fault in naming actually). A blob is what a node receives (and it may be just a chunk of the original data). Being the certificate an aggregation of attestations (refering to a blob) its difficult the certificate can return a single id of a single blob. In this case I see two things. I can rename it to Blob (the whole data) and BlobChunk (a blob that a node receives). Or we consider the orinal data blob just data and pass away the hash of the whole thing (but I am not really sure we need this anyway, as it may be better to have validation methods where needed instead?).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I was confusing those terms. I think the fact that it might be sharded it's a protocol detail (e.g. we do that with rs encoding but not with full replication), while the concept of bl-blobs is independent of the way we do data availability, it's just a piece of data that is available somewhere.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But anyway, in the case of a full replication we would just create a Blob that contains the same data for each of them (encoding non-encoding 😂 ). I think for now we can assume that the certificate can be validated by itself, and we can change it later if we see fit?

}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this supposed to validate a single attestation or the full proof?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validate the attestation for an specific blob. We should probably add a trait method to create a certificate over the set of attestations. I'll add now as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe also a method to validate said certificate?

2 changes: 1 addition & 1 deletion nomos-core/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pub mod account;
pub mod blob;
pub mod block;
pub mod crypto;
pub mod da;
pub mod fountain;
pub mod staking;
pub mod tx;
Expand Down
0 nomos-da-core/kzg/Cargo.toml → nomos-da/kzg/Cargo.toml
View file
Open in desktop
File renamed without changes.
0 nomos-da-core/kzg/benches/nomos_kzg.rs → nomos-da/kzg/benches/nomos_kzg.rs
View file
Open in desktop
File renamed without changes.
0 nomos-da-core/kzg/src/dynamic_kzg.rs → nomos-da/kzg/src/dynamic_kzg.rs
View file
Open in desktop
File renamed without changes.
0 nomos-da-core/kzg/src/lib.rs → nomos-da/kzg/src/lib.rs
View file
Open in desktop
File renamed without changes.
0 nomos-da-core/kzg/src/types.rs → nomos-da/kzg/src/types.rs
View file
Open in desktop
File renamed without changes.
0 nomos-da-core/reed-solomon/Cargo.toml → nomos-da/reed-solomon/Cargo.toml
View file
Open in desktop
File renamed without changes.
0 nomos-da-core/reed-solomon/src/lib.rs → nomos-da/reed-solomon/src/lib.rs
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion nomos-services/data-availability/src/backend/memory_cache.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::backend::{DaBackend, DaError};
use moka::future::{Cache, CacheBuilder};
use nomos_core::blob::Blob;
use nomos_core::da::blob::Blob;
use std::time::Duration;

#[derive(Clone, Copy)]
Expand Down
2 changes: 1 addition & 1 deletion nomos-services/data-availability/src/backend/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
mod memory_cache;

use nomos_core::blob::Blob;
use nomos_core::da::blob::Blob;
use overwatch_rs::DynError;

#[derive(Debug)]
Expand Down
2 changes: 1 addition & 1 deletion nomos-services/data-availability/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ use tokio::sync::oneshot::Sender;
// internal
use crate::backend::{DaBackend, DaError};
use crate::network::NetworkAdapter;
use nomos_core::blob::Blob;
use nomos_core::da::blob::Blob;
use nomos_network::NetworkService;
use overwatch_rs::services::handle::ServiceStateHandle;
use overwatch_rs::services::relay::{Relay, RelayMessage};
Expand Down
Loading