Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Receipts working with Generic Struct Extraction #430

Open
wants to merge 73 commits into
base: generic-extraction-integration-test-mapping-of-mappings
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
c3c1002
chroe: update dependencies (#411)
silathdiir Nov 27, 2024
195946f
test with receipts encoding
nikkolasg Oct 17, 2024
129984f
wip
nikkolasg Oct 18, 2024
485dcb9
further testing
nikkolasg Oct 21, 2024
68a0532
WIP: Receipt Trie leaves
Nov 7, 2024
4261eab
Receipt Leaf Circuit added with tests
Nov 8, 2024
7f9702c
Change Receipt query test
Zyouell Nov 8, 2024
9b81835
Address review comments
Zyouell Nov 11, 2024
f1abff8
Value digest computation corrected
Zyouell Nov 27, 2024
d071575
Moved receipt value extraction location
Zyouell Nov 27, 2024
a3a596f
Added unit tests for receipt leaf api
Zyouell Dec 2, 2024
931ab2d
Rebased onto feat/receipt-trie
Zyouell Dec 2, 2024
b72c9f1
Reworked block extraction to extract all three roots
Zyouell Dec 2, 2024
57c99f2
Added testing for final extraction API
Zyouell Dec 3, 2024
2e4e060
Addressed review comments
Zyouell Dec 17, 2024
030dc32
Fixed tests to pass CI
Zyouell Dec 17, 2024
551495c
Resolves CRY-22
Zyouell Dec 18, 2024
1cb06af
Changed TableSource to be a trait
Zyouell Dec 19, 2024
39810b7
Fixed stack too deep error
Zyouell Dec 19, 2024
2e98183
Changed final extraction circuit set size constant
Zyouell Dec 19, 2024
b0b4c13
resolves CRY-23
Zyouell Dec 23, 2024
a94068c
resolves CRY-25
Zyouell Dec 23, 2024
d7b6ff7
Fixed to_receipt method
Zyouell Dec 30, 2024
f2a0cd8
Updated ethers receipt compatibility test
Zyouell Dec 31, 2024
d693970
Receipt value extraction prover
Zyouell Dec 31, 2024
025d8db
Resolves CRY-26
Zyouell Dec 31, 2024
268f116
Fixed topic cell id discrepancy
Zyouell Jan 2, 2025
017a15a
Fixed receipt value digest
Zyouell Jan 2, 2025
727b00a
RowTreeUpdate Debugging
Zyouell Jan 2, 2025
0c7a982
Correct Receipt Row Tree
Zyouell Jan 2, 2025
01793ab
Fixed slot input lengths for test
Zyouell Jan 13, 2025
14cf959
StorageSlotInfo Single Value fix
Zyouell Jan 13, 2025
317351f
WIP: Review comments
Zyouell Jan 21, 2025
73f8212
Error types for retries in mp2_common::eth
Zyouell Jan 29, 2025
3783c4f
Added empty block tree proof
Zyouell Jan 29, 2025
d7c6135
Integrated querying encodes 0 to hex string correctly now
Zyouell Jan 30, 2025
85dd6ed
IQ debugging
Zyouell Jan 30, 2025
da96604
Rebased back on top of Generic Extraction
Zyouell Jan 31, 2025
1ccc212
Added dummy value extraction circuit
Zyouell Jan 31, 2025
7f610bc
Support non-contiguous block numbers
nicholas-mainardi Jan 3, 2025
c59a1e1
fmt + clippy
nicholas-mainardi Jan 3, 2025
cc7fd0b
Make current_epoch in EpochStorage trait return error
nicholas-mainardi Jan 3, 2025
72c45e9
fmt
nicholas-mainardi Jan 3, 2025
2bf0658
Fix table name provided to core_index_keys
nicholas-mainardi Jan 3, 2025
cf1c60f
Avoid expenmsive JOIN in some queries
nicholas-mainardi Jan 10, 2025
cfc65ad
Optimize row cache SQL queries
nicholas-mainardi Jan 10, 2025
652e732
Optimize JOIN wide lineage rows tree + fix eth test
nicholas-mainardi Jan 10, 2025
64a30c9
Address comments
nicholas-mainardi Jan 13, 2025
2d99d5e
Optimize InMemoryEpochMapper + imporve handling of incremental user e…
nicholas-mainardi Jan 14, 2025
f5f7db2
Add untracked file
nicholas-mainardi Jan 14, 2025
91191ae
Add maximum size to epoch mapper cache
nicholas-mainardi Jan 14, 2025
869a968
fmt
nicholas-mainardi Jan 14, 2025
e548c1e
Improve fetch_many_at
nicholas-mainardi Jan 15, 2025
8b591ab
Address comments
nicholas-mainardi Jan 16, 2025
57c5c10
Add indexes + optimizes bracketer secondary index
nicholas-mainardi Jan 20, 2025
33077f4
fmt
nicholas-mainardi Jan 20, 2025
7ffb465
Fix pidgy_pinguit test
nicholas-mainardi Jan 20, 2025
8eab5b5
Split bracketer queries for performance
nicholas-mainardi Jan 21, 2025
ea59525
fmt
nicholas-mainardi Jan 21, 2025
737079d
Improve comments to bracketer secondary index
nicholas-mainardi Jan 21, 2025
9ae7296
Values extraction API working
Zyouell Feb 3, 2025
8c963fa
Updated Extraction Planner
Zyouell Feb 3, 2025
1298394
Fix after rebase
nicholas-mainardi Feb 3, 2025
73540da
fmt
nicholas-mainardi Feb 3, 2025
6fae445
Removed ReceiptQuery struct
Zyouell Feb 4, 2025
cc8326c
Change indexing integration test to support non-consecutive blocks
nicholas-mainardi Feb 4, 2025
eab6236
Updated for review comments
Zyouell Feb 5, 2025
e130bd9
Changed receipt proof query to make fewer calls
Zyouell Feb 5, 2025
143ffa6
Changed receipt proof query to make fewer calls
Zyouell Feb 5, 2025
f5c75af
Changed receipt proof query to make fewer calls
Zyouell Feb 5, 2025
2f13f53
Query in integration test compatible with non-conseuctive blocks
nicholas-mainardi Feb 6, 2025
54a23a2
Fix typo in bracketer_primary_index query
nicholas-mainardi Feb 6, 2025
71961da
merged with feat/support_non_consecutive_blocks_ryhope
Zyouell Feb 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
833 changes: 453 additions & 380 deletions Cargo.lock

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -25,12 +25,14 @@ alloy = { version = "0.6", default-features = false, features = [
"rlp",
"rpc",
"rpc-types",
"json-rpc",
"signer-local",
"sol-types",
"transport-http",
"transports",
"postgres",
] }

anyhow = "1.0"
base64 = "0.22"
bb8 = "0.8.5"
@@ -75,7 +77,7 @@ rand_chacha = "0.3.1"
revm = { version = "3.5", default-features = false }
rlp = "0.5"
rstest = "0.23"
serde = { version = "1.0", features = ["derive"] }
serde = { version = "1.0", features = ["derive", "std"] }
serde_json = "1.0"
serial_test = "3.0"
sha2 = "0.10"
2 changes: 1 addition & 1 deletion inspect/src/index.rs
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ use crate::repl::PayloadFormatter;
pub(crate) type IndexDb = MerkleTreeKvDb<
BlockTree,
IndexNode<BlockPrimaryIndex>,
PgsqlStorage<BlockTree, IndexNode<BlockPrimaryIndex>>,
PgsqlStorage<BlockTree, IndexNode<BlockPrimaryIndex>, false>,
>;

struct IndexPayloadFormatterDisplay {
11 changes: 7 additions & 4 deletions inspect/src/main.rs
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ use repl::Repl;
use rows::{RowDb, RowPayloadFormatter};
use ryhope::{
storage::pgsql::{SqlServerConnection, SqlStorageSettings, ToFromBytea},
Epoch, InitSettings,
InitSettings, UserEpoch,
};
use serde::Serialize;

@@ -26,7 +26,7 @@ struct Args {

#[arg(short = 'E', long = "at")]
/// If set, try to view the tree at this epoch
epoch: Option<Epoch>,
epoch: Option<UserEpoch>,

#[command(subcommand)]
/// The type of tree to load from the database
@@ -77,6 +77,8 @@ async fn main() -> Result<()> {
SqlStorageSettings {
source: SqlServerConnection::NewConnection(args.db_uri.clone()),
table: args.db_table,
external_mapper: None, // not necessary even if there is an external epoch mapper,
// since we are initializing the tree with `InitSettings::MustExist`
},
)
.await?;
@@ -91,7 +93,7 @@ async fn main() -> Result<()> {

let mut repl = Repl::new(tree_db, payload_fmt).await?;
if let Some(epoch) = args.epoch {
repl.set_epoch(epoch)?;
repl.set_epoch(epoch).await?;
}
repl.run().await
}
@@ -101,6 +103,7 @@ async fn main() -> Result<()> {
SqlStorageSettings {
source: SqlServerConnection::NewConnection(args.db_uri.clone()),
table: args.db_table,
external_mapper: None,
},
)
.await?;
@@ -109,7 +112,7 @@ async fn main() -> Result<()> {

let mut repl = Repl::new(tree_db, payload_fmt).await?;
if let Some(epoch) = args.epoch {
repl.set_epoch(epoch)?;
repl.set_epoch(epoch).await?;
}
repl.run().await
}
22 changes: 11 additions & 11 deletions inspect/src/repl.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use anyhow::{anyhow, bail};
use anyhow::{anyhow, bail, Result};
use colored::Colorize;
use dialoguer::{console, theme::ColorfulTheme, FuzzySelect, Input};
use itertools::Itertools;
@@ -8,7 +8,7 @@ use ryhope::{
TreeStorage,
},
tree::{MutableTree, PrintableTree, TreeTopology},
Epoch, MerkleTreeKvDb, NodePayload,
MerkleTreeKvDb, NodePayload, UserEpoch,
};
use std::io::Write;
use tabled::{builder::Builder, settings::Style};
@@ -57,7 +57,7 @@ pub(crate) struct Repl<
F: PayloadFormatter<V>,
> {
current_key: T::Key,
current_epoch: Epoch,
current_epoch: UserEpoch,
db: MerkleTreeKvDb<T, V, S>,
tty: console::Term,
payload_fmt: F,
@@ -77,7 +77,7 @@ impl<
{
pub async fn new(db: MerkleTreeKvDb<T, V, S>, payload_fmt: F) -> anyhow::Result<Self> {
let current_key = db.root().await?.ok_or(anyhow!("tree is empty"))?;
let current_epoch = db.current_epoch();
let current_epoch = db.current_epoch().await?;

Ok(Self {
current_key,
@@ -105,19 +105,19 @@ impl<
.unwrap();
}

pub fn set_epoch(&mut self, epoch: Epoch) -> anyhow::Result<()> {
if epoch < self.db.initial_epoch() {
pub async fn set_epoch(&mut self, epoch: UserEpoch) -> Result<()> {
if epoch < self.db.initial_epoch().await {
bail!(
"epoch `{}` is older than initial epoch `{}`",
epoch,
self.db.initial_epoch()
self.db.initial_epoch().await
);
}
if epoch > self.db.current_epoch() {
if epoch > self.db.current_epoch().await? {
bail!(
"epoch `{}` is newer than latest epoch `{}`",
epoch,
self.db.current_epoch()
self.db.current_epoch().await?
);
}

@@ -147,9 +147,9 @@ impl<

async fn travel(&mut self) -> anyhow::Result<()> {
loop {
let epoch: Epoch = Input::new().with_prompt("target epoch:").interact_text()?;
let epoch: UserEpoch = Input::new().with_prompt("target epoch:").interact_text()?;

self.set_epoch(epoch)?;
self.set_epoch(epoch).await?;
}
}

2 changes: 1 addition & 1 deletion inspect/src/rows.rs
Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@ use crate::repl::PayloadFormatter;
pub(crate) type RowDb = MerkleTreeKvDb<
RowTree,
RowPayload<BlockPrimaryIndex>,
PgsqlStorage<RowTree, RowPayload<BlockPrimaryIndex>>,
PgsqlStorage<RowTree, RowPayload<BlockPrimaryIndex>, true>,
>;

struct RowPayloadFormatterDisplay {
2 changes: 1 addition & 1 deletion mp2-common/Cargo.toml
Original file line number Diff line number Diff line change
@@ -32,9 +32,9 @@ hex.workspace = true
rand.workspace = true
rstest.workspace = true
tokio.workspace = true

mp2_test = { path = "../mp2-test" }


[features]
ci = ["mp2_test/ci"]
original_poseidon = []
Loading