Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Murisi/masp separate parallel sync #2474

Closed
wants to merge 8 commits into from

Conversation

murisi
Copy link
Contributor

@murisi murisi commented Jan 30, 2024

Describe your changes

This PR creates a separate synchronization command and heavily draws code from #2422 . More specifically, this PR's changes are as follows:

  • Parallelized the downloading of transactions with the scanning of transactions and updating of the note Merkle tree
    • This reduces the wall-clock time required to synchronize the shielded context
    • This allows us to avoid having to cache downloaded transactions until transaction scanning starts
  • Created a separate command to synchronize the shielded context called shielded-sync
  • Adjusted all other commands to no longer implicitly synchronize the shielded pool before executing their actions
  • Modified the synchronization function to save state frequently so that the interrupted and resumed without data loss
  • Adjusted the integration tests so that they now explicitly do shielded-syncs before doing any actions

Indicate on which release or other PRs this topic is based on

#2458

Checklist before merging to draft

  • I have added a changelog
  • Git history is in acceptable state

@murisi murisi marked this pull request as ready for review January 30, 2024 14:29
fn parse(matches: &ArgMatches) -> Self {
let ledger_address = LEDGER_ADDRESS_DEFAULT.parse(matches);
let last_query_height = BLOCK_HEIGHT_OPT.parse(matches);
let viewing_keys = VIEWING_KEYS.parse(matches);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we not want to support passing in spending keys here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whenever a spending key is added to the wallet, its corresponding viewing key is also added under the same name. So I'm thinking that that this code should be sufficient for now. No?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I just wondered if we considered it important.

Copy link
Member

@batconjurer batconjurer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of small changes. Also, I'm not seeing marco's changes in here. I think there is a divergence between base and main.

.collect::<Vec<_>>();
let mut shielded = namada.shielded_mut().await;
let _ = shielded.load().await;
for vk in wallet_keys {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave the internals of vk heights inside of the sync function. Instead lets concatenate all of the keys together and pass them into the shielded sync function.

}
// the latest block height which has been added to the witness Merkle
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually surprised that the integration tests pass. There is bug here that one failed for me on. We need to save the shielded context here to persist the keys.

Copy link
Contributor

@grarco grarco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thank you!

@murisi murisi closed this Feb 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants