added support for multiproof generation within Prover

[noahfigueras]

I needed multiproof generation to verify multiple fields of data against the beacon block root in the evm. I came with this solution for minimal changes using Prover, not sure if that is acceptable, but works good for me. This is an idea to close #123

[mempirate]

Have been testing this branch on a mainnet block trying to prove the inclusion of a single transaction against the transactions_root with multiproofs (https://github.com/chainbound/bolt/blob/7fa63655ab02c4074913fc062fba6712fbc218ae/bolt-boost/src/proofs.rs). Some notes on performance:

• Generating single Merkle proofs in release mode:

Transactions root: 0x3834068daa909f5553bd06299dc60ccd43fa2c24da3778dda30b6c7fc767f240, num transactions: 129
Index to prove: 106
Generated proof in 821.146292ms
Verified proof in 8.333µs

• Generating a multiproof for a single leaf in release mode:

Transactions root: 0x3834068daa909f5553bd06299dc60ccd43fa2c24da3778dda30b6c7fc767f240, num transactions: 129
Index to prove: 14
Generated multiproof in 17.343940208s
Verified multiproof in 25.458µs

Flamegraph:

The majority of time is spent in building the Merkle tree, specifically in SHA256 hashing.

I can see a couple reasons for these results:

• The asm feature of sha2 is not enabled, resulting in slower hashing
• In the multiproof generation, the Merkle tree is rebuilt $$i \times h$$ times where $$i$$ is the number of generalized indeces and $$h$$ is the number of helper indices. This might help: Add Merkle tree caching for Vector #152

[noahfigueras]

Hi @mempirate! Yeah I agree, the way it's performed it's not really efficient. I'm currently computing a tree to collect each branch for the proof. Ideally, you would want to call prover one time and traverse the tree at once if you keep track of indices and helper indices you can just collect the branches from the leaves. Last time I tried, I had some issues to keep that inside the current Prover implementation, I'll have to rethink this a little.

[mempirate]

#158 also helps with performance by enabling assembly support for hashing. I think combining this with a more efficient multiproof generation algo should already help a lot with proof generation time.

[mempirate]

Was able to get proof generation speed down to around 2-3 seconds as a first step, first with sha2-asm:

Generated multiproof in 2.909094792s
Verified multiproof in 24.084µs

And with the hashtree_rs crate (https://crates.io/crates/hashtree-rs):

Generated multiproof in 2.328004958s
Verified multiproof in 30.042µs