stdlib smt replacement (single key-value leaf only) (#1215)

* (temporary) smt_new with `smt_new::get` * fix `mvdnw` typo * advice map: keys are now leaf hashes * smt.set: empty case * insert: single leaf * remove TODO * rename to LEAF_DEPTH * Change miden-crypto branch * `test_smt_get` * `test_smt_set` * gitignore vscode * fmt * masm: fix key[3] -> key[0] * add advice map to test * tests: fix advice_map * test_smt_get passes * fix `smt_new::set` * split `test_smt_key` * tests: change LEAVES keys * push_mapvaln typo * update `adv.push_smtpeek` * fix arguments to `mtree_set` * fix `adv.push_mapvaln` arguments * fix stack * fix inserting to existing key * add test: insert empty value to empty tree * `set_empty_leaf` proc * insert empty value on empty leaf * file sections * `set_single_leaf` * set_single_leaf: check insert or remove * fix insert single leaf * `test_smt_set`: run the test twice * set_single_leaf: implement removal * extract code to helper functions * comment * document masm functions * rename `smt_new` * `push_smtget_inputs`: make `unimplemented!()` * `push_smtset_inputs`: make unimplemented * `push_smtpeek_result`: fix docstring * `smt` injectors: remove unused methods * `AdviceMapValueInvalidLength` rename * update miden-crypto to branch `next` * Remove docstrings for unimplemented advice injectors * update mdbook for `smt` * Specify `Advice Stack` in docstring * test: change key * change test name * `stdlib`: Remove `miden-core` dependency * Rename `AdviceMapValueInvalidLength` error variant * `ExecutionError`: add error types for `Smt` advice map * empty leaf case: `mtree_verify` the advice provider claim * `set_empty_leaf` cycle count * `insert_single_leaf` * cycles: set * cycle count: get * change to `K[3]` * don't update advice map on removal * mdbook * fix doc for `adv.push_smt*` io ops * temporarily use `plafer-smt-get-value` branch for miden-crypto * Revert "temporarily use `plafer-smt-get-value` branch for miden-crypto" This reverts commit b62c021. * Fix advice map update in `Smt` masm (#1225) * update `Test` struct to make `Host` accessible * test_set_advice_map * fix masm * fix test * add test * Revert "update `Test` struct to make `Host` accessible" This reverts commit 825544e. * Fix `Smt` empty leaf check (#1228) * failing test * fix masm * fix `insert_single_leaf` * fix `remove_single_leaf` - prev case * implement new case * fix comment * replace unknown cycle count with X * fixed branch * chore: revert miden-crypto branch to next * Re-export `SmtProof{Error}` --------- Co-authored-by: Bobbin Threadbare <[email protected]>
0xPolygonMiden · Feb 7, 2024 · 9ca7ab3 · 9ca7ab3
1 parent 0a4fa7a
commit 9ca7ab3
Show file tree

Hide file tree

Showing 13 changed files with 707 additions and 2,551 deletions.
diff --git a/.gitignore b/.gitignore
@@ -19,4 +19,7 @@ stdlib/assets/std.masl
 **/.DS_Store
 
 # File present in Intellij IDE's.
-.idea/
+.idea/
+
+# VS Code
+.vscode/
diff --git a/core/Cargo.toml b/core/Cargo.toml
@@ -22,7 +22,6 @@ std = ["miden-crypto/std", "math/std", "winter-utils/std"]
 
 [dependencies]
 math = { package = "winter-math", version = "0.7", default-features = false }
-# miden-crypto = { package = "miden-crypto", version = "0.8", default-features = false }
 miden-crypto = { package = "miden-crypto", git = "https://github.com/0xPolygonMiden/crypto", branch = "next", default-features = false }
 winter-utils = { package = "winter-utils", version = "0.7", default-features = false }
 

diff --git a/core/src/lib.rs b/core/src/lib.rs
@@ -13,7 +13,8 @@ pub mod crypto {
         pub use miden_crypto::merkle::{
             DefaultMerkleStore, EmptySubtreeRoots, InnerNodeInfo, LeafIndex, MerkleError,
             MerklePath, MerkleStore, MerkleTree, Mmr, MmrPeaks, NodeIndex, PartialMerkleTree,
-            RecordingMerkleStore, SimpleSmt, StoreNode, TieredSmt,
+            RecordingMerkleStore, SimpleSmt, Smt, SmtProof, SmtProofError, StoreNode, TieredSmt,
+            SMT_DEPTH,
         };
     }
 

diff --git a/docs/src/user_docs/assembly/io_operations.md b/docs/src/user_docs/assembly/io_operations.md
@@ -55,9 +55,7 @@ Advice injectors fall into two categories: (1) injectors which push new data ont
 | adv.push_u64div                              | [b1, b0, a1, a0, ...]      | [b1, b0, a1, a0, ...]      | Pushes the result of `u64` division $a / b$ onto the advice stack. Both $a$ and $b$ are represented using 32-bit limbs. The result consists of both the quotient and the remainder. |
 | adv.push_ext2intt                            | [osize, isize, iptr, ... ] | [osize, isize, iptr, ... ] | Given evaluations of a polynomial over some specified domain, interpolates the evaluations into a polynomial in coefficient form and pushes the result into the advice stack. |
 | adv.push_sig.*kind*                          | [K, M, ...]                | [K, M, ...]                | Pushes values onto the advice stack which are required for verification of a DSA with scheme specified by *kind* against the public key commitment $K$ and message $M$. |
-| adv.smt_get                                  | [K, R, ... ]               | [K, R, ... ]               | Pushes values onto the advice stack which are required for successful retrieval of a value under the key $K$ from a Sparse Merkle Tree with root $R$. |
-| adv.smt_set                                  | [V, K, R, ...]             | [V, K, R, ...]             | Pushes values onto the advice stack which are required for successful insertion of a key-value pair $(K, V)$ into a Sparse Merkle Tree with root $R$. |
-| adv.smt_peek                                 | [K, R, ... ]               | [K, R, ... ]               | Pushes value onto the advice stack which is associated with key $K$ in a Sparse Merkle Tree with root $R$. |
+| adv.push_smtpeek                                 | [K, R, ... ]               | [K, R, ... ]               | Pushes value onto the advice stack which is associated with key $K$ in a Sparse Merkle Tree with root $R$. |
 | adv.insert_mem                               | [K, a, b, ... ]            | [K, a, b, ... ]            | Reads words $data \leftarrow mem[a] .. mem[b]$ from memory, and save the data into $advice\_map[K] \leftarrow data$. |
 | adv.insert_hdword <br> adv.insert_hdword.*d* | [B, A, ... ]               | [B, A, ... ]               | Reads top two words from the stack, computes a key as $K \leftarrow hash(A || b, d)$, and saves the data into $advice\_map[K] \leftarrow [A, B]$. $d$ is an optional domain value which can be between $0$ and $255$, default value $0$. |
 | adv.insert_hperm                             | [B, A, C, ...]             | [B, A, C, ...]             | Reads top three words from the stack, computes a key as $K \leftarrow permute(C, A, B).digest$, and saves data into $advice\_mpa[K] \leftarrow [A, B]$. |

diff --git a/docs/src/user_docs/stdlib/collections.md b/docs/src/user_docs/stdlib/collections.md
@@ -17,7 +17,7 @@ The following procedures are available to read data from and make updates to a M
 | pack        | Computes a commitment to the given MMR and copies the MMR to the Advice Map using the commitment as a key.<br /><br />Inputs: `[mmr_ptr, ...]`<br />Outputs: `[HASH, ...]`<br /><br /> |
 | unpack      | Load the MMR peak data based on its hash.<br /><br />Inputs: `[HASH, mmr_ptr, ...]`<br />Outputs: `[...]`<br /><br />Where:<br />- `HASH`: is the MMR peak hash, the hash is expected to be padded to an even length and to have a minimum size of 16 elements.<br />- The advice map must contain a key with `HASH`, and its value is `num_leaves \|\| hash_data`, and hash_data is the data used to computed `HASH`<br />- `mmt_ptr`: the memory location where the MMR data will be written, starting with the MMR forest (the total count of its leaves) followed by its peaks. |
 
-## Sparse Merkle Tree (64)
+## Sparse Merkle Tree (64-bit key)
 
 Module `std::collections::smt64` contains procedures for manipulating key-value maps with single-element keys and 4-element values. The current implementation is a thin wrapper over a simple Sparse Merkle Tree of depth 64. In the future, this will be replaced with a compact Sparse Merkle Tree implementation.
 
@@ -29,14 +29,13 @@ The following procedures are available to read data from and make updates to a S
 | set         | Inserts the specified value under the specified key in a Sparse Merkle Tree defined by the specified root. If the insert is successful, the old value located under the specified key is returned via the stack.<br /><br />If `VALUE` is an empty word, the new state of the tree is guaranteed to be equivalent to the state as if the updated value was never inserted.<br /><br />Inputs: `[VALUE, key, ROOT, ...]`<br />Outputs: `[OLD_VALUE, NEW_ROOT, ...]`<br /><br />Fails if the tree with the specified root does not exits in the VM's advice provider. |
 | insert      | Inserts the specified value under the specified key in a Sparse Merkle Tree defined by the specified root. If the insert is successful, the old value located under the specified key is returned via the stack.<br /><br />This procedure requires that `VALUE` be a non-empty word.<br /><br />Inputs: `[VALUE, key, ROOT, ...]`<br />Outputs: `[OLD_VALUE, NEW_ROOT, ...]`<br /><br />Fails if:<br />- The tree with the specified root does not exits in the VM's advice provider.<br />- The provided value is an empty word. |
 
-## Sparse Merkle Tree (256)
+## Sparse Merkle Tree (256-bit key)
 
-Module `std::collections::smt` contains procedures for manipulating key-value maps with 4-element keys and 4-element values. The underlying implementation is a Tiered (compacted) Sparse Merkle where leaves can exist only at specific depths called "tiers". These depths are: 16, 32, 48, and 64. Initially, when a tree is empty, it is equivalent to an empty Sparse Merkle Tree of depth 64 (i.e., leaves at depth 64 are set to [ZERO; 4]). As non-empty values are inserted into the tree, they are added to the first available tier.
+Module `std::collections::smt` contains procedures for manipulating key-value maps with 4-element keys and 4-element values. The underlying implementation is a Sparse Merkle Tree where leaves can exist only at depth 64. Initially, when a tree is empty, it is equivalent to an empty Sparse Merkle Tree of depth 64 (i.e., leaves at depth 64 are set and hash to [ZERO; 4]). When inserting non-empty values into the tree, the most significant element of the key is used to identify the corresponding leaf. All key-value pairs that map to a given leaf are inserted (ordered) in the leaf.
 
 The following procedures are available to read data from and make updates to a Sparse Merkle Tree.
 
 | Procedure   | Description   |
 | ----------- | ------------- |
 | get         | Returns the value located under the specified key in the Sparse Merkle Tree defined by the specified root.<br /><br />If no values had been previously inserted under the specified key, an empty word is returned.<br /><br />Inputs: `[KEY, ROOT, ...]`<br />Outputs: `[VALUE, ROOT, ...]`<br /><br />Fails if the tree with the specified root does not exist in the VM's advice provider. |
 | set         | Inserts the specified value under the specified key in a Sparse Merkle Tree defined by the specified root. If the insert is successful, the old value located under the specified key is returned via the stack.<br /><br />If `VALUE` is an empty word, the new state of the tree is guaranteed to be equivalent to the state as if the updated value was never inserted.<br /><br />Inputs: `[VALUE, KEY, ROOT, ...]`<br />Outputs: `[OLD_VALUE, NEW_ROOT, ...]`<br /><br />Fails if the tree with the specified root does not exits in the VM's advice provider. |
-| insert      | Inserts the specified value under the specified key in a Sparse Merkle Tree defined by the specified root. If the insert is successful, the old value located under the specified key is returned via the stack.<br /><br />This procedure requires that `VALUE` be a non-empty word.<br /><br />Inputs: `[VALUE, KEY, ROOT, ...]`<br />Outputs: `[OLD_VALUE, NEW_ROOT, ...]`<br /><br />Fails if:<br />- The tree with the specified root does not exits in the VM's advice provider.<br />- The provided value is an empty word. |
diff --git a/processor/src/errors.rs b/processor/src/errors.rs
@@ -19,7 +19,6 @@ use std::error::Error;
 #[derive(Debug, PartialEq, Eq)]
 pub enum ExecutionError {
     AdviceMapKeyNotFound(Word),
-    AdviceMapValueInvalidLength(Word, usize, usize),
     AdviceStackReadFailed(u32),
     CallerNotInSyscall,
     CodeBlockNotFound(Digest),
@@ -61,6 +60,8 @@ pub enum ExecutionError {
     NotBinaryValue(Felt),
     NotU32Value(Felt, Felt),
     ProverError(ProverError),
+    SmtNodeNotFound(Word),
+    SmtNodePreImageNotValid(Word, usize),
     SyscallTargetNotInKernel(Digest),
     UnexecutableCodeBlock(CodeBlock),
     MalformedSignatureKey(&'static str),
@@ -76,13 +77,6 @@ impl Display for ExecutionError {
                 let hex = to_hex(Felt::elements_as_bytes(key))?;
                 write!(f, "Value for key {hex} not present in the advice map")
             }
-            AdviceMapValueInvalidLength(key, expected, actual) => {
-                let hex = to_hex(Felt::elements_as_bytes(key))?;
-                write!(
-                    f,
-                    "Expected value for key {hex} to contain {expected} elements, but was {actual}"
-                )
-            }
             AdviceStackReadFailed(step) => write!(f, "Advice stack read failed at step {step}"),
             CallerNotInSyscall => {
                 write!(f, "Instruction `caller` used outside of kernel context")
@@ -174,6 +168,14 @@ impl Display for ExecutionError {
                     "An operation expected a u32 value, but received {v} (error code: {err_code})"
                 )
             }
+            SmtNodeNotFound(node) => {
+                let node_hex = to_hex(Felt::elements_as_bytes(node))?;
+                write!(f, "Smt node {node_hex} not found")
+            }
+            SmtNodePreImageNotValid(node, preimage_len) => {
+                let node_hex = to_hex(Felt::elements_as_bytes(node))?;
+                write!(f, "Invalid pre-image for node {node_hex}. Expected pre-image length to be a multiple of 8, but was {preimage_len}")
+            }
             ProverError(error) => write!(f, "Proof generation failed: {error}"),
             SyscallTargetNotInKernel(proc) => {
                 let hex = to_hex(&proc.as_bytes())?;