-
Notifications
You must be signed in to change notification settings - Fork 160
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Alternatives to TSMT #1136
Comments
Do you think it makes sense to wait for that until we have a first testnet? |
My 2 cents re: complexity. I will not touch on whether applying these suggestions would make using TSMT still useful. I don't think the data structure is fundamentally too complex; i.e. it is relatively simple to explain, and understand how it works at a high level. I believe there are 2 low-hanging fruits to reduce the accidental complexity of the current implementations:
|
Perhaps the depth of the single tier TSMT could be parameterised. The depth of the TSMT would be use case specific i.e. for account vaults we would use a single tier TSMT of depth 16 (or maybe even less), for account and nullifier db's we would have a TSMT of depth 64. For account storage it would be the responsibility of the user to specify the depth of the TSMT as part of the type information.
Similarly we could paramaterise the depth of the upper tier in the 2 tier construction. I'm in favour of a single tier TSMT with parameterisable depth as through the depth configurability it gives a good balance between performance and simplicity for many use cases. |
Setting the depth much smaller than 64 could become a significant attack vector (at least if we keep colliding keys in a sorted list). The main factor is the cost of generating key collisions. I estimate that with RPO, finding a 64-bit pre-image probably costs many million dollars (probably north of $100M) and so the cost of generating more than a few hundred keys with the same 64-bits prefix as some existing key is prohibitively expensive - even in the medium term. 16-bit (and even 32-bit) pre-images are pretty easy to find though. So, this opens up various attack vectors (e.g., someone could make accessing a specific key in account storage prohibitively expensive). I'm sure these can be worked around, but it does require users to be aware of these edge-case behaviors and is probably undesirable. |
Once 0xPolygonMiden/crypto#243 is implemented, we should update This will also close #1050 and #1057. Updating
|
Closed by #1215. |
Our Tiered Sparse Merkle tree is very efficient but also is very complex - much more complex than I would like it to be (both the Rust and the MASM implementations). So, it might be good to consider alternatives, which may be not as efficient but are dramatically simpler.
First, to set the context about efficiency, inside the VM we care about two types of cycles: (1) cycles on the stack, and (2) cycles in the hasher chiplet. TSMT has roughly the following metrics (depth 64 is TBD because it is not yet implemented in MASM):
As can be seen from the above, TSMT reads take up just around 100 stack cycles, while writes are between 1.3x and 2.5x more expensive (the variability is due to the type of update). The number of hasher cycles grows with the tree depth since we need to verify longer Merkle paths.
Second, the main purpose of TSMT is to provide a way to simulate a key-value map with 4-element keys. So, whatever new construction we come up with, needs to support this.
The simplest alternative
Perhaps the simplest alternative is to reduce the number of tiers in TSMT to just one (at depth 64). This would make it almost equivalent to the SimpleSmt. The tree would work as follows:
At depth 64, we would have 3 types of leaves:
ZERO
).Both Rust and MASM implementations of these should be very straight-forward. In terms of efficiency in the VM, it would look something like this:
As can be seen from the above, number of stack cycles would actually be smaller as compared to our current TSMT, but the number of hasher cycles would increase.
In the context of Miden rollup, we care about TSMT in several contexts:
For account nullifier DBs, we expect that the trees would be loaded with many values (i.e., > 100M) and so this simpler approach would roughly double the number of cycles in the hasher.
For account storage, it is difficult to tell since there are other options which will probably be more widely used. But overall, worse performance here may not be too big of a deal.
The biggest issue though is the account vault. The main reason is that account vaults will likely contain relatively few items (e.g., less than 10K), and so the simpler approach is close to 4x worse than the current approach. To make matters worse, we update the vault 3 times during transaction execution - once in prologue, once during note/script processing, and finally during the epilogue.
So, if a transaction was consuming notes with about 100 assets, account vault manipulations alone would consume over 300K cycles (vs. about 75K cycles with the current approach). This is a substantial difference - but maybe it is still worth the simplicity.
Alternative with 2 tiers
We could also reduce the number of tiers to 2 - at depth 32 and 64 (or 16 and 64). This would not be as simple as the previous approach, but also, would be much more efficient for trees containing a small number of values.
The text was updated successfully, but these errors were encountered: