Merge pull request #1101 from 0xPolygonMiden/bobbin-v0.7-release-prep

v0.7 release prep
0xPolygonMiden · Oct 11, 2023 · cf698d6 · cf698d6
2 parents 85b6c80 + 1be3caa
commit cf698d6
Show file tree

Hide file tree

Showing 15 changed files with 161 additions and 83 deletions.
diff --git a/README.md b/README.md
@@ -31,7 +31,7 @@ Miden VM is a fully-featured virtual machine. Despite being optimized for zero-k
 * **Cryptographic operations.** Miden assembly provides built-in instructions for computing hashes and verifying Merkle paths. These instructions use the Rescue Prime Optimized hash function (which is the native hash function of the VM).
 * **External libraries.** Miden VM supports compiling programs against pre-defined libraries. The VM ships with one such library: Miden `stdlib` which adds support for such things as 64-bit unsigned integers. Developers can build other similar libraries to extend the VM's functionality in ways which fit their use cases.
 * **Nondeterminism**. Unlike traditional virtual machines, Miden VM supports nondeterministic programming. This means a prover may do additional work outside of the VM and then provide execution *hints* to the VM. These hints can be used to dramatically speed up certain types of computations, as well as to supply secret inputs to the VM.
-* **Custom advice providers.** Miden VM can be instantiated with user-defined advice providers. These advice providers are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls).
+* **Customizable hosts.** Miden VM can be instantiated with user-defined hosts. These hosts are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls).
 
 #### Planned features
 In the coming months we plan to finalize the design of the VM and implement support for the following features:
@@ -78,41 +78,58 @@ A few general notes on performance:
 * Both proof generation and proof verification times are greatly influenced by the hash function used in the STARK protocol. In the benchmarks below, we use BLAKE3, which is a really fast hash function.
 
 ### Single-core prover performance
-When executed on a single CPU core, the current version of Miden VM operates at around 10 - 15 KHz. In the benchmarks below, the VM executes a [Fibonacci calculator](miden/README.md#fibonacci-calculator) program on Apple M1 Pro CPU in a single thread. The generated proofs have a target security level of 96 bits.
+When executed on a single CPU core, the current version of Miden VM operates at around 20 - 25 KHz. In the benchmarks below, the VM executes a [Fibonacci calculator](miden/README.md#fibonacci-calculator) program on Apple M1 Pro CPU in a single thread. The generated proofs have a target security level of 96 bits.
 
 | VM cycles       | Execution time | Proving time | RAM consumed  | Proof size |
 | :-------------: | :------------: | :----------: | :-----------: | :--------: |
-| 2<sup>10</sup>  |  1 ms          | 80 ms        | 20 MB         | 47 KB      |
-| 2<sup>12</sup>  |  2 ms          | 260 ms       | 52 MB         | 57 KB      |
-| 2<sup>14</sup>  |  8 ms          | 0.9 sec      | 240 MB        | 66 KB      |
-| 2<sup>16</sup>  |  28 ms         | 4.6 sec      | 950 MB        | 77 KB      |
-| 2<sup>18</sup>  |  85 ms         | 15.5 sec     | 3.7 GB        | 89 KB      |
-| 2<sup>20</sup>  |  310 ms        | 67 sec       | 14 GB         | 100 KB     |
+| 2<sup>10</sup>  |  1 ms          | 60 ms        | 20 MB         | 46 KB      |
+| 2<sup>12</sup>  |  2 ms          | 180 ms       | 52 MB         | 56 KB      |
+| 2<sup>14</sup>  |  8 ms          | 680 ms       | 240 MB        | 65 KB      |
+| 2<sup>16</sup>  |  28 ms         | 2.7 sec      | 950 MB        | 75 KB      |
+| 2<sup>18</sup>  |  81 ms         | 11.4 sec     | 3.7 GB        | 87 KB      |
+| 2<sup>20</sup>  |  310 ms        | 47.5 sec     | 14 GB         | 100 KB     |
 
 As can be seen from the above, proving time roughly doubles with every doubling in the number of cycles, but proof size grows much slower.
 
 We can also generate proofs at a higher security level. The cost of doing so is roughly doubling of proving time and roughly 40% increase in proof size. In the benchmarks below, the same Fibonacci calculator program was executed on Apple M1 Pro CPU at 128-bit target security level:
 
 | VM cycles       | Execution time | Proving time | RAM consumed  | Proof size |
 | :-------------: | :------------: | :----------: | :-----------: | :--------: |
-| 2<sup>10</sup>  | 1 ms           | 300 ms       | 30 MB         | 61 KB      |
-| 2<sup>12</sup>  | 2 ms           | 590 ms       | 106 MB        | 78 KB      |
-| 2<sup>14</sup>  | 8 ms           | 1.7 sec      | 500 MB        | 91 KB      |
-| 2<sup>16</sup>  | 28 ms          | 6.7 sec      | 2.0 GB        | 106 KB     |
-| 2<sup>18</sup>  | 85 ms          | 27.5 sec     | 8.0 GB        | 122 KB     |
-| 2<sup>20</sup>  | 310 ms         | 126 sec      | 24.0 GB       | 138 KB     |
+| 2<sup>10</sup>  | 1 ms           | 120 ms       | 30 MB         | 61 KB      |
+| 2<sup>12</sup>  | 2 ms           | 460 ms       | 106 MB        | 77 KB      |
+| 2<sup>14</sup>  | 8 ms           | 1.4 sec      | 500 MB        | 90 KB      |
+| 2<sup>16</sup>  | 27 ms          | 4.9 sec      | 2.0 GB        | 103 KB     |
+| 2<sup>18</sup>  | 81 ms          | 20.1 sec     | 8.0 GB        | 121 KB     |
+| 2<sup>20</sup>  | 310 ms         | 90.3 sec     | 20.0 GB       | 138 KB     |
 
 ### Multi-core prover performance
-STARK proof generation is massively parallelizable. Thus, by taking advantage of multiple CPU cores we can dramatically reduce proof generation time. For example, when executed on an 8-core CPU (Apple M1 Pro), the current version of Miden VM operates at around 100 KHz. And when executed on a 64-core CPU (Amazon Graviton 3), the VM operates at around 250 KHz.
+STARK proof generation is massively parallelizable. Thus, by taking advantage of multiple CPU cores we can dramatically reduce proof generation time. For example, when executed on an 8-core CPU (Apple M1 Pro), the current version of Miden VM operates at around 140 KHz. And when executed on a 64-core CPU (Amazon Graviton 3), the VM operates at around 250 KHz.
 
 In the benchmarks below, the VM executes the same Fibonacci calculator program for 2<sup>20</sup> cycles at 96-bit target security level:
 
-| Machine                        | Execution time | Proving time | Execution % |
-| ------------------------------ | :------------: | :----------: | :---------: |
-| Apple M1 Pro (8 threads)       | 310 ms         | 9.8 sec      | 3.1%        |
-| Apple M2 Max (16 threads)      | 290 ms         | 7.7 sec      | 3.6%        |
-| AMD Ryzen 9 5950X (16 threads) | 270 ms         | 10.7 sec     | 2.6%        |
-| Amazon Graviton 3 (64 threads) | 330 ms         | 3.7 sec      | 9.0%        |
+| Machine                        | Execution time | Proving time | Execution % | Implied Frequency |
+| ------------------------------ | :------------: | :----------: | :---------: | :---------------: |
+| Apple M1 Pro (16 threads)      | 310 ms         | 7.0 sec      | 4.2%        | 140 KHz           |
+| Apple M2 Max (16 threads)      | 280 ms         | 5.8 sec      | 4.5%        | 170 KHz           |
+| AMD Ryzen 9 5950X (16 threads) | 270 ms         | 10.0 sec     | 2.6%        | 100 KHz           |
+| Amazon Graviton 3 (64 threads) | 330 ms         | 3.6 sec      | 8.5%        | 265 KHz           |
+
+### Recursive proofs
+Proofs in the above benchmarks are generated using BLAKE3 hash function. While this hash function is very fast, it is not very efficient to execute in Miden VM. Thus, proofs generated using BLAKE3 are not well-suited for recursive proof verification. To support efficient recursive proofs, we need to use an arithmetization-friendly hash function. Miden VM natively supports Rescue Prime Optimized (RPO), which is one such hash function. One of the downsides of arithmetization-friendly hash functions is that they are considerably slower than regular hash functions.
+
+In the benchmarks below we execute the same Fibonacci calculator program for 2<sup>20</sup> cycles at 96-bit target security level using RPO hash function instead of BLAKE3:
+
+| Machine                        | Execution time | Proving time | Proving time (HW) |
+| ------------------------------ | :------------: | :----------: | :---------------: |
+| Apple M1 Pro (16 threads)      | 310 ms         | 94.3 sec     | 42.0 sec          |
+| Apple M2 Max (16 threads)      | 280 ms         | 75.1 sec     | 20.9 sec          |
+| AMD Ryzen 9 5950X (16 threads) | 270 ms         | 59.3 sec     |                   |
+| Amazon Graviton 3 (64 threads) | 330 ms         | 21.7 sec     | 14.9 sec          |
+
+In the above, proof generation on some platforms can be hardware-accelerated. Specifically:
+
+* On Apple M1/M2 platforms the built-in GPU is used for a part of proof generation process.
+* On the Graviton platform, SVE vector extension is used to accelerate RPO computations.
 
 ## References
 Proofs of execution generated by Miden VM are based on STARKs. A STARK is a novel proof-of-computation scheme that allows you to create an efficiently verifiable proof that a computation was executed correctly. The scheme was developed by Eli Ben-Sasson, Michael Riabzev et al. at Technion - Israel Institute of Technology. STARKs do not require an initial trusted setup, and rely on very few cryptographic assumptions.

diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md
@@ -20,6 +20,7 @@
     - [Debugging](./user_docs/assembly/debugging.md)
   - [Miden Standard Library](./user_docs/stdlib/main.md)
     - [std::collections](./user_docs/stdlib/collections.md)
+    - [std::crypto::dsa](./user_docs/stdlib/crypto/dsa.md)
     - [std::crypto::fri](./user_docs/stdlib/crypto/fri.md)
     - [std::crypto::hashes](./user_docs/stdlib/crypto/hashes.md)
     - [std::math::u64](./user_docs/stdlib/math/u64.md)

diff --git a/docs/src/assets/intro/vm_components.png b/docs/src/assets/intro/vm_components.png
diff --git a/docs/src/intro/main.md b/docs/src/intro/main.md
@@ -17,7 +17,7 @@ Miden VM is a fully-featured virtual machine. Despite being optimized for zero-k
 * **Cryptographic operations.** Miden assembly provides built-in instructions for computing hashes and verifying Merkle paths. These instructions use Rescue Prime Optimized hash function (which is the native hash function of the VM).
 * **External libraries.** Miden VM supports compiling programs against pre-defined libraries. The VM ships with one such library: Miden `stdlib` which adds support for such things as 64-bit unsigned integers. Developers can build other similar libraries to extend the VM's functionality in ways which fit their use cases.
 * **Nondeterminism**. Unlike traditional virtual machines, Miden VM supports nondeterministic programming. This means a prover may do additional work outside of the VM and then provide execution *hints* to the VM. These hints can be used to dramatically speed up certain types of computations, as well as to supply secret inputs to the VM.
-* **Custom advice providers.** Miden VM can be instantiated with user-defined advice providers. These advice providers are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls).
+* **Customizable hosts.** Miden VM can be instantiated with user-defined hosts. These hosts are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls).
 
 ### Planned features
 In the coming months we plan to finalize the design of the VM and implement support for the following features:

diff --git a/docs/src/intro/overview.md b/docs/src/intro/overview.md
@@ -1,19 +1,20 @@
 ## Miden VM overview
 Miden VM is a stack machine. The base data type of the MV is a field element in a 64-bit [prime field](https://en.wikipedia.org/wiki/Finite_field) defined by modulus $p = 2^{64} - 2^{32} + 1$. This means that all values that the VM operates with are field elements in this field (i.e., values between $0$ and $2^{64} - 2^{32}$, both inclusive).
 
-Miden VM consists of three high-level components as illustrated below.
+Miden VM consists of four high-level components as illustrated below.
 
 ![](../assets/intro/vm_components.png)
 
 These components are:
 * **Stack** which is a push-down stack where each item is a field element. Most assembly instructions operate with values located on the stack. The stack can grow up to $2^{32}$ items deep, however, only the top 16 items are directly accessible.
 * **Memory** which is a linear random-access read-write memory. The memory is word-addressable, meaning, four elements are located at each address, and we can read and write elements to/from memory in batches of four. Memory addresses can be in the range $[0, 2^{32})$.
-* **Advice provider** which is a way for the prover to provide nondeterministic inputs to the VM. The advice provider is composed of a single *advice stack*, an *advice map*, and a *merkle store*. The *advice stack* yields elements to the VM stack; the *advice map* stores key-mapped element lists which can be pushed onto the advice stack; finally, the Merkle store contains structured Merkle tree data and serves Merkle paths to the VM.
+* **Chiplets** which are specialized circuits for accelerating certain types of computations. These include Rescue Prime Optimized (RPO) hash function, 32-bit binary operations, and 16-bit range checks.
+* **Host** which is a way for the prover to communicate with the VM during runtime. This includes responding to the VM's requests for non-deterministic inputs and handling messages sent by the VM (e.g., for debugging purposes). The requests for non-deterministic inputs are handled by the host's *advice provider*.
 
-In the future, additional components (e.g., storage, logs) may be added to the VM.
+Miden VM comes with a default implementation of the host interface (with an in-memory advice provider). However, the users are able to provide their own implementations which can connect the VM to arbitrary data sources (e.g., a database or RPC calls) and define custom logic for handling events emitted by the VM.
 
 ## Writing programs
-Our goal is to make Miden VM an easy compilation target for high-level blockchain-centric languages such as Solidity, Move, Sway, and others. We believe it is important to let people write programs in the languages of their choice. However, compilers to help with this have not been developed yet. Thus, for now, the primary way to write programs for Miden VM is to use [Miden assembly](../user_docs/assembly/main.md).
+Our goal is to make Miden VM an easy compilation target for high-level languages such as Rust, Move, Sway, and others. We believe it is important to let people write programs in the languages of their choice. However, compilers to help with this have not been developed yet. Thus, for now, the primary way to write programs for Miden VM is to use [Miden assembly](../user_docs/assembly/main.md).
 
 While writing programs in assembly is far from ideal, Miden assembly does make this task a little bit easier by supporting high-level flow control structures and named procedures.
 
@@ -29,8 +30,6 @@ The number of public inputs and outputs of a program can be reduced by making us
 
 For example, if we wanted to provide a thousand public input values to the VM, we could put these values into a Merkle tree, initialize the stack with the root of this tree, initialize the advice provider with the tree itself, and then retrieve values from the tree during program execution using `mtree_get` instruction (described [here](../user_docs/assembly/cryptographic_operations.md#hashing-and-merkle-trees)).
 
-In the future, other ways of providing public inputs and reading public outputs (e.g., storage commitments) may be added to the VM.
-
 ### Stack depth restrictions
 For reasons explained [here](../design/stack/main.md), the VM imposes the restriction that the stack depth cannot be smaller than $16$. This has the following effects: