There is a large and increasing number of libraries that enable verifiable computation with zero-knowledge succinct non-interactive arguments (zkSNARKs). Whereas the performance of zkSNARKs is well-understood in theory, various factors make it difficult to compare different proof systems without favoring some approaches over others. Evaluating the practical performance of a specific library can be difficult due to various factors, such as the underlying elliptic curve, the proof system at hand, the arithmetization supported, or due to application-specific factors such as the desired security level.
zk-Harness is a benchmarking framework that aims to address these barriers by providing a unified benchmark for standardized evaluation of existing libraries for zkSNARKS. It is designed to be easily extendable - libraries can be easily added and compared for standardized functionalities, whereas circuit developers can simply add their already developed circuit to evaluate its performance. zk-Harness provides benchmarks along the whole zkSNARK stack. The backend for mathematical operations (field & curve) is inherited from zka.lc, a project from Michele Orrù and George Kadianakis. The benchmark results are hosted at zk-bench.org.
git clone --recurse-submodules -j8 [email protected]:zkCollective/zk-Harness.git
# or
git clone [email protected]:zkCollective/zk-Harness.git
git submodule update --init --recursive
To pull the latest results run:
git submodule update --remote benchmarks
benchmarks: Directory containing the results of the benchmarks.src: Python code to run the benchmarks and parse the resultsinput: Configuration and input filesinput/circuit: Input files for each circuit, i.e., values to be used as the inputs in circuits for benchmarking.input/config: configurations for executing benchmarks for a specific framework usingzkbench
app: UI code for presenting the results.frameworks: Directory containing the harness for each framework to benchmark circuits.data: Auxilary datadata/circuits.json: Supported zkSNARK frameworksdata/math.json: Supported math libraries
scripts: Other auxiliary scripts
The current framework supports a set of payloads for each library.
| Exponentiate | SHA-256 | |
|---|---|---|
| Bellman | ✔️ (custom) | ✔️ (implementation) |
| Circom | ✔️ (custom) | ✔️ (implementation) |
| Gnark | ✔️ (custom) | ✔️ (custom) |
| Halo2-PSE | ✔️ (custom) | ✔️ (implementation) |
| Starky | ✔️ (custom) | ❌ |
| Language | Curves/Fields | Frameworks | |
|---|---|---|---|
| blstrs | Rust | BLS12-381 | |
| gnark-crypto | Go | BN254, BLS12-377, BLS12-378, BLS12-381, BLS12-387, BLS24-315, BLS24-317, BW6-761, BW6-756, BW6-633, secp256k1, stark-curve, goldilocks | gnark |
| arkworks-curves | Rust | BN254, BLS12-377, BLS12-381, MNT4-298, MNT4-753, MNT6-298, MNT6-753, Grumpkin, BW6-761, CP6-782, secp256k1, secp256r1, secp384r1, secq256k1 | arkworks |
| curve25519-dalek | Rust | Curve25519 | |
| ffjavascript | JavaScript/WASM | BN128, BLS12-381 | snarkjs |
| ffiasm | C++ | BN128, BLS12-381 | rapidsnark |
| halo2curves | Rust | BN256, Pallas, Vesta | halo2-PSE |
| pairing_ce | Rust | BN256, BLS12-381 | bellman-ce |
| pairing | Rust | jubjub, BLS12-381 | bellman |
| pasta_curves | Rust | Pallas, Vesta | halo2 |
To run the benchmarks, you will first need to follow the installation instructions.
To run all benchmarks for mathematical operations, run make math.
Run test benchmarks for end-to-end circuits, run make circuits-test.
To run all benchmarks for end-to-end circuits, run make circuits.
To keep logs for the runs you can use the tee command, e.g., make circuits 2>&1 | tee logs
zk-Harness is easily extensible and modular, which means that you should be able to integrate you own circuits with ease.
Each framework in framework/<framework_name> includes a detailed description on how to add a self-developed circuit that goes beyond the standard payloads already integrated.
See ARCHITECTURE.md
We aim to successively expand this list to further include benchmarks for other ZKP frameworks, more circuits, and recursive composition of proofs.
There are many ways in which you can contribute to the zk-Harness:
- [] Add benchmarks for circuits in an already integrated framework
- [] Integrate a new framework into the zk-Harness
- [] Integrate new math benchmarks in zkalc
- [] Run existing benchmarks to additional machines
- [] Work on any open GitHub issue
- [] Propose new visualizations for the results
- [] Propose new benchmark categories, such as recursion benchmarking
Please read the Contribution Guidelines before creating a PR or opening an issue.
zk-Harness is developed as part of the zk-Hackathon hosted by the Berkeley Center for Responsible Decentralized Intelligence. Further, zk-Harness is part of the zk-Collective.