Mopro wasm halo2 #254

sifnoc · 2024-11-07T07:35:31Z

To enable multithreading in WASM, wasm-bindgen-rayon must be used.

Usage with WebAssembly
By default, when building to WebAssembly, Rayon will treat it as any other platform without multithreading support and will fall back to sequential iteration. This allows existing code to compile and run successfully with no changes necessary, but it will run slower as it will only use a single CPU core.

from: Rayon - github

Currently, wasm-bindgen-rayon version 1.2.1 can only be compiled with the nightly rust, specifically nightly-2024-07-18

Adding test case on test-e2e based on js
- Perform the test case in headless with selenium driver
- Update GH action for test-e2e/web in headless way
Update related documentations, README and doc
Merge Feat: support wasm on prove/verify funcs sifnoc/plonkish-fibonacci-sample#4

Related issue: #202

vivianjeng · 2024-11-26T08:00:46Z

I followed the wasm-pack tutorial
and found that wasm-bindgen can be done with stable toolchain
see this commit and CI result
(the feature flags should be fixed)

but the commit is built for nodejs, not sure if we can have both nodejs and web targets

Nov 27th update:
In order to support wasm-bindgen-rayon, we should keep mopro-wasm folder and only have web target

cloudflare-workers-and-pages · 2024-11-27T17:46:59Z

Deploying mopro with Cloudflare Pages

Latest commit:	`63e90b9`
Status:	✅ Deploy successful!
Preview URL:	https://90c087f1.mopro.pages.dev
Branch Preview URL:	https://mopro-wasm.mopro.pages.dev

View logs

vivianjeng · 2024-12-02T10:52:21Z

test-e2e/web/README.md

+mopro-wasm $ wasm-pack build --target web --out-dir ../test-e2e/web/halo2-plonk-fibonacci -- --features plonk
+mopro-wasm $ wasm-pack build --target web --out-dir ../test-e2e/web/halo2-hyperplonk-fibonacci -- --features hyperplonk
+mopro-wasm $ wasm-pack build --target web --out-dir ../test-e2e/web/halo2-gemini-fibonacci -- --features gemini


I am thinking can this be

wasm-pack build --target web --out-dir ../test-e2e/web/mopro -- --features plonk --features hyperplonk --features gemini

so the users can simply use

import * as mopro_wasm from './mopro/mopro_wasm.js';

and call each function with

mopro_wasm.generate_plonk_proof(...) mopro_wasm.generate_hyperplonk_proof(...) mopro_wasm.generate_gemini_proof(...) // and could be others in the future

I assumed that the user would eventually use only one type of backend—Plonk, HyperPlonk, or Gemini in Halo2—even for testing purposes. However, if you think it's better to have a single Wasm package in test-e2e/web, such as test-e2e/web/halo2-wasm-circuit, I can modify it to follow your suggestion

or maybe even not specify halo2?
I hope it can be more general for applications
so like test-e2e/web/mopro or test-e2e/web/mopro-pkg
to let users know the package is generated by mopro and in the documentation we can guide them to use

import * as mopro_wasm from './mopro/mopro_wasm.js';

to import and use functions

vivianjeng · 2024-12-02T10:57:52Z

test-e2e/web/halo2-gemini-fibonacci/README.md

+```bash
+wasm-pack test --chrome --headless -- --features plonk
+```
+
+```bash
+wasm-pack test --chrome --headless -- --features hyperplonk
+```
+
+```bash
+wasm-pack test --chrome --headless -- --features gemini
+```


same here
if you rename the function like

generate_plonk_proof(...) generate_hyperplonk_proof(...) generate_gemini_proof(...)

you can test the functions with

wasm-pack test --chrome --headless -- --all-features

FYI, this doc is simply copied from mopro-wasm during the compilation of the wasm code with wasm-pack. It will be updated

vivianjeng · 2024-12-02T11:01:16Z

test-e2e/web/test_halo2.js

+        const PROVING_KEY = await fetchBinaryFile('./halo2-hyperplonk-fibonacci/parameters/hyperplonk_fibonacci_pk.bin');
+        const VERIFYING_KEY = await fetchBinaryFile('./halo2-hyperplonk-fibonacci/parameters/hyperplonk_fibonacci_vk.bin');
+
+        const result = await halo2_hyperplonk_wasm.generate_proof(SRS_KEY, PROVING_KEY, input);


I am also thinking to print out (or show on screen) the execution time of generate_proof

Actually, I had added a feature that displayed the proof generation time alongside the test results. However, I eventually removed it because showing the generation time without any additional metadata—such as device information or circuit configuration—could lead to unmanaged benchmarking results.

If you think it's not a significant issue, I would be happy to add the feature back, either by logging the proof generation time to the console or showing it alongside the test results

I think it is interesting to benchmark different backends or different circuits
and we can update the result here https://zkmopro.org/docs/performance#halo2
(but update here is optional)

I will update halo2 wasm benchmark result in another PR. I want to add managing number thread feature in test-e2e/web

test-e2e/web/README.md

vivianjeng · 2024-12-02T11:09:49Z

mopro-wasm/src/wasm.rs

+
+#[cfg(feature = "plonk")]
+#[wasm_bindgen]
+pub fn prove_and_verify_plonk_proof() {


not sure what is the function for?
there is generate_proof and verify_proof in each file (plonk, hyperplonk, and gemini)
and there is also tests in tests/wasm.rs
maybe this file can be removed?

My bad. This file is left-over, I will remove this.

…lock

vivianjeng

It works well for me
just need to change halo2 to more general term like mopro supported proving systems
(But when mentioning rayon, we can still mention halo2)

docs/docs/mopro-wasm.md

mopro-wasm/README.md

test-e2e/web/README.md

test-e2e/web/index.html

...e/web/mopro-pkg/snippets/wasm-bindgen-rayon-9d40dbf53d170728/src/workerHelpers.no-bundler.js

test-e2e/web/README.md

…f root

… on dev tool

sifnoc · 2024-12-04T14:26:46Z

You can see how many web-worker spawned in browser from dev tool like one below.

It's updated at 63e90b9

vivianjeng

LGTM
Thank you!
It is such a big progress of mopro :)

sifnoc added 4 commits November 7, 2024 06:54

feat: added wasm exporter for halo2

55df378

chore: added gh action for testing halo2 wasm

01e7455

fix: lint

cfdde57

fix: repo updates with Cargo

ce30b43

feat: added simple web page for testing halo2 wasm code

6c3b932

sifnoc added 6 commits November 28, 2024 10:00

feat: added html style, test script with selenium driver

69f28c4

feat: add test steps for halo2-wasm-web in gh action workflow

6d4a876

fix: gh action workflow

2b541b0

fix: halo2 wasm app workflow

66fa25f

chore: updated documentation and readme

7b04f7f

Update dependency and compiled wasm circuits

d1a994c

sifnoc marked this pull request as ready for review December 2, 2024 08:46

sifnoc requested a review from vivianjeng December 2, 2024 08:46

vivianjeng reviewed Dec 2, 2024

View reviewed changes

sifnoc added 4 commits December 3, 2024 09:32

refactor: applied suggestions in PR review

f03978a

fix: using multi-thread on wasm

0a9c596

chore: update compiled wasm

1e2f0c3

fix: wasm proof & verify functions

71a61bc

sifnoc force-pushed the mopro-wasm branch from d014c43 to 71a61bc Compare December 3, 2024 14:01

sifnoc added 3 commits December 3, 2024 14:05

Merge branch 'main' into 'mopro-wasm' and resolve conflicts in Cargo.…

b403703

…lock

fix: initialize wasm instance in test_halo2.js

8c0de02

fix: test cases in workflow and local

0cc084f

sifnoc force-pushed the mopro-wasm branch from 24865f6 to 0cc084f Compare December 3, 2024 14:40

fix: check test case validity while testing on halo2 wasm

fa543ff

sifnoc requested a review from vivianjeng December 3, 2024 15:12

vivianjeng reviewed Dec 4, 2024

View reviewed changes

sifnoc force-pushed the mopro-wasm branch from 6c7ea45 to d9419d8 Compare December 4, 2024 13:01

fix: use 'lite-server' instead of 'serve' for serving files outside o…

514bd4d

…f root

sifnoc force-pushed the mopro-wasm branch from d9419d8 to 514bd4d Compare December 4, 2024 13:15

sifnoc added 2 commits December 4, 2024 13:46

updated docs and names following PR reviews

e6c0fbc

refactor: test web page and testing script for expliciting web-worker…

63e90b9

… on dev tool

sifnoc requested a review from vivianjeng December 4, 2024 14:27

vivianjeng approved these changes Dec 5, 2024

View reviewed changes

vivianjeng merged commit 5b1c721 into main Dec 5, 2024
33 checks passed

vivianjeng deleted the mopro-wasm branch December 5, 2024 11:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mopro wasm halo2 #254

Mopro wasm halo2 #254

sifnoc commented Nov 7, 2024 •

edited

Loading

vivianjeng commented Nov 26, 2024 •

edited

Loading

cloudflare-workers-and-pages bot commented Nov 27, 2024 •

edited

Loading

vivianjeng Dec 2, 2024

sifnoc Dec 2, 2024

vivianjeng Dec 2, 2024

vivianjeng Dec 2, 2024

sifnoc Dec 2, 2024

vivianjeng Dec 2, 2024

sifnoc Dec 2, 2024

vivianjeng Dec 2, 2024

sifnoc Dec 3, 2024

vivianjeng Dec 2, 2024

sifnoc Dec 2, 2024

vivianjeng left a comment

sifnoc commented Dec 4, 2024

vivianjeng left a comment

Mopro wasm halo2 #254

Mopro wasm halo2 #254

Conversation

sifnoc commented Nov 7, 2024 • edited Loading

vivianjeng commented Nov 26, 2024 • edited Loading

cloudflare-workers-and-pages bot commented Nov 27, 2024 • edited Loading

Deploying mopro with Cloudflare Pages

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vivianjeng left a comment

Choose a reason for hiding this comment

sifnoc commented Dec 4, 2024

vivianjeng left a comment

Choose a reason for hiding this comment

sifnoc commented Nov 7, 2024 •

edited

Loading

vivianjeng commented Nov 26, 2024 •

edited

Loading

cloudflare-workers-and-pages bot commented Nov 27, 2024 •

edited

Loading