feat: Body Parsing With Audit Fixes (#59)

* feat: email_auth body parsing circuit working with tests * fix: minor test changes * chore: update dependencies * (wip) chore: pass github workflow * Add a verifier contract * Update proving key url * feat: add body parsing * chore: update * chore: separated body parsing tests * fix: refactring in recipent_enabled test * fix: minor * chore: update relayer * fix: updated command regex * chore: changed max_command_bytes to 605 * Update verifier * feat: sha precompute test * chore: update relayer utils version * feat: command update + relayer changes * Fix circuit name in core.py * chore: update version * chore: update circuit test * chore: update relayer-uitls dep * Update verifier * Change the max header/body sizes in the body-parsing circuits. * Add test circuits for body parsing * Update verifier * Fixing integration test * Update emails for integration tests * Update yarn.lock * Fix scripts and email for integration test * chore: update circuit test * Integration test worked * fix: body parsing test * feat: complete flow * Replace subject with command. * Fix compile errors in relayer * Remove skipped_command_bytes * Merge audit-fix into body-parsing * Add IGroth16Verifier * Recover skipped_commad_bytes * Apply audit-fixes to the body-parsing circuit. * Fix relayer * Disable auto script tests in contracts * Update verifier and proving key * Feat/fix zksync impl (#60) * Fix yarn zkbuild * Fix yarn zktest * Fix integration test except for eml files * comment out specific zksync lines * Update test emails for zksync * Update integration test for zksync * Update bytecode hash * Update proof.json path * Update publicInputFile path * Update bytecode hash * Update docs for integration tests for zksync * Update package.json in contracts * Let functions in EmailAccountRecoveryZKSync be virtual * fix: subject template parsing * Add requestGuardian to SimpleWallet * Fix scripts * Fix scripts * Update READMEs * Fix circuit package.json * Fix typo and remove unnecessary file * Remove space from invitation code * chore: refactor * WIP: Add custom errors * Update .env.example * Split up handle_email fn; Remove code duplication * Custom errors for email * Rebase solve conflicts of body-parser refacor * Rebase solve conflicts of body-parser refacor * Rebase fixes * Acknowledgement as replies in email thread * Add Re: to subject for reply emails * Update email templates UI * Add cloud build to build on gcp * Bump relayer version * fix: add command in email templates * feat: add CONTRIBUTING.md & CODING_GUIDELINES.md * chore: move md files * feat: add github workflows * chore: remove abis * chore: update github actions and add PR template * fix: github action * Remove hard-coded file_paths; Fix clippy warnings * Move util files from ether-email-auth to utils repo * Fix relayer-utils version to refactor commit * Force db initialization on startup * AI generated doc comments * Prevent duplicates by checking if it is a valid reply * Add AI generated comments * Rename func to extract_template_vals_from_command * Exclude abi folder as workspace member * Ignore abis for fmt * Add github actions to build docker image * ui: Update email template UI - powered by zk email is odd that theyre different sizes - the social icons at the bottom look squished - Update the email template from The Zk team Aayush to The ZK Email Team - Check vertical padding in the mail * feat: GPU prover (#66) * feat: use GPU for proving * chore: update Dockerfile * chore: cleanup * chore: update params.zip * chore: update circom_proofgen.sh * chore: update * fix: regex json path * chore: update k8s manifest * feat: use GPU for proving * chore: add comment in example env * feat: keep modal warm * ui: update email template design * chore: cleanup * chore: update dockerignore * chore: update dockerignore * chore: update k8s manifest * Add an invalid utf8 test email * Update circom_tester * Fix circom version * Fix circom version * Make test cases as a function named files * Add test cases for DKIM registries * Add test cases for EmailAccountRecovery * Add general test cases * Update proxy bytecode hash * Add more events * Fix circom version to 2.1.9 in github action * audit fixes * Fix email_auth_with_recipient * Add send_email.py for test * Add upgrade test * Update circom tests * Fix recipient test * Fix circom import in email_auth.circom * Fix arg name in removePrefix * Update dependencies * Fix interfaces to call EmailVerifier. * Change the regex of forced_subject * change the positon dependency in forced_subject * Add "ZK" to the relayer's email subjects * Fix force_subject regex error * Integration test on base worked. * update version * Update integration test emails * Fix zksync intergation test * Not hardcode proxyBytecodeHash in EmailAccountRecoveryZKSync * Update prover * Update README and call args for RecoveryControllerZKSync * Change forced_subject to accept "Re: " * Update zk-email circuits/contracts versions * Add RequestGuardianSimpleWallet script * add test_forced_subject_regex * Update test emails for integration tests * Remove console from EmailAuth * remove console.log * Use PROXY_BYTECODE_HASH in .env * Use default bytes32 value if PROXY_BYTECODE_HASH is not set * if PROXY_BYTECODE_HASH it not set the script will revert * Update verifier * Update versions * relayer update * Update prover with gpus * Update prover * chore: node version >=18 * chore: prover * Revert "Update prover" This reverts commit 9b3e29c. * Revert "Update prover with gpus" This reverts commit b00f205. * Update prover version * chore: change gpu * Update dkim registries * Update integration test * Update scripts to use useroverriable registry * E2E test with user overridable dkim registry worked. * Remove DeployCommonScript * Fix integration test for foundry * Fix IntegrationZKSync * Update dependency * Update yarn.lock * chore: update * chore: update version & Cargo.lock * Remove ForwardDKIMRegistry and Add scripts for UseroverridableDKIMRegistry * Fix relayer * Remove ForwardDKIMRegistry.sol * Update yarn.lock * Update yarn.lock * Fix clippy bug * Fix integration test for zksync era-test-node * Remove code which relates to ForwardDKIMRegistry * Abstract Verifier as IVerifier * Update test signature for test_Dfinity_Oracle_Response * Fix script test errors * chore: update * Update circuit depndencies * chore: update version * downgrade typescript * Format * Remove setEnv from DeployRecoveryController * Specify commits in zk-email-verify * Update version * Remove testRequestGuardianNotOwner and add verifier test case for upgrading * Fix test_Revert_IfSignatureIsInvalid * Update zk-email/contracts repo hash * Update zk-regex and circuit tests * update zk-email/circuit and contract * Update verifier and integration tests * Update prover * Use wasm&cpu for the prover temporarily. * Fix chain.rs for updates of Useroverridable dkim registry * Fix the forced subject * Update versions * Add gas limit param (#75) * Send cycles to call the dkim canister function * remove warning * fix warning * Add string utils to ether-email-auth (#76) * Add string utils to ether-email-auth * Install solidity string utils * Feat/audit fix 2024 09 gpus (#77) * update prover * Fix prover * update local_setup.sh * Update circom_proofgen.sh * Disable logger temporary * Fix dockerfile and scripts * Update zk-email/contracts * Fix script * Feat/deploy via oz defender (#73) * forge install: openzeppelin-foundry-upgrades v0.3.6 * forge install: openzeppelin-contracts-upgradeable v5.1.0 * Test OZ Defender WIP * Add package.json * Add unsafeSkipAllChecks * Remove @openzeppelin/upgrades-core * remove OZ upgradeable submodule * Finish updating scripts * Clean up * Remove vm.setEnv from script * Remove use of setEnv in deployment script --------- Co-authored-by: wshino <[email protected]> Co-authored-by: SoraSuegami <[email protected]> * update version * Use the address from deployerPrivateKey if no initialOwner is provided. * Fix BaseDeployScript * Update integration test * Feat/improve alerts (#72) * Send mail if contract call failed * Apply cargo fmt * Add logic to mail a user error email to system user's address * Revert send_email function to private * Add more info for error in recover function * Add PROVER_ADDRESS to yaml files for kubernetes * Fix env name * Add env * Remove duplicate tr * Apply cargo fmt * Fix cargo clippy warnings * Add metrics and alerts as file * Add support section (#78) * Feat/add emergency docs (#71) * Add doc for emergency * Rename emergency.md -> upgrade.md * Feat/e2e zksync (#79) * Update missing libraries in README.md and remove defender import * Remove OZ Defender dependencies * remove console in StringUtils * Import StringUtils in CommandUtils * Feat/fix zksync commands (#80) * Update testcases which uses era-test-node * Fix LLVM problems * Update foundry.toml * Update zkbuild and zktest commands * Update README.md * Add install era-test-node in README.md * Export a circuit with body parsing and the recipient feature. * Push the missing circuit * Add one more circuit test * Separate tests for the recipient --------- Co-authored-by: shreyas-londhe <[email protected]> Co-authored-by: Aditya Bisht <[email protected]> Co-authored-by: Aditya Bisht <[email protected]> Co-authored-by: wshino <[email protected]> Co-authored-by: Yush G <[email protected]> Co-authored-by: Dimitri <[email protected]> Co-authored-by: Shubham Gupta <[email protected]> Co-authored-by: John Guilding <[email protected]>
zkemail · Nov 3, 2024 · fac4274 · fac4274
1 parent e351186
commit fac4274
Show file tree

Hide file tree

Showing 216 changed files with 22,342 additions and 15,192 deletions.
diff --git a/.dockerignore b/.dockerignore
@@ -26,6 +26,7 @@ packages/contracts/test/build_integration/*.json
 packages/contracts/test/build_integration/*.zkey
 packages/contracts/test/build_integration/*.wasm
 packages/contracts/test/build_integration/*.txt
+packages/contracts/test/EmailAccountRecoveryZkSync
 
 # NFT Relayer
 packages/nft_relayer/sendgrid.env

diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -0,0 +1,28 @@
+## Description
+<!-- Please include a summary of the change and which issue is fixed. Also, list any dependencies that are required for this change. -->
+
+<!-- Fixes # (issue) -->
+
+## Type of change
+
+- [ ] Bug fix (non-breaking change which fixes an issue)
+- [ ] New feature (non-breaking change which adds functionality)
+- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
+- [ ] This change requires a documentation update
+
+## How Has This Been Tested?
+<!-- Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration -->
+
+- [ ] Test A
+- [ ] Test B
+
+## Checklist:
+
+- [ ] My code follows the style guidelines of this project
+- [ ] I have performed a self-review of my own code
+- [ ] I have commented my code, particularly in hard-to-understand areas
+- [ ] I have made corresponding changes to the documentation
+- [ ] My changes generate no new warnings
+- [ ] I have added tests that prove my fix is effective or that my feature works
+- [ ] New and existing unit tests pass locally with my changes
+- [ ] Any dependent changes have been merged and published in downstream modules
diff --git a/.github/workflows/build-fmt.yml b/.github/workflows/build-fmt.yml
@@ -0,0 +1,64 @@
+name: Build and Format
+
+on: [push]
+
+jobs:
+  build-and-format:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+
+      - run: rustup show
+
+      - name: Install rustfmt and clippy
+        run: |
+          rustup component add rustfmt
+          rustup component add clippy
+
+      - uses: Swatinem/rust-cache@v2
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v3
+        with:
+          node-version: 18
+          cache: "yarn"
+
+      - name: Install dependencies
+        run: yarn install --frozen-lockfile
+
+      - name: Install Foundry
+        uses: foundry-rs/foundry-toolchain@v1
+        with:
+          version: nightly-0079a1146b79a4aeda58b0258215bedb1f92700b
+
+      - name: Build contracts
+        working-directory: packages/contracts
+        run: yarn build
+
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          # this might remove tools that are actually needed,
+          # if set to "true" but frees about 6 GB
+          tool-cache: false
+
+          # all of these default to true, but feel free to set to
+          # "false" if necessary for your workflow
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: true
+          swap-storage: true
+
+      - name: Build and check for warnings
+        env:
+          RUSTFLAGS: "-D warnings"
+        run: cargo build --release
+
+      - name: Check formatting
+        run: cargo fmt -- --check
+
+      - name: Run clippy
+        run: cargo clippy -- -D warnings
diff --git a/.github/workflows/build-img.yml b/.github/workflows/build-img.yml
@@ -0,0 +1,51 @@
+name: Build and Push Docker Image
+
+on:
+  push:
+    branches:
+      - refactor
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository }}
+
+jobs:
+  build-and-push:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+
+      - name: Log in to the Container registry
+        uses: docker/login-action@v2
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          tags: |
+            type=sha,prefix=
+            type=raw,value=latest
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v4
+        with:
+          context: .
+          file: ./Full.Dockerfile
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
diff --git a/.github/workflows/build-test-fmt.yml b/.github/workflows/build-test-fmt.yml
diff --git a/.github/workflows/unit-tests.yml b/.github/workflows/unit-tests.yml
@@ -1,4 +1,4 @@
-name: unit-tests
+name: Unit Tests
 
 on: [push]
 
@@ -21,8 +21,8 @@ jobs:
           override: true
           components: rustfmt, clippy
 
-      - name: Download circom (Linux)
-        run: git clone https://github.com/iden3/circom.git && cd circom && cargo build --release && cargo install --path circom
+      - name: Download circom v2.1.9 (Linux)
+        run: wget https://github.com/iden3/circom/releases/download/v2.1.9/circom-linux-amd64 -O /usr/local/bin/circom && chmod +x /usr/local/bin/circom
 
       - name: Print circom version
         run: circom --version
@@ -47,7 +47,7 @@ jobs:
         uses: actions/setup-node@v3
         with:
           node-version: 18
-      
+
       - name: Install yarn
         run: npm install -g yarn
 
@@ -62,3 +62,37 @@ jobs:
       - name: Run tests
         working-directory: packages/contracts
         run: yarn test
+
+      - name: Run script tests
+        working-directory: packages/contracts
+        run: yarn test:script
+
+  relayer:
+    name: relayer
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Install Node.js
+        uses: actions/setup-node@v3
+        with:
+          node-version: 18
+
+      - name: Install yarn
+        run: npm install -g yarn
+
+      - name: Install dependencies
+        run: yarn install --frozen-lockfile
+
+      - name: Install Foundry
+        uses: foundry-rs/[email protected]
+        with:
+          version: nightly-0079a1146b79a4aeda58b0258215bedb1f92700b
+
+      - name: Build contracts
+        working-directory: packages/contracts
+        run: yarn build
+
+      - name: Run tests
+        working-directory: packages/relayer
+        run: cargo test
diff --git a/.gitignore b/.gitignore
@@ -43,6 +43,7 @@ packages/nft_relayer/logs
 sql_database.db
 .sqlx
 .ic.pem
+config.json
 
 # Relayer
 packages/relayer/sendgrid.env
@@ -57,6 +58,10 @@ sql_database.db
 .sqlx
 .ic.pem
 
+# ABIs
+packages/relayer/src/abis/*
+!packages/realyer/src/abis/mod.rs
+
 # Prover
 packages/prover/build/*
 packages/prover/params/*.zkey
@@ -80,6 +85,9 @@ book
 # Vs code settings
 .vscode
 
+# Editor settings
+.idea
+
 # For zksync
 zkout
 .cache