Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repo reorg fix maturin develop #14

Merged
merged 12 commits into from
Aug 15, 2023
Merged
45 changes: 28 additions & 17 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
# This file is autogenerated by maturin v1.1.0
# To update, run
#
# maturin generate-ci github
#
name: CI

on:
Expand All @@ -11,7 +6,7 @@ on:
- main
- master
tags:
- '*'
- "*"
pull_request:
workflow_dispatch:

Expand All @@ -25,16 +20,24 @@ jobs:
matrix:
target: [x86_64, x86, aarch64, armv7, s390x, ppc64le]
steps:
- uses: actions/checkout@v3
# - uses: actions/checkout@v3
# - name: Git Sumbodule Update
# run: |
# git pull --recurse-submodules
# git submodule update --remote --recursive
- name: Checkout repository and submodules
uses: actions/checkout@v2
with:
submodules: recursive
- uses: actions/setup-python@v4
with:
python-version: '3.10'
python-version: "3.10"
- name: Build wheels
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist --find-interpreter
sccache: 'true'
sccache: "true"
manylinux: auto
- name: Upload wheels
uses: actions/upload-artifact@v3
Expand All @@ -48,17 +51,20 @@ jobs:
matrix:
target: [x64, x86]
steps:
- uses: actions/checkout@v3
- name: Checkout repository and submodules
uses: actions/checkout@v2
with:
submodules: recursive
- uses: actions/setup-python@v4
with:
python-version: '3.10'
python-version: "3.10"
architecture: ${{ matrix.target }}
- name: Build wheels
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist --find-interpreter
sccache: 'true'
sccache: "true"
- name: Upload wheels
uses: actions/upload-artifact@v3
with:
Expand All @@ -71,16 +77,19 @@ jobs:
matrix:
target: [x86_64, aarch64]
steps:
- uses: actions/checkout@v3
- name: Checkout repository and submodules
uses: actions/checkout@v2
with:
submodules: recursive
- uses: actions/setup-python@v4
with:
python-version: '3.10'
python-version: "3.10"
- name: Build wheels
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist --find-interpreter
sccache: 'true'
sccache: "true"
- name: Upload wheels
uses: actions/upload-artifact@v3
with:
Expand All @@ -90,7 +99,10 @@ jobs:
sdist:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Checkout repository and submodules
uses: actions/checkout@v2
with:
submodules: recursive
- name: Build sdist
uses: PyO3/maturin-action@v1
with:
Expand All @@ -105,7 +117,6 @@ jobs:
release:
name: Release
runs-on: ubuntu-latest
if: "startsWith(github.ref, 'refs/tags/')"
needs: [linux, windows, macos, sdist]
steps:
- uses: actions/download-artifact@v3
Expand Down
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ lib/
include/
target/
maturin
python

python3
python3.11
pip
Expand All @@ -21,4 +21,5 @@ packages.png
.vscode
.devcontainer
.ipynb_checkpoints
dsl-ipynb_checkpoints
dsl-ipynb_checkpoints
*.so
19 changes: 15 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,44 @@
# Quick Start

## Setup

PyChiquito uses PyO3 and Maturin to expose Rust APIs to Python. Maturin requires the user to locally build a Python virtual environment.
Run the following script to create a Python virtual environment, install required packages, and build the project.
```

```bash
# clone this repo and its submodules
git clone --recursive https://github.com/qwang98/PyChiquito

# Create a virtual environment
python3 -m venv .env

# Activate the virtual environment
source .env/bin/activate

# Install the required packages
pip install maturin
pip install py_ecc
pip install -r requirements.txt

# Build the project
maturin develop
```

If the above doesn't work, follow the guide here: https://pyo3.rs/main/getting_started#python

## Testing with examples

Run fibonacci.py example file using the following script:

```
python3 pychiquito/fibonacci.py
python3 examples/fibonacci.py
```

If setup is correct, you should see a print out of the parsed Rust AST circuit and TraceWitness. All Halo2 and Chiquito Debug messages for generating and verifying proof should also appear in the terminal.

# Technical Design

Python front end -> Python AST object/TraceWitness -> serialize to JSON string -> pass JSON string to Rust using PyO3 -> deserialize JSON string to Chiquito AST/TraceWitness -> store AST in Rust HashMap<UUID, AST> -> pass back UUID to Python -> generate and verify proof from Python with AST UUID and TraceWitness JSON

## Notes:

- Rust bindings to expose to Python are in lib.rs
- Boilerplate functions and `Deserialize` trait implementations for Rust Chiquito AST, TraceWitness, and their sub types are in frontend.rs of Rust Chiquito: https://github.com/privacy-scaling-explorations/chiquito
14 changes: 10 additions & 4 deletions pychiquito/fibonacci.py → examples/fibonacci.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
from __future__ import annotations
from typing import Tuple

from dsl import Circuit, StepType
from cb import eq
from query import Queriable
from util import F
# from chiquito import (dsl, cb, query, util)
# from dsl import Circuit, StepType
# from cb import eq
# from query import Queriable
# from util import F

from chiquito.dsl import Circuit, StepType
from chiquito.cb import eq
from chiquito.query import Queriable
from chiquito.util import F


class Fibonacci(Circuit):
Expand Down
19 changes: 19 additions & 0 deletions examples/simple.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
from __future__ import annotations
from typing import Tuple

from chiquito.dsl import Circuit, StepType
from chiquito.cb import eq
from chiquito.query import Queriable
from chiquito.util import F
import chiquito
# from chiquito import chiquito


# def main():
print("Hello, world!")
# print(Circuit)
# print(StepType)
# print(eq)
# print(Queriable)
# print(chiquito.__all__)
print(dir(chiquito))
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,14 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 3,
"id": "bf77f66c-2e81-4b51-b7f1-24c1be6c9b99",
"metadata": {},
"outputs": [],
"source": [
"from dsl import Circuit, StepType\n",
"from cb import eq\n",
"from util import F"
"from chiquito.dsl import Circuit, StepType\n",
"from chiquito.cb import eq\n",
"from chiquito.util import F"
]
},
{
Expand Down Expand Up @@ -428,9 +428,9 @@
],
"metadata": {
"kernelspec": {
"display_name": "pychiquito_kernel",
"display_name": ".env",
"language": "python",
"name": "pychiquito_kernel"
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand Down
File renamed without changes.
6 changes: 4 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = ["maturin>=1.1,<2.0"]
build-backend = "maturin"

[project]
name = "rust_chiquito"
name = "chiquito"
requires-python = ">=3.7"
classifiers = [
"Programming Language :: Rust",
Expand All @@ -13,5 +13,7 @@ classifiers = [


[tool.maturin]
bindings = 'pyo3'
features = ["pyo3/extension-module"]
python-source = "pychiquito"
python-source = "python"
module-name = "chiquito.rust_chiquito"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this makes rust_chiquito a sub module of package chiquito?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, an example using it here
In general, chiquito.rust_chiquito module has function listed here

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that makes much more sense structure wise

File renamed without changes.
8 changes: 4 additions & 4 deletions pychiquito/cb.py → python/chiquito/cb.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
from enum import Enum, auto
from typing import List

from util import F
from expr import Expr, Const, Neg, to_expr, ToExpr
from query import StepTypeNext
from chiquito_ast import ASTStepType
from chiquito.util import F
from chiquito.expr import Expr, Const, Neg, to_expr, ToExpr
from chiquito.query import StepTypeNext
from chiquito.chiquito_ast import ASTStepType


class Typing(Enum):
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
from __future__ import annotations
from typing import Callable, List, Dict, Optional, Any, Tuple
from dataclasses import dataclass, field, asdict
# from chiquito import wit_gen, expr, query, util

from wit_gen import FixedGenContext, StepInstance
from expr import Expr
from util import uuid
from query import Queriable
from chiquito.wit_gen import FixedGenContext, StepInstance
from chiquito.expr import Expr
from chiquito.util import uuid
from chiquito.query import Queriable


# pub struct Circuit<F, TraceArgs> {
Expand Down
13 changes: 7 additions & 6 deletions pychiquito/dsl.py → python/chiquito/dsl.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
from __future__ import annotations
from enum import Enum
from typing import Callable, Any
import rust_chiquito # rust bindings
from chiquito import rust_chiquito # rust bindings
import json
from chiquito import (chiquito_ast, wit_gen)

from chiquito_ast import ASTCircuit, ASTStepType, ExposeOffset
from query import Internal, Forward, Queriable, Shared, Fixed
from wit_gen import FixedGenContext, StepInstance, TraceWitness
from cb import Constraint, Typing, ToConstraint, to_constraint
from util import CustomEncoder, F
from chiquito.chiquito_ast import ASTCircuit, ASTStepType, ExposeOffset
from chiquito.query import Internal, Forward, Queriable, Shared, Fixed
from chiquito.wit_gen import FixedGenContext, StepInstance, TraceWitness
from chiquito.cb import Constraint, Typing, ToConstraint, to_constraint
from chiquito.util import CustomEncoder, F


class CircuitMode(Enum):
Expand Down
2 changes: 1 addition & 1 deletion pychiquito/expr.py → python/chiquito/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from typing import List
from dataclasses import dataclass

from util import F
from chiquito.util import F


# pub enum Expr<F> {
Expand Down
2 changes: 1 addition & 1 deletion pychiquito/query.py → python/chiquito/query.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from __future__ import annotations

from expr import Expr
from chiquito.expr import Expr

# Commented out to avoid circular reference
# from chiquito_ast import InternalSignal, ForwardSignal, SharedSignal, FixedSignal, ASTStepType
Expand Down
File renamed without changes.
4 changes: 2 additions & 2 deletions pychiquito/wit_gen.py → python/chiquito/wit_gen.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
from typing import Dict, List, Callable, Any
import json

from query import Queriable, Fixed
from util import F, CustomEncoder
from chiquito.query import Queriable, Fixed
from chiquito.util import F, CustomEncoder

# Commented out to avoid circular reference
# from dsl import Circuit, StepType
Expand Down
40 changes: 40 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
appnope==0.1.3
asttokens==2.2.1
backcall==0.2.0
cached-property==1.5.2
comm==0.1.4
cytoolz==0.12.2
debugpy==1.6.7.post1
decorator==5.1.1
eth-hash==0.5.2
eth-typing==3.4.0
eth-utils==2.2.0
executing==1.2.0
ipykernel==6.25.1
ipython==8.14.0
jedi==0.19.0
jupyter_client==8.3.0
jupyter_core==5.3.1
matplotlib-inline==0.1.6
maturin==1.2.0
mypy-extensions==1.0.0
nest-asyncio==1.5.7
packaging==23.1
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
platformdirs==3.10.0
prompt-toolkit==3.0.39
psutil==5.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
py-ecc==6.0.0
Pygments==2.16.1
python-dateutil==2.8.2
pyzmq==25.1.1
six==1.16.0
stack-data==0.6.2
toolz==0.12.0
tornado==6.3.2
traitlets==5.9.0
wcwidth==0.2.6
Loading