Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): print protocol circuits gate counts #6832

Closed
wants to merge 16 commits into from
16 changes: 16 additions & 0 deletions .github/scripts/merge-gates-reports.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash
set -eu

combined_reports='{"programs": []}'

# Iterate over each report and merge them
for report in ./reports/*; do
# The report is saved under ./$REPORT_NAME_{ matrix_report }/$REPORT_NAME_{ matrix_report }.json
FILE_PATH=$(echo $(ls $report))

# Extract the $NAME_PLURAL array from each report and merge it
combined_reports=$(jq --argjson COMBINED_REPORTS "$combined_reports" '.programs += $COMBINED_REPORTS.programs' <<< "$(cat "$report/$FILE_PATH")")
done

echo "$combined_reports"

12 changes: 12 additions & 0 deletions .github/scripts/single_artifact_gates_report.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
set -e

BACKEND=${BACKEND:-bb}

ARTIFACT_PATH=$1
ARTIFACT_NAME=$(basename "$ARTIFACT_PATH")

GATES_INFO=$($BACKEND gates -b $ARTIFACT_PATH)
MAIN_FUNCTION_INFO=$(echo $GATES_INFO | jq -r '.functions[0] | .name = "main" | del(.gates_per_opcode)')
echo "{\"programs\": [ {\"package_name\": \"$ARTIFACT_NAME\", \"functions\": [$MAIN_FUNCTION_INFO]} ]}"

68 changes: 67 additions & 1 deletion .github/workflows/reports.yml
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,8 @@ jobs:
with:
path: scripts
sparse-checkout: |
scripts/install_bb.sh
.github/scripts/single_artifact_gates_report.sh
test_programs/compilation_report.sh
sparse-checkout-cone-mode: false

Expand All @@ -352,7 +354,6 @@ jobs:
if: ${{ !matrix.project.take_average }}
run: |
mv /home/runner/work/noir/noir/scripts/test_programs/compilation_report.sh ./compilation_report.sh
chmod +x ./compilation_report.sh
./compilation_report.sh 1

- name: Generate compilation report with averages
Expand Down Expand Up @@ -412,6 +413,33 @@ jobs:
retention-days: 3
overwrite: true

# While we've got the compiled artifacts, we measure the number of gates in the circuits.

- name: Install `bb`
if: ${{ !matrix.project.is_library }}
run: |
./scripts/scripts/install_bb.sh
echo "$HOME/.bb/" >> $GITHUB_PATH

- name: Generate gates report
if: ${{ !matrix.project.is_library }}
id: gates_report
run: |
PACKAGE_NAME=${{ matrix.project.path }}
PACKAGE_NAME=$(basename $PACKAGE_NAME)
./scripts/.github/scripts/single_artifact_gates_report.sh ./test-repo/${{ matrix.project.path }}/target/*.json > gates_report_$PACKAGE_NAME.json
cat gates_report_$PACKAGE_NAME.json
echo "gates_report_name=$PACKAGE_NAME" >> $GITHUB_OUTPUT

- name: Upload gates report
if: ${{ !matrix.project.is_library }}
uses: actions/upload-artifact@v4
with:
name: gates_report_${{ steps.gates_report.outputs.gates_report_name }}
path: gates_report_${{ steps.gates_report.outputs.gates_report_name }}.json
retention-days: 3
overwrite: true

upload_compilation_report:
name: Upload compilation report
needs: [generate_compilation_and_execution_report, external_repo_compilation_and_execution_report]
Expand Down Expand Up @@ -456,6 +484,44 @@ jobs:
header: compilation
message: ${{ steps.compilation_report.outputs.markdown }}

upload_gates_report:
name: Upload gates report
needs: [external_repo_compilation_report]
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
# We want this job to run even if one variation of the matrix in `external_repo_compilation_report` fails
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
if: always()
runs-on: ubuntu-latest
permissions:
pull-requests: write

steps:
- uses: actions/checkout@v4

- name: Download matrix gates reports
uses: actions/download-artifact@v4
with:
pattern: gates_report_*
path: ./reports

- name: Merge gates reports using jq
run: |
mv ./.github/scripts/merge-gates-reports.sh merge-gates-reports.sh
./merge-gates-reports.sh > protocol_circuits_gates_report.json
cat protocol_circuits_gates_report.json

- name: Compare gates reports
id: gates_diff
uses: noir-lang/noir-gates-diff@10f066ecad44098afef183327a13b4a7c4189ae6
with:
report: protocol_circuits_gates_report.json
summaryQuantile: 0.9 # only display the 10% most significant circuit size diffs in the summary (defaults to 20%)

- name: Add memory report to sticky comment
if: github.event_name == 'pull_request' || github.event_name == 'pull_request_target'
uses: marocchino/sticky-pull-request-comment@v2
with:
header: protocol_gates
message: ${{ steps.gates_diff.outputs.markdown }}

external_repo_memory_report:
needs: [build-nargo]
runs-on: ubuntu-latest
Expand Down
Loading