Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: nargo test -q (or nargo test --format terse) #6776

Merged
merged 9 commits into from
Dec 12, 2024

Conversation

asterite
Copy link
Collaborator

@asterite asterite commented Dec 11, 2024

Description

Made on top of #6728

Problem

Towards #6733

Summary

Eventually we'll have nargo test --format json, which is what #6733. As a first step I wanted to introduce the format flag and, to try that abstraction in code I introduced a terse formatter similar to Rust.

On failure we don't immediately print the error. Instead, it's printed at the end (like in Rust). I can imagine this could be used in CI to reduce the output verboseness, because I guess all that matters in CI is seeing which tests failed. NOTE: I already added this -q flag to our external checks, you can see how it looks, I think it's a bit better as it's less noisy.

nargo-test-q

Like in Rust, there are two ways to trigger this terse behavior: -q (or --quiet) or --format terse. We could just keep --format terse but I think -q is convenient (and format is more general as it will soon support json).

Here's the output when all tests pass:

image

Additional Context

  1. The proptest output that shows up in the middle of the run will be gone with fix: disable failure persistance in nargo test fuzzing #6777
  2. I just noticed that we have duplicate and now outdated code to run stdlib tests here. We've been able to run nargo test inside noir_stdlib for a while now so we can probably move those to a CI test matrix (in this PR I tried to reuse the code from nargo_cli but I couldn't because nargo_cli is a binary and we can't use that binary as a dependency in test integrations).

Documentation

Check one:

  • No documentation needed.
  • Documentation included in this PR.
  • [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

Copy link
Contributor

github-actions bot commented Dec 11, 2024

Compilation Sample

Program Compilation Time %
sha256_regression 0m1.439s -4%
regression_4709 0m0.737s -6%
ram_blowup_regression 0m17.030s 1%
private-kernel-tail 0m1.292s 7%
private-kernel-reset 0m9.348s 1%
private-kernel-inner 0m2.803s 10%
parity-root 0m1.166s 33%
noir-contracts 2m45.216s -1%

Copy link
Contributor

github-actions bot commented Dec 11, 2024

Peak Memory Sample

Program Peak Memory
keccak256 79.14M
workspace 122.04M
regression_4709 295.98M
ram_blowup_regression 2.44G
private-kernel-tail 210.43M
private-kernel-reset 862.21M
private-kernel-inner 307.94M
parity-root 175.78M

@asterite asterite force-pushed the ab/nargo-test-machine-readable-output branch from a4d7df8 to 94aabde Compare December 11, 2024 19:59
Base automatically changed from ab/nargo-test-improvements to master December 12, 2024 02:10
@asterite asterite force-pushed the ab/nargo-test-machine-readable-output branch from 94aabde to df37a64 Compare December 12, 2024 09:36
@asterite asterite added the run-external-checks Trigger CI job to run tests on external repos label Dec 12, 2024
@asterite asterite force-pushed the ab/nargo-test-machine-readable-output branch from fd0e854 to 04a53ea Compare December 12, 2024 10:23
@asterite asterite requested a review from a team December 12, 2024 10:33
@asterite asterite marked this pull request as ready for review December 12, 2024 12:13
Copy link
Contributor

@aakoshh aakoshh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, nice to see it broken up!

@aakoshh
Copy link
Contributor

aakoshh commented Dec 12, 2024

It would be nice to change this to contain the path like this one does.

Curiously even though it fails on another PR that one has a green status. Maybe we don't want to be blocked by other repos checking in broken code?

@asterite
Copy link
Collaborator Author

It would be nice to change this to contain the path like this one does.

Done!

@aakoshh aakoshh added this pull request to the merge queue Dec 12, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 12, 2024
@asterite asterite added this pull request to the merge queue Dec 12, 2024
Merged via the queue into master with commit 919149d Dec 12, 2024
79 of 81 checks passed
@asterite asterite deleted the ab/nargo-test-machine-readable-output branch December 12, 2024 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run-external-checks Trigger CI job to run tests on external repos
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants