Add option for printing the number of QuickCheck tests and shrinks #431
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@jorisdral I'm keen to avoid adding more command-line options. The interface is already quite cluttered. I'd prefer the following behaviour to be implemented unconditionally: firstly show progress from 0% to 100% while testing ( |
|
@jorisdral what do you think about my suggestion above? No promises, but we might be making a release soon; it would be great if this improvement lands before that. |
|
@Bodigrim I think your suggestion makes sense. I'll see if I can adjust the PR soon |
jorisdral
force-pushed
the
jdral/show-qc-tests-shrinks
branch
from
968bfe4 to
cc442ff
Compare
|
@Bodigrim I've updated the PR. Let me know if this is what you had in mind |
Bodigrim
reviewed
Nov 4, 2024
Currently, `tasty` and `tasty-quickcheck` will print a progress percentage, so one can see test progression. However, once a property finds a failure and starts shrinking, then test progression stops. Importantly, it is not clear how the shrinker is progressing, though it would be useful information to show, e.g., to judge whether a test is shrinking too slow, whether the shrinker loops, or whether a shrunk test case hangs. This commit changes the progress bar to print the number of shrinks in case of test failures. Succesful tests will still print the progress percentage.
jorisdral
force-pushed
the
jdral/show-qc-tests-shrinks
branch
from
cc442ff to
d2d951a
Compare
Bodigrim
approved these changes
Nov 5, 2024
|
Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently,
tastyandtasty-quickcheckwill print a progress percentage, so one can see test progression. However, once a property finds a failure and starts shrinking, then test progression stops. Importantly, it is not clear how the shrinker is progressing, though it would be useful information to show, e.g., to judge whether a test is shrinking too slow, whether the shrinker loops, or whether a shrunk test case hangs. This commit adds a new option to enable printing the number of QuickCheck tests (and shrinks on test failure) in addition to the percentage.I know this feature has been suggested before here: #419 (comment)). In my opinion it would be a generally useful feature to include, and it's optional so that it does not use up too much UI space by default. Let me know what you think!