Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test.check_exn and more unit tests #174

Closed
wants to merge 8 commits into from
Closed

Conversation

jmid
Copy link
Collaborator

@jmid jmid commented Sep 8, 2021

When writing the shrinking logs I noticed that the Test.check_exn was wrong:
It said it would throw a Test_error exception on both failures and errors.
I figured it would be nice to add a couple of unit-tests to ensure this for QCheck2.

This PR does both of these.

As a bonus, it adds a few unit tests for QCheck, including some for the int* shrinkers
to bring the unit test status closer to QCheck2.

After having realized that the shrinker's output depend on whether the tested property holds or not,
I split them up into a successful shrink trace and a failure shrink trace,
I then noticed that our current int* shrinkers produce the same number twice at the end, e.g.,
for Shrink.int 100 tested on, e.g., (fun i -> i<100) it tries: [50; 75; 88; 94; 97; 99; 99] 🤔
This is confirmed by shrink_algo_logs/int_smaller_209609_qcheck.expected which tests 209608 twice at the end.

I'll investigate. I think that could make for another potential Shrink.int improvement on top of PR #173.

@jmid
Copy link
Collaborator Author

jmid commented Sep 8, 2021

BTW: This PR builds on top of #172 - which renamed the QCheck2 unit test file.

@jmid
Copy link
Collaborator Author

jmid commented Sep 8, 2021

Quick update on the int shrinking:

qcheck/src/core/QCheck.ml

Lines 470 to 477 in 189a025

(* balanced shrinker for integers (non-exhaustive) *)
let int x yield =
let y = ref x in
(* try some divisors *)
while !y < -2 || !y >2 do y := !y / 2; yield (x - !y); done; (* fast path *)
if x>0 then yield (x-1);
if x<0 then yield (x+1);
()

When x is 3, say, we will enter the loop, setting y := 1 and yielding 2.
After resuming we will then exit the loop only to yield 3-1 = 2 again.

PR #173 doesn't seem to have this problem.

This was referenced Sep 9, 2021
This was referenced Apr 2, 2022
@jmid
Copy link
Collaborator Author

jmid commented Apr 2, 2022

Superseeded by #233 and #234

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant