You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
So, this has to do with generator finding input showing bug A, and shrinker collapsing it to an input showing a different bug B.
Example:
floatsAreInOrder =Test.fuzz2
(Random.float 010)(Random.float 010)"Silly test"(\x y -> x |>Expect.lessThan y)
The generator would first find (1,0) and the shrinker would collapse it down to (0,0). These are possibly two different situations we could care about (GT, then EQ) / two different bugs!
It's hard to find out when this is happening (I feel like the best solution for this would be to compare code coverage "ticks" for each input), but we could approximate this using classification labels (if the user provides them). The test runner would remember and report the best shrink for each label set it has encountered. Label sets are generated from the input value using the user-generated a -> Bool functions, as in:
It's a bit of mouthful, so first let me give the original context and then let me explain a bit.
Origin:
https://github.com/ianmackenzie/elm-random-test/blob/b565db4926962e374f098d9d8b317d531f2b9e72/README.md
->
So, this has to do with generator finding input showing bug A, and shrinker collapsing it to an input showing a different bug B.
Example:
The generator would first find
(1,0)
and the shrinker would collapse it down to(0,0)
. These are possibly two different situations we could care about (GT, then EQ) / two different bugs!It's hard to find out when this is happening (I feel like the best solution for this would be to compare code coverage "ticks" for each input), but we could approximate this using classification labels (if the user provides them). The test runner would remember and report the best shrink for each label set it has encountered. Label sets are generated from the input value using the user-generated
a -> Bool
functions, as in:(There are 2^4 possible subsets of these four labels, so up to 2^4 different bugs to report if our generator+shrinker is lucky enough!)
The text was updated successfully, but these errors were encountered: