Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate DataTree to the new IR #3818

Merged
merged 174 commits into from
Feb 5, 2024
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
174 commits
Select commit Hold shift + click to select a range
9b356da
initial work on datatree ir migration
tybug Dec 19, 2023
cae9b3b
remove outdated comment
tybug Dec 19, 2023
b083a4f
move choice() method to ConjectureData
tybug Dec 27, 2023
b86ae5b
fix label name collision
tybug Dec 28, 2023
0095ec0
fix almost all shrinker tests
tybug Dec 28, 2023
4ab7c6e
fix optimiser tests
tybug Dec 28, 2023
9a01468
remove test duplicated in test_pareto
tybug Dec 28, 2023
842b732
fix most engine tests
tybug Dec 28, 2023
5c335e1
fix most pareto tests
tybug Dec 28, 2023
833c4cd
fix wrong data.choice usage
tybug Dec 28, 2023
23eafff
use existing count_between_floats
tybug Dec 28, 2023
1958a4c
migrate draw_bits in test_test_data
tybug Jan 6, 2024
ff0ffe6
migrate draw_bits in test_shrinking_dfas
tybug Jan 6, 2024
98947c4
remove ConjectureData#write in favor of draw_bytes(forced=...)
tybug Jan 6, 2024
d219cbf
remove test_draw_write_round_trip. This is better covered by test_for…
tybug Jan 6, 2024
720c750
fix test_float_encoding tests via buffers
tybug Jan 6, 2024
4b0a103
more ConjectureData#write / draw_bytes fixes
tybug Jan 6, 2024
98c93b3
add DRAW_FLOAT_INNER_LABEL to bring float shrinking back to normal
tybug Jan 6, 2024
6ed83ae
improve `minimal` readability with nonlocal
tybug Jan 6, 2024
56d78f1
add and use MAX_CHILDREN_EFFECTIVELY_INFINITE
tybug Jan 6, 2024
8a5a163
add cached_property comment
tybug Jan 6, 2024
dc95a63
use bit representation of floats for keys
tybug Jan 6, 2024
70f9c35
use unweighted sampling if rejection sampling is not making progress
tybug Jan 6, 2024
57fc9ac
more draw_bits -> draw_integer migrations
tybug Jan 6, 2024
8ea0dae
avoid 32 bit integers which draws more data
tybug Jan 6, 2024
4177416
fix test_example_depth_marking
tybug Jan 6, 2024
84a9eb3
spacing
tybug Jan 6, 2024
3f87784
fix test_child_indices for additional draw_boolean examples
tybug Jan 6, 2024
56613bc
add TODO for DRAW_BYTES_LABEL
tybug Jan 6, 2024
01b5cf3
increase test_last_block_length buffer to account for >24 bit integers
tybug Jan 6, 2024
4e314a1
fix test_can_observe_draws
tybug Jan 6, 2024
39d3436
more correct test_can_write_empty_bytes test
tybug Jan 7, 2024
4ab240d
never write exact-value pseudo choices to the IR
tybug Jan 7, 2024
bb2ba67
Merge branch 'various-core-touchups' into 'datatree-ir'
tybug Jan 11, 2024
12efba8
simplify compute_max_children for integers
tybug Jan 11, 2024
bacf301
Merge branch 'master' into datatree-ir
tybug Jan 11, 2024
46a0b1f
rename forced -> was_forced in observer
tybug Jan 11, 2024
142a13d
formatting
tybug Jan 11, 2024
e81d0ad
document `observe` argument
tybug Jan 11, 2024
4490070
add TODO for draw_bytes min/max size
tybug Jan 11, 2024
a154d56
update explore_arbitrary_languages test
tybug Jan 14, 2024
38e758b
note string values in test
tybug Jan 14, 2024
a854245
condition test_basic_indices_options on tuples, add more cases
tybug Jan 14, 2024
3e8a051
formatting
tybug Jan 14, 2024
a42c06c
move Intervals strategy to common
tybug Jan 14, 2024
b5178e5
factor our ir kwarg generation into tests.conjecture.common
tybug Jan 14, 2024
0bb0763
add draw_bytes size assert
tybug Jan 14, 2024
bfacdd2
fix incorrect draw_bits migrations
tybug Jan 14, 2024
dbbe785
handle empty intervals case in compute_max_children
tybug Jan 14, 2024
a1896b9
more reasonable draw_string_kwargs sizes
tybug Jan 14, 2024
97f96aa
linting
tybug Jan 14, 2024
b118cf0
add tests for single-value "choices" not writing to bistream
tybug Jan 14, 2024
1b6c46a
formatting
tybug Jan 14, 2024
99345f8
condition test_basic_indices_options on tuples, add more cases
tybug Jan 14, 2024
a74e468
move Intervals strategy to common
tybug Jan 14, 2024
ec3fe52
factor our ir kwarg generation into tests.conjecture.common
tybug Jan 14, 2024
46220fd
add draw_bytes size assert
tybug Jan 14, 2024
566f88e
fix incorrect draw_bits migrations
tybug Jan 14, 2024
03f22b0
more reasonable draw_string_kwargs sizes
tybug Jan 14, 2024
9970233
formatting
tybug Jan 14, 2024
77cd157
remove ConjectureData#write in favor of draw_bytes(forced=...)
tybug Jan 6, 2024
5e5e4e1
more ConjectureData#write / draw_bytes fixes
tybug Jan 6, 2024
76c48dc
update explore_arbitrary_languages test
tybug Jan 14, 2024
c6f1f8a
formatting
tybug Jan 14, 2024
cc58f53
migrate more draw_bits
tybug Jan 14, 2024
e706709
update test_dependent_block_pairs_is_up_to_shrinking_integers
tybug Jan 15, 2024
1b5c376
fix draw_float_kwargs for nan forced
tybug Jan 15, 2024
6df50bd
fix draw_string_kwargs computation
tybug Jan 15, 2024
6d24152
fix test_can_write_empty_bytes
tybug Jan 15, 2024
e41514b
rewrite float drawing for nan/clamper interaction
tybug Jan 15, 2024
017f20b
Merge branch 'master' of https://github.com/tybug/hypothesis into var…
tybug Jan 15, 2024
db69f87
Merge branch 'master' into various-touchups-2
tybug Jan 15, 2024
c5fba69
remove forgotten debug statement
tybug Jan 15, 2024
baf67f7
implement forced for out of bounds p in draw_boolean
tybug Jan 15, 2024
057271b
better draw_integers_kwargs comments and weights
tybug Jan 15, 2024
7e31ebd
rewrite nan/clamper handling, again
tybug Jan 15, 2024
156cfd0
add release notes
tybug Jan 15, 2024
8583a2d
return False with p=0, even if forced True
tybug Jan 15, 2024
7e2f80a
rework forced sampling to account for 0 probabilities
tybug Jan 15, 2024
f15ef5f
oops
tybug Jan 15, 2024
23f5e91
Merge branch 'master' into datatree-ir
tybug Jan 15, 2024
4c2e0c3
update test_integer_range_lower_equals_upper
tybug Jan 15, 2024
3cc51d4
update test_regression_1
tybug Jan 15, 2024
d72adfd
rewrite for clarity in Sampler.sample
tybug Jan 17, 2024
993b51b
more clear assume statement
tybug Jan 17, 2024
c760c55
improve draw_integer_kwargs distribution and speed
tybug Jan 17, 2024
d227ab8
add invariant to comment
tybug Jan 17, 2024
f0cd78b
improve draw_string_kwargs distribution and performance
tybug Jan 17, 2024
372a3e2
improve weight drawing in draw_integer_kwargs
tybug Jan 17, 2024
234f2ea
apparently 64 bit boolean draws are totally fine!
tybug Jan 17, 2024
a0e9cfb
fix accidentally weakened forced testing
tybug Jan 17, 2024
8a503f1
typo
tybug Jan 17, 2024
46cf059
rename intervals strategies, improve speed
tybug Jan 17, 2024
b4b88c5
change test format to avoid st.data()
tybug Jan 17, 2024
c3edf81
add use_{min, max}_value to draw_float_kwargs
tybug Jan 17, 2024
9b67c03
dont force sign bit if nans are allowed
tybug Jan 17, 2024
214b6c4
Merge branch 'test-forced-improvements' into datatree-ir
tybug Jan 17, 2024
575fd0e
Merge branch 'master' into datatree-ir
tybug Jan 17, 2024
280434f
revert fixed workaround for non-str notes
tybug Jan 17, 2024
9d1d316
delete release
tybug Jan 17, 2024
e2e820f
change outdated reference to draw_bits
tybug Jan 19, 2024
43506a9
fix compute_max_children case for integers, add simple test case
tybug Jan 19, 2024
007cc9d
add test for differentiating between 0.0 and -0.0 float draws
tybug Jan 19, 2024
3eb27b2
fix draw_string with single interval and equal bounds, and add test
tybug Jan 19, 2024
21db4bb
add IRType type alias
tybug Jan 19, 2024
6e42732
simplify helpers in generate_novel_prefix
tybug Jan 19, 2024
1b47296
import BUFFER_SIZE instead of redefining
tybug Jan 19, 2024
f575070
rewrite datatree documentation
tybug Jan 20, 2024
bfb8e69
avoid TypeAlias errors on python < 3.10
tybug Jan 20, 2024
b383a5c
extract draw_string max size to shared var
tybug Jan 20, 2024
aa999ba
move max_children > 1 assert
tybug Jan 20, 2024
b91783d
mark was_forced as keyword only
tybug Jan 20, 2024
f21b74d
use unbiased kwargs in data tree when stuck
tybug Jan 21, 2024
ff46d20
Merge branch 'master' into datatree-ir
tybug Jan 21, 2024
f9441a1
reword todo comment
tybug Jan 21, 2024
b41d425
revert accidental changes
tybug Jan 21, 2024
e66c88d
revert float changes
tybug Jan 21, 2024
6e2f394
let's just revert a bunch of stuff, shall we?
tybug Jan 21, 2024
dd6d377
formatting
tybug Jan 21, 2024
3ce8fdb
Merge branch 'more-test-changes' into datatree-ir
tybug Jan 22, 2024
245475f
nocover error case in datatree ir handling
tybug Jan 22, 2024
433ac7a
add observer param to fresh_data
tybug Jan 23, 2024
2defb40
add tests for observing and non-observing draws
tybug Jan 23, 2024
90e9600
rename test
tybug Jan 23, 2024
e7475a3
rewrite compute_max_children string case to be more correct
tybug Jan 23, 2024
42f2d17
add morecover tests for IntList
tybug Jan 23, 2024
a2dcfd9
avoid forcefully splitting on forced nodes
tybug Jan 23, 2024
4909988
Merge branch 'master' into datatree-ir
tybug Jan 23, 2024
e49c829
update test_targeting_can_drive_length_very_high with draw_boolean
tybug Jan 23, 2024
6d60e79
give up on novel prefixes when they are too hard to discover
tybug Jan 24, 2024
e26e5d1
kneecap inquisitor test (unfortunately)
tybug Jan 24, 2024
11c5681
give up completely on novel generation instead of returning non-novel
tybug Jan 24, 2024
4ab69f2
increase attempts budget
tybug Jan 24, 2024
9829729
kneecap test_discards_kill_branches for now
tybug Jan 24, 2024
df121e7
weaken test_one_dead_branch
tybug Jan 24, 2024
0f24890
fix compute_max_children for certain boolean draws
tybug Jan 24, 2024
38b0747
weaken test_lot_of_dead_nodes
tybug Jan 24, 2024
95d4f71
type observe params
tybug Jan 24, 2024
ab47af7
fix float keys being interpreted incorrectly in simulate_test_function
tybug Jan 25, 2024
c4de1c9
formatting
tybug Jan 25, 2024
931508d
add TooHard cover test
tybug Jan 25, 2024
6d5b1a9
nocover TooHard in generate_new_examples
tybug Jan 25, 2024
6469428
add types to ir kwargs/values
tybug Jan 25, 2024
bff153f
formatting
tybug Jan 25, 2024
cf786db
type datatree more
tybug Jan 25, 2024
9245460
some linting adjustments
tybug Jan 25, 2024
3cf8d3e
fix test_can_observe_draws
tybug Jan 25, 2024
4386a9a
fix lint rule
tybug Jan 25, 2024
cfb4439
add test for jsonable large ints
tybug Jan 25, 2024
459e96f
deflake test_finds_multiple_failures_in_generation
tybug Jan 25, 2024
7e422a0
deflake test_shrinks_both_failures
tybug Jan 25, 2024
9f1f0bf
add release notes
tybug Jan 25, 2024
d5a0edc
reword
tybug Jan 25, 2024
5832b5f
rewrite comment
tybug Jan 25, 2024
55e6c76
lint
tybug Jan 26, 2024
c15a19e
more consistent test_novel_prefix_gives_up_when_too_hard
tybug Jan 26, 2024
4deffdd
switch order of kwargs and ir_type in compute_max_children
tybug Jan 29, 2024
9040090
compute and draw from a set of available children instead of giving u…
tybug Jan 30, 2024
77680cb
fix ordering in floats_between
tybug Jan 31, 2024
c9eaf6c
add more ir tests
tybug Jan 31, 2024
bce60d5
no-branch on _draw_from_cache
tybug Jan 31, 2024
797d67c
account for 0 weight integers in children computation
tybug Jan 31, 2024
c8c326a
split asserts
tybug Jan 31, 2024
85316f4
format
tybug Jan 31, 2024
575b53b
deflake test_subtraction_of_intervals
tybug Feb 1, 2024
5058f50
fix float key representation in _draw_from_cache
tybug Feb 1, 2024
d9b0e0f
add cover test for hard floats
tybug Feb 2, 2024
3ff7e49
deflake test_can_generate_hard_floats
tybug Feb 3, 2024
a1eb9a9
more consistent conjecture float coverage
tybug Feb 4, 2024
db565f3
simpler Literal
tybug Feb 4, 2024
081e32e
wording
tybug Feb 4, 2024
285bbf9
more idiomatic unhandled ir_type error
tybug Feb 4, 2024
2c16f46
Merge branch 'master' into datatree-ir
Zac-HD Feb 4, 2024
4d4a32f
formatting
Zac-HD Feb 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
249 changes: 142 additions & 107 deletions hypothesis-python/src/hypothesis/internal/conjecture/data.py

Large diffs are not rendered by default.

262 changes: 219 additions & 43 deletions hypothesis-python/src/hypothesis/internal/conjecture/datatree.py

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
random_selection_order,
)
from hypothesis.internal.conjecture.data import (
DRAW_FLOAT_LABEL,
DRAW_FLOAT_INNER_LABEL,
ConjectureData,
ConjectureResult,
Status,
Expand Down Expand Up @@ -1201,7 +1201,7 @@ def minimize_floats(self, chooser):
ex = chooser.choose(
self.examples,
lambda ex: (
ex.label == DRAW_FLOAT_LABEL
ex.label == DRAW_FLOAT_INNER_LABEL
and len(ex.children) == 2
and ex.children[1].length == 8
),
Expand Down
37 changes: 12 additions & 25 deletions hypothesis-python/src/hypothesis/internal/conjecture/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,14 +81,6 @@ def check_sample(
return tuple(values)


def choice(
data: "ConjectureData", values: Sequence[T], *, forced: Optional[T] = None
) -> T:
forced_i = None if forced is None else values.index(forced)
i = data.draw_integer(0, len(values) - 1, forced=forced_i)
return values[i]


class Sampler:
"""Sampler based on Vose's algorithm for the alias method. See
http://www.keithschwarz.com/darts-dice-coins/ for a good explanation.
Expand All @@ -109,13 +101,12 @@ class Sampler:

table: List[Tuple[int, int, float]] # (base_idx, alt_idx, alt_chance)

def __init__(self, weights: Sequence[float]):
n = len(weights)
def __init__(self, weights: Sequence[float], *, observe: bool = True):
self.observe = observe

n = len(weights)
table: "list[list[int | float | None]]" = [[i, None, None] for i in range(n)]

total = sum(weights)

num_type = type(total)

zero = num_type(0) # type: ignore
Expand Down Expand Up @@ -182,11 +173,13 @@ def sample(self, data: "ConjectureData", forced: Optional[int] = None) -> int:
if forced is None
else next((b, a, a_c) for (b, a, a_c) in self.table if forced in (b, a))
)
base, alternate, alternate_chance = choice(
data, self.table, forced=forced_choice
base, alternate, alternate_chance = data.choice(
self.table, forced=forced_choice, observe=self.observe
)
use_alternate = data.draw_boolean(
alternate_chance, forced=None if forced is None else forced == alternate
alternate_chance,
forced=None if forced is None else forced == alternate,
observe=self.observe,
)
data.stop_example()
if use_alternate:
Expand All @@ -198,7 +191,7 @@ def sample(self, data: "ConjectureData", forced: Optional[int] = None) -> int:


INT_SIZES = (8, 16, 32, 64, 128)
INT_SIZES_SAMPLER = Sampler((4.0, 8.0, 1.0, 1.0, 0.5))
INT_SIZES_SAMPLER = Sampler((4.0, 8.0, 1.0, 1.0, 0.5), observe=False)


class many:
Expand All @@ -221,6 +214,7 @@ def __init__(
average_size: Union[int, float],
*,
forced: Optional[int] = None,
observe=True,
) -> None:
assert 0 <= min_size <= average_size <= max_size
assert forced is None or min_size <= forced <= max_size
Expand All @@ -231,20 +225,14 @@ def __init__(
self.p_continue = _calc_p_continue(average_size - min_size, max_size - min_size)
self.count = 0
self.rejections = 0
self.drawn = False
self.force_stop = False
self.rejected = False
self.observe = observe

def more(self) -> bool:
"""Should I draw another element to add to the collection?"""
if self.drawn:
self.data.stop_example(discard=self.rejected)

self.drawn = True
self.rejected = False

self.data.start_example(ONE_FROM_MANY_LABEL)

if self.min_size == self.max_size:
# if we have to hit an exact size, draw unconditionally until that
# point, and no further.
Expand All @@ -263,14 +251,13 @@ def more(self) -> bool:
elif self.forced_size is not None:
forced_result = self.count < self.forced_size
should_continue = self.data.draw_boolean(
self.p_continue, forced=forced_result
self.p_continue, forced=forced_result, observe=self.observe
)

if should_continue:
self.count += 1
return True
else:
self.data.stop_example()
return False

def reject(self, why: Optional[str] = None) -> None:
Expand Down
3 changes: 3 additions & 0 deletions hypothesis-python/src/hypothesis/internal/intervalsets.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@ def __sub__(self, other):
def __and__(self, other):
return self.intersection(other)

def __eq__(self, other):
return isinstance(other, IntervalSet) and (other.intervals == self.intervals)

def union(self, other):
tybug marked this conversation as resolved.
Show resolved Hide resolved
"""Merge two sequences of intervals into a single tuple of intervals.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -586,7 +586,7 @@ def do_filtered_draw(self, data):
# The speculative index didn't work out, but at this point we've built
# and can choose from the complete list of allowed indices and elements.
if allowed:
i, element = cu.choice(data, allowed)
i, element = data.choice(allowed)
data.draw_integer(0, len(self.elements) - 1, forced=i)
return element
# If there are no allowed indices, the filter couldn't be satisfied.
Expand Down
19 changes: 11 additions & 8 deletions hypothesis-python/tests/common/debug.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,20 @@ class Timeout(BaseException):


def minimal(definition, condition=lambda x: True, settings=None, timeout_after=10):
definition.validate()
runtime = None
result = None

def wrapped_condition(x):
nonlocal runtime
if timeout_after is not None:
if runtime:
runtime[0] += TIME_INCREMENT
if runtime[0] >= timeout_after:
runtime += TIME_INCREMENT
if runtime >= timeout_after:
raise Timeout
result = condition(x)
if result and not runtime:
runtime.append(0.0)
runtime = 0.0
return result

if settings is None:
Expand All @@ -51,16 +56,14 @@ def wrapped_condition(x):
)
def inner(x):
if wrapped_condition(x):
result[:] = [x]
nonlocal result
result = x
raise Found

definition.validate()
runtime = []
result = []
tybug marked this conversation as resolved.
Show resolved Hide resolved
try:
inner()
except Found:
return result[0]
return result
raise Unsatisfiable(
"Could not find any examples from %r that satisfied %s"
% (definition, get_pretty_function_description(condition))
Expand Down
Loading
Loading