Group split kfold #484
Merged
Group split kfold #484
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…into group_split_kfold
jduerholt
requested changes
Dec 22, 2024
| assert model._check_valid_nfolds(20, 10) == 10 | ||
|
|
||
| # Test invalid folds | ||
| with pytest.raises(ValueError): |
There was a problem hiding this comment.
Can you also test that the correct errors are thrown here? you can do this via the match keyword.
There was a problem hiding this comment.
Also for the cases below.
| if group_split_column is not None: | ||
| # check if the group split column is present in the experiments | ||
| if group_split_column not in experiments.columns: | ||
| raise ValueError( |
There was a problem hiding this comment.
Can you also test these errors, so that they are throw correctly?
|
@jkeupp: Do you keep this in mind, so that we can integrate it soon? Best, Johannes |
jduerholt
approved these changes
Jan 14, 2025
dlinzner-bcs
pushed a commit
that referenced
this pull request
Jan 20, 2025
* add group kfold option in cross_validate of any traainable surrogate * changed to GroupShuffleSplit, added test case * improve docstring & add some inline comments in test * refactor cross_validate & add tests * imrpve tests, remove unnecessary case while checking group split col * add push * formatting --------- Co-authored-by: Jim Boelrijk Valcon <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces the GroupShuffleSplit functionality to the cross_validate method in bofire/surrogates/trainable.py. It adds a new parameter group_split_column to ensure that the splits are made such that the same group is not present in both training and testing sets.
Additionally, corresponding tests have been added in tests/bofire/surrogates/test_cross_validate.py to validate the new functionality.
For now this only works for stand-alone cross_validation calls of the surrogate. Next step, which is to use it during hyperparameter optimization will be part of a future PR.