Fix Unit Tests #88
Merged
Fix Unit Tests #88
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix having removed the test_ prefix from all of them, which prevented pytest from detecting them. Now `pytest -vv` at project root actually runs all the unit tests. Also fix a line in pyproject.toml that was supposed to ignore any DeprecationWarning generated by the tests.
Fix #56 There was a matrix entry `experimental` with the values `[false, false, true]`, which meant every unit test was run 3 times changing only that one flag. It looks like the intention was to just run Windows tests with `continue-on-error`, but I just set that flag for all the runs for now. The tests are still pretty broken anyway.
It used to just zip the args and kwargs together, which led to mapping over nothing when args were provided and kwargs defaulted to [].
Closed
DailyDreaming
approved these changes
Jun 6, 2024
There was a problem hiding this comment.
This looks good to me, although a lot of the changes are linting or style changes, which we might want to enforce in a future PR.
I've added decorators to skip failing tests, conditionally failing tests on Windows, and retry decorators for flaky tests. I'll make these into their own issues and they can be resolved and fixed incrementally in future PRs.
Right now this is great to have the testing in, plus some of the clean up. Thanks for doing this!
atspaeth
added a commit
that referenced
this pull request
Jun 7, 2024
* Move test files to tests/ * Delete configure test that no longer makes sense. * Fix smart_open test by updating assertEqual() method name * Fix memoize, common_utils, mmap by updating import paths * Fix path to test maxwell metadata * Remove generated _version file from git * Make pytest actually run the tests Fix having removed the test_ prefix from all of them, which prevented pytest from detecting them. Now `pytest -vv` at project root actually runs all the unit tests. Also fix a line in pyproject.toml that was supposed to ignore any DeprecationWarning generated by the tests. * Ignore warnings for tests loading non-row-major datasets * Remove "deprecated" file that couldn't even be imported * Make joblib a dev dependency so memoize_s3 tests in CI * Remove redundant CI runs Fix #56 There was a matrix entry `experimental` with the values `[false, false, true]`, which meant every unit test was run 3 times changing only that one flag. It looks like the intention was to just run Windows tests with `continue-on-error`, but I just set that flag for all the runs for now. The tests are still pretty broken anyway. * Remove dead code, deprecate unusable functions * Clean more dead code in messages * Normalize unit test formatting * Add credentials as a GitHub secret * Fix common_utils.map2() when no kwargs provided It used to just zip the args and kwargs together, which led to mapping over nothing when args were provided and kwargs defaulted to []. * Does bash fix CI on windows? * Make memoize_s3 transform \ into / for Windows * Skip broken tests. * Skip tests that break only on CI (parallelism?). * Skip Windows-failing tests conditionally. * Mark flaky test. * Mark another flaky test. * Hammer the nail in place. * Actually skip the memoize test. * Mark more flakes. --------- Co-authored-by: DailyDreaming <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I noticed none of the unit tests were executing in CI because pytest looks at /tests, not /src. This PR is about migrating them back in so they run again, except analysis, which is going to be addressed by #87.
State of Tests
The tests all pass locally (Linux, Python 3.10, 3.11, and 3.12), but in CI on Ubuntu/Mac,
Error: Codecov token not found., with a log line mentioning not finding a config.)There are also some Windows-specific failures in CI, which I haven't tried reproducing locally:
test_read_data_maxwell_v2_format,TestMemoizeS3, andSmartOpenTestCase::test_local_path_endpoint)PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\test_cache_eyh8it4l\\cache.db')Changes
CI and Tests
joblib(formemoize_s3) to dev extraMain Library
braingeneers/data/datasets.py, which was "deprecated" but also unimportable.