-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor tests #5
Comments
The stdlib's unittest is lightweight and probably enough for the purposes of |
|
Hmm, maybe pytest, it has a very concise syntax, and features to support even complex setups? (Though... macros?) While at it, we could convert code examples to doctest format, and |
Some thought is required in setting this up, because macro test modules must run under the Coverage measurement may also help discover more bugs (are the tests actually running the specific lines or code paths that they claim to?). |
Let's try to at least factor the tests properly in 0.14.3. |
Hmm. This isn't simple, because:
So while making regressions maximally pinpointable (so that the test most likely to pinpoint the error correctly alerts first) is a worthy goal, maybe our primary focus should be elsewhere for now - getting a whole testset to run even when some tests fail. The To fix that, in the upcoming 0.14.3, we have added a The reason we provide a new expr macro, instead of rewiring Of course, the only reason to use Where this leads to is that right now, I'm leaning toward rewriting |
Now we have also A language being written in itself leads to the thought that language can be a strange loop, so perhaps we shouldn't expect modularity from a project of this nature. Words are often defined in terms of other words. Similarly, in a programming language extension, there may be a few instances where construct A needs to account for construct B, and construct B needs to account for construct A, even if the user-facing sides of these constructs seem orthogonal. So what each of them means (operationally) changes whenever the other one is present. Or from another angle, the whole library is in a state of mutual equilibrium, its constructs having co-evolved in presence of each other. As long as enough "meaning" (operationally) is in place as each definition runs, it "bottoms out" so the code can run. Maybe this is the kind of thing that naturally comes to mind to anyone familiar with Hofstadter's classic book; or the one by Gell-Mann (I seem to recall he called this kind of situation a Hartree-Fock equilibrium). Maybe it's just a lame excuse for not organizing tests properly. ;) But now we at least have a rudimentary testing framework in place for macro-enabled Python code, so we can first make this thing produce proper test reports, before worrying about modularizing the tests further. |
We should also investigate if we can use |
Test framework pretty much complete for now, now just to rewrite the tests in terms of it... |
After much updating, as of ca964e2, the tests for regular code now use |
As of a409d6f, we're finally done! All tests of Now MacroPy is required for development of The test result on CPython 3.6.9 is:
And on PyPy3 7.3.0:
Two of the errors in both cases are due to the macro expander not liking Remaining:
|
Yes, It's not bad for use with macro-enabled code, but of course with block macros, which line numbers get reported as covered depends on which line numbers the macro implementation injects into the resulting AST and/or how MacroPy fills in the missing ones. The coverage is actually surprisingly high, at a total of 84%, and that's including the false misses due to macro expansion. A detailed investigation will take some time. |
Ok, now we have a CI setup that runs the test suite on Pythons 3.4 through 3.7 and PyPy3, as well as runs Turns out there were a couple of bugs that prevented some features from working on Pythons other than 3.6 and PyPy3; these have been fixed. The tests now pass on 3.4, 3.5, 3.6, 3.7 and on PyPy3. The main remaining issue (beside figuring out a technically reasonable, yet highly readable organization for the tests, for which I don't see a straightforward strategy) is the final 15% of missing code coverage. But that's already getting pretty far from "refactor tests". I think that part is done for now. I'm closing this issue, and opening a new one to track the coverage, to be tackled later, perhaps already during the 0.14.x series. Quite a lot of changes have already accumulated since the recently released 0.14.2, so the plan is to get 0.14.3 out soon-ish - no year-long mega-release this time. Python 3.8 support is planned for the 0.14.x series. Whether it will be included in 0.14.3 or not depends on how much regular code there is to fix once I get the macros working and can flip the switch to start CI'ing with 3.8, too. |
Python 3.8 and 3.9 support has arrived in Macro expander changed to |
Tests need some refactoring:
unpythonic/test
andunpythonic/syntax/test
.pytest
. The currentruntests.py
is a hack.(When tackling this, opening sub-issues for each individual point may be a good strategy.)
The text was updated successfully, but these errors were encountered: