-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve unit test coverage for v2/snap.py #132
Improve unit test coverage for v2/snap.py #132
Conversation
On first call of decorated functions
I'm not really sure what's going on with the failed integration test, but when running them on my machine they seemed a bit non-deterministic. Well, one relating to the terraform location ( EDIT: And rerunning the failing test (ubuntu integration test) on github lead to it passing ... |
tests/unit/test_snap.py
Outdated
|
||
shutdown, socket_path = fake_snapd.start_server() | ||
try: | ||
# test path when _UnixSocketConnection.timeout somehow becomes None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these tests actually useful?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've removed some of the overly specific tests and factored the remaining ones out into separate tests.
More generally, I've added some better documentation for the tests added in this PR.
If you think any of the remaining tests added in this PR are too specific and fragile, I can remove them. I don't think they're out of character with the already existing tests, but I don't think it would be worth adding them if they're just going to increase the maintenance burden.
@@ -1,10 +1,13 @@ | |||
# Copyright 2021 Canonical Ltd. | |||
# See LICENSE file for licensing details. | |||
|
|||
# pyright: reportPrivateUsage=false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For lack of a better place to comment: the main thing I'm wondering here is whether all of this test coverage is worth it. Tests are code, so they have a maintenance cost, and I'm wondering whether some of these pay for themselves. In addition, a lot of these tests rely quite a bit on patching and implementation details, which always makes things a bit more fragile.
Overall, I don't mind including some of these at least. Though I think our future efforts might be better spent in creating a better v3 API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks reasonable to me now, thanks for your effort here.
snap.py
currently has pretty decent code coverage of unit tests at 85%. This PR raises coverage to 99%, using the same approaches to mocking external processes (e.g.snapd
) that the existing tests used.Only the following seemingly unreachable branch is not covered:
(Perhaps a follow up PR can remove this seemingly redundant check.)
I've left in the
pyright
directives intest_snap.py
that I added to silence warnings as I added tests. Whilepyright
is not a dependency of this project, it seems to be the recommended type checker forops
and charms, and if this project considers moving towards more rigorously typed code in future these may be useful. The could be removed before merging this PR though.