Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test-coverage checks to CI #199

Open
caiw opened this issue Feb 23, 2024 · 5 comments · May be fixed by #223
Open

Add test-coverage checks to CI #199

caiw opened this issue Feb 23, 2024 · 5 comments · May be fixed by #223
Labels
⏸️ blocked/on-hold Blocked or otherwise waiting for something 👷 design standards 💪 enhancement New feature or request

Comments

@caiw
Copy link
Member

caiw commented Feb 23, 2024

A good software package has good test coverage! We have basic coverage (#5) but still some large holes (#124, #169). When these are covered, it would be good to maintain that coverage in future, using CI to make sure that new functionality must come with new tests.

@caiw caiw added 💪 enhancement New feature or request 👷 design standards ⏸️ blocked/on-hold Blocked or otherwise waiting for something labels Feb 23, 2024
@neukym
Copy link
Member

neukym commented Feb 23, 2024

Can I just check what you mean by this though? We already run unit tests as part of CI, and require 100% pass rate:

Screenshot 2024-02-23 at 14 12 49

... where the action 'perform unit tests' will fail if 100% of the tests do not pass:

Screenshot 2024-02-23 at 14 13 12

As we add in more tests these will be automatically added to the list.

But do you mean something else with this issue?

@caiw
Copy link
Member Author

caiw commented Feb 23, 2024

All tests definitely need to pass, but this is about how much code is tested.

Test-coverage checkers tell us how much of the total codebase (by lines) is covered by at least one test. We can set a benchmark, like 50%, and not allow merging of branches that increase the amount of untested code to 50%.

@neukym
Copy link
Member

neukym commented Feb 23, 2024

Ah, gotcha 👍

@caiw
Copy link
Member Author

caiw commented Feb 23, 2024

Here's an example of a coverage action: https://github.com/marketplace/actions/python-coverage

@caiw caiw linked a pull request Apr 5, 2024 that will close this issue
@caiw caiw linked a pull request Apr 5, 2024 that will close this issue
@caiw
Copy link
Member Author

caiw commented May 29, 2024

Once the repo is open (#265), there are a bunch of free tools we could use, e.g. coveralls.io.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⏸️ blocked/on-hold Blocked or otherwise waiting for something 👷 design standards 💪 enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants