Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add: new page overviewing CI #165

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions ci-data/ci.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# What is continuous integration?

When you’re ready to publish your code online, you can setup Continuous Integration (CI). CI is a platform that allows you to specify and run jobs or workflows that you define.
These workflows include:

- Running your test suite
- Running code checkers / linters / spellcheck
- Building your documentation
- Deploying your documentation to GitHub pages

CI allows you to automate running workflows across a suite of environments including:

- environments containing different Python versions and
- different operating systems (Mac, Linux, Windows).

### What is Continuous Deployment (CD)?

Continuous deployment (CD) extends the CI process by automating the deployment of code changes to production or staging environments. In the case of your open source tool, CD can be used to:

- Automate publishing to PyPI
- Automate publishing your documentation to GitHub Pages or Read the Docs.

It is also used once your conda-forge recipe is set up to keep your package up to date on conda-forge.

### Why Use CI

CI can be configured to run a workflow on every commit pushed to GitHub and every pull request opened. This ensures that any changes made to your package are tested across environments before they are merged into the main branch of your code.

These checks are particularly useful if someone new is contributing to your code. Every change a contributor makes will be tested when it’s pushed to your code repository.

Together, CI and CD streamline the process of building, testing, and deploying code. They aim to improve the efficiency, quality, and reliability of software development and publication.

```{note}
All pyOpenSci packages must use some form of continuous integration. Even if you are not planning to go through peer review, we strongly recommend that you use continuous integration too!
```

In the case of GitHub actions (which we will focus on here), CI workflows are running on online servers that support GitHub.

## CI / CD Platforms

There are numerous platforms available for CI/CD. Here, we will focus on GitHub Actions (GHA) which is built into GitHub. GitHub is the most commonly used platform to store scientific open source software.

:::{note}
If you are using [GitLab](https://about.gitlab.com/) CI/CD many of the principles described here will apply, however the workflow files may look different.
:::

### If you aren't sure, use GitHub Actions

While you are welcome to use the continuous integration platform of your choice,
we recommend GitHub Actions because it is free-to-use and integrated tightly
into the GitHub user interface. There is also an entire store of GitHub action
templates that you can easily use and adapt to your own needs.

:::{admonition} Other platforms that you may run into
:class: info

- [Appveyor:](https://www.appveyor.com/): Supports running tests on Windows operating systems and predated the release of GitHub Actions. Today, AppVeyor supports operating systems beyond Windows.
- [Travis CI:](https://www.travis-ci.com/) had been a common CI platform choice in our ecosystem. Usage dropped after Travis CI ended free support for open-source projects.
- [CircleCI:](https://circleci.com/) CircleCI can be useful for automated builds of websites and documentation since it offers a preview of the PR changes.
:::

## Embrace automation

By embracing CI/CD, you can ensure that your code runs as you expect it to across the diverse landscapes of user environments. Further you can
automate certain checks (and in some cases code fixes) including linting and code style. You can even automate spell checking your documentation
and docstrings!
11 changes: 11 additions & 0 deletions ci-data/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Continuous Integration and Data for your package


:::{toctree}
:hidden:
:caption: CI & Data


What is CI? <ci.md>
Data <data.md>
:::
3 changes: 2 additions & 1 deletion conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@
"image_light": "logo-light-mode.png",
"alt_text": "pyOpenSci Python Package Guide. The pyOpenSci logo is a purple flower with pyOpenSci under it. The o in open sci is the center of the flower",
},
"header_links_before_dropdown": 4,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes space for a CI&data section in our navigation.

"header_links_before_dropdown": 5,
"use_edit_page_button": True,
"show_nav_level": 2,
"navigation_depth": 3,
Expand All @@ -111,6 +111,7 @@
"footer_end": [],
}


html_context = {
"github_user": "pyopensci",
"github_repo": "python-package-guide",
Expand Down
Binary file added images/code-cov-stravalib.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
25 changes: 21 additions & 4 deletions index.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,18 +294,35 @@ Packaging <package-structure-code/intro>

```

```{toctree}
:::{toctree}
:hidden:
:caption: Documentation

Documentation <documentation/index>

```
:::

```{toctree}
:::{toctree}
:hidden:
:caption: Build & Publish

Packaging <package-structure-code/intro>

:::

:::{toctree}
:hidden:
:caption: Testing

Tests <tests/index>

```
:::


:::{toctree}
:hidden:
:caption: CI & Data

CI & Data <ci-data/index>

:::
4 changes: 2 additions & 2 deletions package-structure-code/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ if you are looking for pyOpenSci's Python package review requirements!

:::{toctree}
:hidden:
:caption: Package structure & code style
:caption: Create & Build Your Package

Intro <self>

Expand All @@ -176,7 +176,7 @@ Complex Builds <complex-python-package-builds>

:::{toctree}
:hidden:
:caption: Publishing a package
:caption: Publish your package

Publish with Conda / PyPI <publish-python-package-pypi-conda>
Package versions <python-package-versions>
Expand Down
13 changes: 13 additions & 0 deletions tests/code-cov.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Code coverage for your Python package test suite

Code coverage is the amount of your package's codebase that is run as a part of running your project's tests. A good rule of thumb is to ensure that every line of your code is run at least once during testing. However, note that good code coverage does not guarantee that your package is well-tested. For example, you may run all of your lines of code, but not account for many edge-cases that users may have. Ultimately, you should think carefully about the way your package will be used, and decide whether your tests adequately cover all of that usage.

Some common services for analyzing code coverage are [codecov.io](https://codecov.io/) and [coveralls.io](https://coveralls.io/). These projects are free for open source tools, and will provide dashboards that tell you how much of your codebase is covered during your tests. We recommend setting up an account (on either CodeCov or Coveralls), and using it to keep track of your code coverage.

```{figure} ../images/code-cov-stravalib.png
:height: 450px
:alt: Screenshot of the code cov service - showing test coverage for the stravalib package. in this image you can see a list of package modules and the associated number of lines and % lines covered by tests. at the top of the image you can see what branch is being evaluated and the path to the repository being shown.

The CodeCov platform is a useful tool if you wish to visually track code coverage. Using it you can not only get the same summary information that you can get with **pytest-cov** extension. You can also get a visual representation of what lines are covered by your tests and what lines are not covered. Code coverage is mostly useful for evaluating unit tests and/or how much of your package code is "covered". It however will not evaluate things like integration tests and end-to-end workflows.

```
1 change: 1 addition & 0 deletions tests/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,5 @@ Write tests <write-tests>
Test types <test-types>
Run tests locally <run-tests>
Run tests online (using CI) <tests-ci>
Code coverage <code-cov>
```
Loading