Migrate CI to GitHub Actions as Travis CI is ending their free OSS builds #107
Comments
|
I think all of the repos I manage are Travis, due to inertia :( |
|
I'm happy to drive an effort to switch to Actions. I can make a start on this now. Proposed steps:
While we are at it should we consider standardising on GitHub Actions? We also have some CircleCI around too right? |
Migration stepsHere are some general steps to migrate from Travis CI to GitHub Actions. I'll update this as I work through some migrations to add more detail. Convert configCreate new CI config Travis generally has some environment config such as OS, langauge, services, etc along with a build matrix. Then it has a few sections for running bash commands In GitHub Actions we specify a trigger (e.g on pull request) and a list of jobs to be run. Each job has options such as OS selection and the option to specify a matrix. Then it has a series of steps. Each step can be a bash command or a prebaked "Action". To keep the migration simple it makes sense to use a couple of official Actions to configure the environment and then copy the existing bash commands from ExampleHere's a simplified example of converting from one to the other. # .travis.yml
language: python
matrix:
include:
- python: 3.7
- python: 3.8
install:
- python -m pip install --upgrade pip
- pip install flake8 pytest
- pip install -r requirements.txt
script:
- flake8
- pytest# .github/workflows/ci.yml
name: CI
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.7, 3.8]
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
pip install -r requirements.txt
- name: Lint with flake8
run: |
flake8
- name: Test with pytest
run: |
pytestTest workflowRaise a PR and test the new workflow. Ensure Travis is disabledYour PR to migrate should delete the Update badgesWe also need to update the README badges to point to the new workflow. Badge URL: For example Markdown [](https://github.com/dask/dask-jobqueue/actions)RST .. |Build Status| image:: https://github.com/dask/dask-jobqueue/workflows/CI/badge.svg
:target: https://github.com/dask/dask-jobqueue/actionsPreview
|
A note of light caution -- currently if you have a matrix of say Python versions and OS in the same workflow file, if one of those matrix entries failed you can't restart it individually, you have to trigger a full re-run of everything defined in that file. On |
|
|
From the intake org: fsspec (and a bucket full of intake specific things) |
|
The table above is now complete. This is a list of all active repos and their current CI. Any currently using Travis I have opened an issue to migrate to GitHub Actions. Any using something else like Circle of Azure DevOps I've opened an issue asking whether we should migrate. Feedback on individual issues from folks who work on those projects is appreciated. I'll also keep the table up to date as we open PRs and migrate over. |
|
Is there a script to convert from one YAML build spec to another? There may already be tool written in Python for converting between various CI configs? Looks like drone-yaml (which is depended upon by drone-cli) can convert from BitBucket & GitLab, but not yet CircleCI, TravisCI, or GitHub If you can put most of the CI config in tox.ini with tox-travis or tox-gh-actions, you can more easily run equivalent local builds (and have less build config to convert) Note that self-hosted GitHub runners (~GitLab CI Runners (Go)) are an option for faster local or cloud builds: |
|
Would "Move to GitHub Actions instead of just asking Travis CI for more OSS build quota" be a more accurate issue title? |
Possibly, but part of the goal here is to make use of GitHub Actions features, simplify CI and add more consistency. So doing a direct conversion may not be the best approach here.
Much of the benefit of using tox is for creating environments for you, then automating the commands. We already do much of this with conda and CI scripts. I'm not sure if we've ever discussed using a tool like tox, but if we did there would need to be a clear benefit over what we have today.
In order to use self hosted runners we would need to self host them somewhere. A big part of the motivation here is to move to a reliable and free provider.
Adding a reference to the GitHub Actions migration seems sensible. Asking Travis for more quota does not feel like a sustainable solution. |
jacobtomlinson
changed the title
|
One thing you might consider is using conda-forge's tooling to provision CIs. This way you are somewhat abstracted from the underlying CI system and as new CI tooling comes out you can take advantage of it without having to do the migration yourself. A simple |
|
https://conda-forge.org/docs/maintainer/adding_pkgs.html ::
https://conda-forge.org/docs/maintainer/updating_pkgs.html#example-workflow-for-updating-a-package https://conda-forge.org/docs/maintainer/updating_pkgs.html#pushing-to-regro-cf-autotick-bot-branch :
bot commands recognized in GH PR comments: |
|
TIL there's also a |
Travis CI is used by several repos in the org. Today they announced they are shutting down their free OSS builds (https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing). Existing users get 10,000 free credits (1000 mins of linux build time), which is not nearly enough to support our CI needs. Per the blog post users can ask for more free credits, which they'll evaluate on a case-by-case basis.
Should we start migrating projects to a different CI system?
The text was updated successfully, but these errors were encountered: