Skip to content

Release Procedures

Jump to bottom
James Davies edited this page Jul 19, 2021 · 19 revisions

We assume you have a local clone of your fork of https://github.com/spacetelescope/stcal and that origin points to your fork and upstream points the central repo. I.e. the following (or ssh equivalents):

$ git remote -v
origin	https://github.com/jdavies-st/stcal.git (fetch)
origin	https://github.com/jdavies-st/stcal.git (push)
upstream	https://github.com/spacetelescope/stcal.git (fetch)
upstream	https://github.com/spacetelescope/stcal.git (push)

Prepare master branch for release

These steps should be undertaken on the main branch:

  1. Update CHANGES.rst to have the correct release version and date. Change "unreleased" next to the intended release to the current date in YYYY-MM-DD format.

  2. Create a PR against upstream/main and merge it.

Prepare the release branch

If you're making a major or minor version release, then the release branch will not yet exist. If you're releasing a patch version, then a release branch will already exist and patch may include everything on master or need cherry-picking. Following one of the next 3 options accordingly. The following steps assume that the spacetelescope/stcal remote is named upstream, just as above.

Option 1: Major or minor version release

  1. Update pointer to main:

    $ git fetch upstream
$ git checkout main
$ git pull upstream main

  2. Verify that the last commit in the log is as expected:

    $ git log

  3. Create a new release branch. The name of the release branch should share the major and minor version of your release version, but the patch version should be x. For example, when releasing 0.16.0, name the branch 0.16.x.

    $ git checkout -b a.b.x upstream/main

  4. Push the branch to the upstream remote.

    $ git push -u upstream HEAD

Github actions CI should notice the updated branch and run the tests which you can check here. Wait until the build passes before proceeding.

Option 2: Patch release (including all of main)

If your patch release will include everything on main, the procedure is very similar to a major/minor release, but the release branch will already exist. So if you need to deliver a 0.16.1 patch to 0.16.0, then you will do your work in branch 0.16.x.

  1. Checkout and freshen release branch (this assumes that your local branch is already tracking upstream/a.b.x):

    $ git checkout a.b.x
$ git pull

  2. Pull all changes on main into the release branch:

    $ git fetch upstream
$ git pull upstream main

  3. Push updates to the upstream remote:

    $ git push upstream HEAD

Github actions CI should notice the updated branch and run the tests which you can check here. Wait until the build passes before proceeding.

Option 3: Patch release (requiring cherry-picking from main)

In the case of a patch release, the release branch will already exist. So if you need to deliver a 0.16.1 patch to 0.16.0, then you will do your work in branch 0.16.x.

  1. Checkout and freshen release branch (this assumes that your local branch is already tracking upstream/a.b.x):

    $ git checkout a.b.x
$ git pull

  2. Cherry-pick relevant commits from main that should be included in the patch release (including the new changelog commit):

    $ git cherry-pick ...

  3. Push updates to the upstream remote:

    $ git push upstream HEAD

Github actions CI should notice the updated branch and run the tests which you can check here. Wait until the build passes before proceeding.

Cherry-picking tips

A bug needs to be fixed in 0.16.0. It is also present on main.

Fix the bug on main through a PR. Merge it and get the hash of the commit. As an example, let's say the commit hash is f41120e.

Cherry pick all commits that need to be included in the patch release.

$ git cherry-pick -x f41120e

If there are conflicts resolve them before you continue with other commits. In general start with the oldest commit so that conflicts are minimized.

Sometimes a commit is a merge commit with more than one parent. The above command will fail with an error message:

error: Commit f41120e is a merge but no -m option was given.
fatal: cherry-pick failed

This discussion is helpful:

In this case look at the parents and choose which one is the ancestor. The command then is

git cherry-pick -x -m 1 f41120e

This tells git to use the first parent.

Use git show to verify what changes are going in the branch.

Github actions CI should notice the updated branch and run the tests which you can check here. Wait until the build passes before proceeding.

Create the release tag locally

At this point, you should have the release branch checked out and ready to tag.

  1. Create an annotated tag with a name that matches your intended release and a useful message. For our purposes, having the DMS Build number that this tag goes with is the current useful message:

    $ git tag -a a.b.c -m "Tagged a.b.c from main"

Publish to test.pypi.org (not currently doing)

Before proceeding, you will need to pip install twine build.

  1. Checkout the release tag, clean the directory, and make sure umask and permissions are set correctly:

    $ git checkout a.b.c
$ git clean -xdf
$ umask 0022
$ chmod -R a+Xr .

  2. Check the package setup and create the package sdist and wheel:

    $ python -m build .

  3. Upload the package to PyPi's test server (you need an account and be added as maintainer):

    $ twine check --strict dist/*
$ twine upload --repository testpypi dist/*

  4. (Needs editing. Current tests are run from JWST using STCAL as a dependency). Check that it looks good on the test server. Make sure it installs without errors in a new virtual env:

    $ pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple stcal[test]

  5. Run the tests on the installed package.

    $ pytest tests
$ pytest --pyargs jwst
$ pytest --pyargs romancal

    Or if not the above, some other way to verify that tests pass on the installed version.

  6. If the package looks good on test.pypi.org and installs OK and the tests pass, then proceed.

Make the release official on Github and Pypi

  1. If publishing to the PyPi test server above worked well, push the new tag to the upstream remote on Github:

    $ git push upstream a.b.c

  2. Go to https://github.com/spacetelescope/stcal/releases and draft a new release with the existing a.b.c tag used above. Use a release title "Tagged a.b.c from main". When saved, this will kick off the Github action to deliver the package to Pypi officially.

  3. Check that it was delivered to Pypi (https://pypi.org/project/stcal/).

Maintain the stable branch (if necessary)

The stable branch points to the latest official release of stcal. If the current release has become the latest, then the next step is to rewrite the stable branch to point our new tag.

$ git checkout stable
$ git reset --hard a.b.c
$ git push upstream stable --force

Prepare master for further development

Create and merge a PR to master updating CHANGES.rst, opening it up for new features and bugfixes. The diff should look something like this

$ git diff
diff --git a/CHANGES.rst b/CHANGES.rst
index e8771c7b..d7fac87e 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,3 +1,8 @@
+0.16.1 (unreleased)
+===================
+
+
+
0.16.0 (2020-05-04)
===================

Tag main for further development (if necessary)

If we're not already working on a later release on main (i.e. if we had to cherry-pick from main onto our release branch), then we'll need to tag the HEAD of master with a development tag. This allows the setuptools-scm plugin to identify code installed from master as the latest version. I.e. this only really needs to be done if we've done a patch release on a release branch that has cherry-picked commits on it.

$ git fetch upstream
$ git checkout upstream/master
$ git tag -a a.b.d.dev -m "Tagging a.b.d.dev"
$ git push upstream a.b.d.dev
Clone this wiki locally