Skip to content

Commit

Permalink
REL/CI: Automate release workflow
Browse files Browse the repository at this point in the history
Update release documentation and prep for automatic releases from git tags
  • Loading branch information
greglucas authored and medley56 committed Oct 1, 2024
1 parent 3bf1bf5 commit 27ac73c
Show file tree
Hide file tree
Showing 5 changed files with 190 additions and 36 deletions.
144 changes: 144 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
name: Build and upload to PyPI and create GitHub release
# https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/

concurrency:
group: ${{ github.workflow }}-${{ github.event.number }}-${{ github.event.ref }}
cancel-in-progress: true

on:
push:
tags:
- '[0-9]*.[0-9]*.[0-9]*' # Push events to every tag that looks like a semver
- '[0-9]*.[0-9]*.[0-9]*rc[0-9]*' # Push events to every tag that looks like a release candidate

jobs:
build:
# This job uses vanilla Python tools rather than Poetry, so we don't have to use third party GitHub actions
# e.g. pip, build, twine
# If we even want to, we could switch to using something like actions/setup-poetry (but do a search for current
# best implementations)
name: Build distribution 📦
runs-on: ubuntu-latest

steps:
- name: Checkout repo
uses: actions/checkout@v4

- name: Install Python 🐍
uses: actions/setup-python@v5
with:
python-version: '3.11'

- name: Install dependencies
run: python -m pip install build twine

- name: Build wheel and source distribution
run: |
python -m build
- name: Check README rendering for PyPI
run: twine check dist/*

# Save ("upload") the distribution artifacts for use by downstream Actions jobs
- name: Upload sdist artifacts 📦
uses: actions/upload-artifact@v4 # This allows us to persist the dist directory after the job has completed
with:
name: python-package-distributions
path: dist/
if-no-files-found: error

publish-to-pypi:
name: Upload release to PyPI
if: startsWith(github.ref, 'refs/tags/') # Belt and suspenders, only ever publish based on a tag
needs: build
runs-on: ubuntu-latest
environment:
name: pypi-publish
url: https://pypi.org/p/space_packet_parser
permissions:
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing

steps:
# This downloads the build artifacts from the build job
- name: Download all the dists 📦
uses: actions/download-artifact@v4
with:
name: python-package-distributions
path: dist/

- name: Publish distribution 📦 to PyPI
uses: pypa/[email protected]

create-github-release:
name: >-
Sign the Python 🐍 distribution 📦 with Sigstore
and upload them to GitHub Release
needs:
- publish-to-pypi
runs-on: ubuntu-latest
environment:
name: pypi-publish
permissions:
contents: write # IMPORTANT: mandatory for making GitHub Releases
id-token: write # IMPORTANT: mandatory for sigstore

steps:
- name: Download all the dists 📦
uses: actions/download-artifact@v4
with:
name: python-package-distributions
path: dist/

- name: Sign the dists 📦 with Sigstore 🔑
uses: sigstore/[email protected]
with:
inputs: >-
./dist/*.tar.gz
./dist/*.whl
- name: Determine if it's a pre-release
# Dynamically sets the --prerelease option passed to the release create CLI based on matching the *rc*
# substring in the git tag. If rc not present, does not pass --prerelease to the CLI.
run: |
if [[ "${{ github.ref_name }}" == *rc* ]]; then
echo "PRE_RELEASE_OPTION=--prerelease" >> $GITHUB_ENV
else
echo "PRE_RELEASE_OPTION=''" >> $GITHUB_ENV
fi
- name: Get latest non-prerelease release
# This fetches the "latest" (non-prerelease) release ref,
# so we can generate release notes from that point instead of the most recent prerelease.
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
latest_release=$(gh release list --repo "${{ github.repository }}" --limit 100 --json tagName,isPrerelease --jq '.[] | select(.isPrerelease == false) | .tagName' | head -n 1)
if [ -z "$latest_release" ]; then
echo "No non-prerelease release found."
exit 1
fi
echo "LATEST_RELEASE_TAG=$latest_release" >> $GITHUB_ENV
- name: Create GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
# Uses the GitHub CLI to generate the Release and auto-generate the release notes. Also generates
# the Release title based on the annotation on the git tag.
run: >-
gh release create
'${{ github.ref_name }}'
--repo '${{ github.repository }}'
${{ env.PRE_RELEASE_OPTION }}
--generate-notes
--notes-start-tag '${{ env.LATEST_RELEASE_TAG }}'
- name: Upload artifact 📦 signatures to GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
# Upload to GitHub Release using the `gh` CLI.
# `dist/` contains the built packages, and the
# sigstore-produced signatures and certificates.
run: >-
gh release upload
'${{ github.ref_name }}' dist/**
--repo '${{ github.repository }}'
23 changes: 12 additions & 11 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
cff-version: 1.2.0
title: 'space_packet_parser'
type: software
version: '4.2.0'
version: '5.0.0rc6'
description: A CCSDS telemetry packet decoding library based on the XTCE packet format description standard.
license: BSD-3-Clause
abstract: The space_packet_parser Python library is a generalized, configurable packet decoding library for CCSDS telemetry
packets based on the XTCE standard for packet structure definitions. The core functionality of the library is the
configuration of a Parser object by a static XTCE XML document. The configured Parser can then iterate through binary
data, parsing and returning ParsedPacket objects containing the decoded packet field values in a memory efficient
generator pattern. The binary string may originate from an in-memory bit stream, a buffered file reader opened in
binary mode, or a python socket object; in every case, a small buffer is used to read chunks of data to ensure that
memory usage is minimal. The space_packet_parser library supports robust error handling and is capable of skipping malformed
packet structures and unidentified packet structures and can dynamically parse mixed APID packet streams. The
`Parser.generator` object supports keyword arguments for debugging such as the ability to return unraised parsing
errors, which contain partial instances of parsed packet data to aid in debugging.
abstract: The Space Packet Parser Python library is a generalized, configurable packet decoding library for CCSDS telemetry
packets based on the XTCE standard for packet structure definitions. It supports complex and polymorphic
packet structures, using the XTCE UML model to represent dynamic inheritance structures and conditionals
based on previously parsed data fields. The core functionality of the library is the
configuration of an XtcePacketDefinition object from a static XTCE XML document. The configured definition
object can then iterate over binary data, parsing and yielding parsed Packet objects containing the decoded
packet field values in a generator pattern. The binary data may originate from an in-memory binary object,
a buffered file reader opened in binary mode, or a python socket object; in every case, a small buffer is
used to read chunks of data to reduce memory footprint.
The space_packet_parser library supports robust error handling, is capable of handling malformed
packet structures, and can dynamically parse muxed APID packet streams.
authors:
- email: [email protected]
name: Gavin Medley
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Space Packet Parser

----------
[![Test Status](https://github.com/medley56/space_packet_parser/actions/workflows/tests.yml/badge.svg)](https://github.com/medley56/space_packet_parser/actions/workflows/tests.yml)
[![Test Status](https://github.com/medley56/space_packet_parser/actions/workflows/pr_tests.yml/badge.svg)](https://github.com/medley56/space_packet_parser/actions/workflows/pr_tests.yml)
[![Doc Status](https://readthedocs.org/projects/space-packet-parser/badge/?version=latest)](https://readthedocs.org/projects/space-packet-parser/)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7735001.svg)](https://doi.org/10.5281/zenodo.7735001)

Expand Down
55 changes: 32 additions & 23 deletions docs/source/developers.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ docker-compose up --build && docker-compose down
```

## Building Documentation with Sphinx
Documentation is automatically built on ReadTheDocs but you can also build it locally with:
Documentation is automatically built on ReadTheDocs in response to every PR and release,
but you can also build it locally with:
```bash
# From docs directory
make html && open build/html/index.html
Expand All @@ -32,11 +33,12 @@ Feel free to fork this repo and submit a PR!
- Please fill out the PR template that is populated when creating a PR in the GitHub interface.

## Release Process
Reference: [https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)
Releases are automatically created using a GitHub Actions workflow that responds to pushes of annotated git tags.

### Preparing for Release
1. Create a release candidate branch named according to the version to be released. This branch is used to polish
the release but is fundamentally not different from any other feature branch in trunk-based development.
The naming convention is `release/X.Y.Z`.
The naming convention is `release/X.Y.Z`.

2. Bump the version of the package to the version you are about to release, either manually by editing `pyproject.toml`
or by running `poetry version X.Y.Z` or bumping according to a valid bump rule like `poetry version minor`
Expand All @@ -45,7 +47,7 @@ Reference: [https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-
3. Update the version identifier in `CITATION.cff`.

4. Update `changelog.md` to reflect that the version is now "released" and revisit `README.md` to keep it up to date.

5. Open a PR to merge the release branch into main. This informs the rest of the team how the release
process is progressing as you polish the release branch. You may need to rebase the release branch onto
any recent changes to `main` and resolve any conflicts on a regular basis.
Expand All @@ -54,22 +56,29 @@ Reference: [https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-

7. Check out the `main` branch, pull the merged changes, and tag the newly created merge commit with the
desired version `X.Y.Z` and push the tag upstream.

```bash
git tag -a X.Y.Z -m "version release X.Y.Z"
git push origin X.Y.Z
```

8. Ensure that you have `main` checked out and build the package (see below).
Check that the version of the built artifacts is as you expect (should match the version git tag and the
output from `poetry version --short`).

9. Optionally distribute the artifacts to PyPI/Nexus if desired (see below).


## Building and Distribution
1. Ensure that `poetry` is installed by running `poetry --version`.

2. To build the distribution archives, run `poetry build`.

3. To upload the wheel to Nexus, run `poetry publish`. You will need credentials to sign in to PyPI.

### Automatic Release Process
GitHub Actions has an automatic release process that responds to pushes of annotated git tags. When a tag matching
a semantic version (`[0-9]*.[0-9]*.[0-9]*` or `[0-9]*.[0-9]*.[0-9]*rc[0-9]*`) is pushed, a workflow runs that builds
the package, pushes the artifacts to PyPI, and creates a GitHub Release from the distributed artifacts. Release notes
are automatically generated from commit history and the Release name is taken from the annotation on the tag.

To trigger a release, push a tag reference to the commit you want to release, like so:

```bash
git tag -a X.Y.Z -m "Version X.Y.Z"
git push origin X.Y.Z
```

To tag and publish a Release Candidate, your tag should look like the following:

```bash
git tag -a X.Y.Zrc1 -m "Release Candidate X.Y.Zrc1"
git push origin X.Y.Zrc1
```

Release candidate tags are always marked as Prereleases in GitHub and release notes are generated from the latest
non-prerelease Release.

**For production releases, tags should always reference commits in the `main` branch. Release candidates are less
important and tags can reference any commit.**
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "space_packet_parser"
version = "4.2.0"
version = "5.0.0rc6"
description = "A CCSDS telemetry packet decoding library based on the XTCE packet format description standard."
license = "BSD-3-Clause"
readme = "README.md"
Expand Down

0 comments on commit 27ac73c

Please sign in to comment.