Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add directions to playbook for Science Pipelines quick-builds #242

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add directions to playbook for Science Pipelines quick-builds #242

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b37ec0e
Add directions to playbook for Science Pipelines quick-builds
eigerx Dec 6, 2024
b3e5bcd
Fix numbering for Release Management playbook section
eigerx Dec 6, 2024
62e1136
Small edits to playbook formatting, for clarity
eigerx Dec 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 98 additions & 58 deletions doc/playbook.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,18 @@ Containers
==========

The service consists of two containers.
The first is a base container with the Science Pipelines "stack" code and networking utilities.
The second is a service container made from the base that has the Prompt Processing service code.
The first is a **base container** with the Science Pipelines "stack" code and networking utilities.
The second is a **service container** made from the base that has the Prompt Processing service code.
All containers are managed by `GitHub Container Registry <https://github.com/orgs/lsst-dm/packages?repo_name=prompt_processing>`_ and are built using GitHub Actions.

To build the base container:

* If there are changes to the container, push them to a branch, then open a PR.
The container should be built automatically.
* If there are no changes (typically because you want to use an updated Science Pipelines container), go to the repository's `Actions tab <https://github.com/lsst-dm/prompt_processing/actions/workflows/build-base.yml>`_ and select "Run workflow".
From the dropdown, select the branch whose container definition will be used, and the label of the Science Pipelines container.
This will build the container automatically.
* If there are no changes (typically because you want to use an updated Science Pipelines container), go to the repository's `Actions tab <https://github.com/lsst-dm/prompt_processing/actions/workflows/build-base.yml>`_ and select "**Run workflow**". From the dropdown menu, select:

#. The branch whose container definition will be used
#. The label of the Science Pipelines container.
Copy link
Member

@kfindeisen kfindeisen Dec 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to include an entry for the science pipelines container (possibly adding it on commit "Add directions to playbook..."?) AFAICT that's the step you and Erin missed over the weekend.

* New containers built from ``main`` are tagged with the corresponding Science Pipelines release (plus ``w_latest`` or ``d_latest`` if the release was requested by that name).
For automatic ``main`` builds, or if the corresponding box in the manual build is checked, the new container also has the ``latest`` label.
Containers built from a branch use the same scheme, but prefixed by the ticket number or, for user branches, the branch topic.
Expand All @@ -44,14 +46,16 @@ To build the base container:
To build the service container:

* If there are changes to the service, push them to a branch, then open a PR.
The container should be built automatically using the ``latest`` base container.
* To force a rebuild manually, go to the repository's `Actions tab <https://github.com/lsst-dm/prompt_processing/actions/workflows/build-service.yml>`_ and select "Run workflow".
This will build the container automatically using the ``latest`` base container.
* To force a rebuild manually, go to the repository's `Actions tab <https://github.com/lsst-dm/prompt_processing/actions/workflows/build-service.yml>`_ and select "**Run workflow**".
From the dropdown, select the branch whose code should be built.
The container will be built using the ``latest`` base container, even if there is a branch build of the base.
* To use a base other than ``latest``, edit ``.github/workflows/build-service.yml`` on the branch and override the ``BASE_TAG_LIST`` variable.
Be careful not to merge the temporary override to ``main``!
By default, the container is built using the ``latest`` base container, even if there is a branch build of the base.

* If this is the desired base container, no action is necessary.
* To use a base other than ``latest``, edit ``.github/workflows/build-service.yml`` on the branch and override the ``BASE_TAG_LIST`` variable.
Be careful not to merge the temporary override to ``main``!
* New service containers built from ``main`` have the tags of their base container.
Containers built from a branch are prefixed by the ticket number or, for user branches, the branch topic.
Containers built from a branch should be prefixed by the ticket number or, for user branches, the branch topic.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this rewording. Have you seen cases where this doesn't happen (if so, that's a bug in the script)?


.. note::

Expand All @@ -67,6 +71,36 @@ Any subsequent builds of the service container will build against both bases.

This is the only situation in which a change to ``BASE_TAG_LIST`` should be committed to ``main``.

In case your intended Base Container is not a tagged Science Pipelines Release: Building the Science Pipelines Manually
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be shortened to "Building the Science Pipelines Manually"? The current title wraps rather awkwardly on my browser.

Or maybe "Using the Quick-Stack Build", since that's almost universally what we call it elsewhere and what a new user would be looking for?

-----------------------------------------------------------------------------------------------------------------------
It will sometimes be necessary to compile a container with the LSST Science Pipelines manually. Generally, this only occurs if the intended daily or weekly stack does not compile. In these cases, the Science Pipelines themselves must be built ahead of the base container. Here are instructions for building the Science Pipelines.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me that, since this is not Prompt Processing functionality, it should be documented elsewhere (https://github.com/lsst/gha_build/?) and merely linked from here. That would keep our documentation focused (see DMTN-030), and would make the build documentation visible to other users of the quick-stack system.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please write (at most) one sentence per line, per the dev guide. That makes for much cleaner diffs in future editing; otherwise, changing one word might be treated as a rewrite of an entire paragraph!


#. Check the `Science Pipelines changelog <https://lsst-dm.github.io/lsst_git_changelog/weekly/>`_ to make sure only the intended changes are on ``main``.

If ``main`` contains only changes that are intended for the Science Pipelines build, proceed and build the main branch! Otherwise, a cherry-pick or backport branch should be built.
Copy link
Member

@kfindeisen kfindeisen Dec 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given my next comment, I assume this is actually backport branches in each Science Pipelines package with changes of interest. If this is correct, please say so.


#. Go to the `GitHub Actions build system for the Science Pipelines. <https://github.com/lsst/gha_build/actions/workflows/build.yaml>`_

#. Select "**Run Workflow**."" In the drop-down menu, fill in the following details:

#. **Use workflow from**

In almost all cases, you'll want the ``main`` branch. Generally speaking, this is the branch of the Science Pipelines that you're trying to build. If there is a specific branch for a cherry-pick or backport, you'll want that branch instead.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is conflating "Use workflow from" (which I agree you usually don't want to touch) and "List of refs to build" (where I would expect to almost always see backport branches).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More a style question, but... why use ordered (numbered) lists instead of unordered lists for fields? They all appear together on the same form. Same question for the reformatting of the PP workflow later.


#. **List of products to build**

Insert products to build, just like for Jenkins, but the default is what the standard containers have (except ``pipelines_check``).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see only lsst_distrib lsst_sitcom, and not pipelines_check?


#. **Version of rubin-env conda environment**

Make sure this matches the version for the current LSST Science Pipelines! At time of writing, the GitHub Actions page suggests version ``8.0.0`` and this needed to be set to version ``9.0.0``

#. **Container tag**

Select a tag for the container that reflects its contents (e.g. ``d_2024_06_21_DM-44996`` or ``or4_pp_20240625``). The rubin-env version will be appended.

Click "**Run Workflow**". The resulting container, if things succeed, will be available at ``ghcr.io/lsst/quick-stack:{your tag here}-{rubin-env version}``.

Release Management
==================

Expand All @@ -78,76 +112,82 @@ From the ``main`` branch you can release a new major version (``X.0.0``), a new
Release tags are semantic version identifiers following the `pep 440 <https://peps.python.org/pep-0440/>`_ specification.
Please note that the tag does not include a ``v`` at the beginning.

1. Choose Version Number
#. Choosing the Version Number

On GitHub.com, navigate to the main page of the repository.
To the right of the list of files, click the latest release.
At the top of the page, click **## commits to main since this release**.
(If there's no such link or it doesn't mention ``main``, the release is probably based off a branch; go up to Releases and try older versions until you find one.)
This is the list of internal changes that will be included in the next release.
The first step in making a release is choosing an appropriate version number to describe it. This subsection offers guidance on this choice, as well as on differentiating between major and minor releases.

If you are planning to update the Science Pipelines tag, you should also check the `Science Pipelines changelog <https://lsst-dm.github.io/lsst_git_changelog/weekly/>`_.
In practice, almost any Science Pipelines update is at least a minor version, because new features are added constantly.
In the future, there may be "patched weekly" builds, which would justify a patch version of Prompt Processing.
**To see which changes have occurred since the last release:**

For the ``prompt_processing`` service, a new major version is triggered by any of the following:
On GitHub.com, navigate to the main page of the repository.
To the right of the list of files, click the latest release.
At the top of the page, click **## commits to main since this release**.
(If there's no such link or it doesn't mention ``main``, the release is probably based off a branch; go up to Releases and try older versions until you find one.)
This is the list of internal changes that will be included in the next release.

* Incompatibility with old fanned-out ``nextVisit`` messages (almost any change to ``Visit`` qualifies)
* Incompatibility with an old `APDB schema`_, `ApdbSql`_, or `ApdbCassandra`_ version (see `DMTN-269`_ for the distinction)
* Incompatibility with an old `Butler dimensions-config`_ version
* A new major version of the `Alerts schema`_ (see `DMTN-093`_ for details)
If you are planning to update the Science Pipelines tag, you should also check the `Science Pipelines changelog <https://lsst-dm.github.io/lsst_git_changelog/weekly/>`_.
In practice, almost any Science Pipelines update is at least a minor version, because new features are added constantly.
In the future, there may be "patched weekly" builds, which would justify a patch version of Prompt Processing.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could swear we removed this line, but now it should probably refer to quick-stack.


For the `next_visit_fan_out`_ service, a new major version is triggered by any of the following:
**Guidance on designating major releases:**

* Incompatibility with old Summit ``nextVisit`` messages
* Breaking changes in the fanned-out ``nextVisit`` messages (almost any change to ``NextVisitModel`` qualifies)
For the ``prompt_processing`` service, a new major version is triggered by any of the following:

2. Create a Release
* Incompatibility with old fanned-out ``nextVisit`` messages (almost any change to ``Visit`` qualifies)
* Incompatibility with an old `APDB schema`_, `ApdbSql`_, or `ApdbCassandra`_ version (see `DMTN-269`_ for the distinction)
* Incompatibility with an old `Butler dimensions-config`_ version
* A new major version of the `Alerts schema`_ (see `DMTN-093`_ for details)

On GitHub.com, navigate to the main page of the repository.
To the right of the list of files, click **Releases**.
At the top of the page, click **Draft a new release**.
Type a tag using semantic versioning described in the previous section.
The Target should be the main branch.
For the `next_visit_fan_out`_ service, a new major version is triggered by any of the following:

Select **Generate Release Notes**.
This will generate a list of commit summaries and of submitters.
Add text as follows.
* Incompatibility with old Summit ``nextVisit`` messages
* Breaking changes in the fanned-out ``nextVisit`` messages (almost any change to ``NextVisitModel`` qualifies)

* Any specific motivation for the release (for example, including a specific feature, preparing for a specific observing run)
* Science Pipelines version and rubin-env version
* Supported `APDB schema`_ and `ApdbSql`_/`ApdbCassandra`_ versions (see `DMTN-269`_ for rationale).
You do *not* need to consider the `ApdbCassandraReplica` version.
A stack quoting a given minor version is compatible with *older* APDBs of that major version but not necessarily newer ones; for example, a release whose baseline is APDB schema 1.4.0 can access a schema 1.0.0 or 1.4.1 database, but not schema 1.5.
* Supported `Butler dimensions-config`_ versions
* The `Alerts schema`_ version used for output (see `DMTN-093`_ for details)
#. Create a Release

.. _DMTN-093: https://dmtn-093.lsst.io/#alertmanagement
On GitHub.com, navigate to the main page of the repository.
To the right of the list of files, click **Releases**.
At the top of the page, click **Draft a new release**.
Type a tag using semantic versioning described in the previous section.
The Target should be the main branch.

.. _DMTN-269: https://dmtn-269.lsst.io/
Select **Generate Release Notes**.
This will generate a list of commit summaries and of submitters.
Add text as follows:

.. _Butler dimensions-config: https://pipelines.lsst.io/v/daily/modules/lsst.daf.butler/dimensions.html#dimension-universe-change-history
* Any specific motivation for the release (for example, including a specific feature, preparing for a specific observing run)
* Science Pipelines version and rubin-env version
* Supported `APDB schema`_ and `ApdbSql`_/`ApdbCassandra`_ versions (see `DMTN-269`_ for rationale).
You do *not* need to consider the `ApdbCassandraReplica` version.
A stack quoting a given minor version is compatible with *older* APDBs of that major version but not necessarily newer ones; for example, a release whose baseline is APDB schema 1.4.0 can access a schema 1.0.0 or 1.4.1 database, but not schema 1.5.
* Supported `Butler dimensions-config`_ versions
* The `Alerts schema`_ version used for output (see `DMTN-093`_ for details)

.. _APDB schema: https://github.com/lsst/sdm_schemas/blob/main/python/lsst/sdm_schemas/schemas/apdb.yaml#L4
.. _DMTN-093: https://dmtn-093.lsst.io/#alertmanagement

.. _ApdbSql: https://github.com/lsst/dax_apdb/blob/main/python/lsst/dax/apdb/sql/apdbSql.py#L72-L76
.. _DMTN-269: https://dmtn-269.lsst.io/

.. _ApdbCassandra: https://github.com/lsst/dax_apdb/blob/main/python/lsst/dax/apdb/cassandra/apdbCassandra.py#L85-L89
.. _Butler dimensions-config: https://pipelines.lsst.io/v/daily/modules/lsst.daf.butler/dimensions.html#dimension-universe-change-history

.. _Alerts schema: https://github.com/lsst/alert_packet/blob/main/python/lsst/alert/packet/schema/latest.txt
.. _APDB schema: https://github.com/lsst/sdm_schemas/blob/main/python/lsst/sdm_schemas/schemas/apdb.yaml#L4

Select **Publish Release**.
.. _ApdbSql: https://github.com/lsst/dax_apdb/blob/main/python/lsst/dax/apdb/sql/apdbSql.py#L72-L76

The `ci-release.yaml <https://github.com/lsst-dm/prompt_processing/actions/workflows/ci-release.yaml>`_ GitHub Actions workflow uploads the new release to GitHub packages.
.. _ApdbCassandra: https://github.com/lsst/dax_apdb/blob/main/python/lsst/dax/apdb/cassandra/apdbCassandra.py#L85-L89

3. Tag the release
.. _Alerts schema: https://github.com/lsst/alert_packet/blob/main/python/lsst/alert/packet/schema/latest.txt

At the HEAD of the ``main`` branch, create and push a tag with the semantic version:
Select **Publish Release**.

.. code-block:: sh
The `ci-release.yaml <https://github.com/lsst-dm/prompt_processing/actions/workflows/ci-release.yaml>`_ GitHub Actions workflow uploads the new release to GitHub packages.

#. Tag the release

At the HEAD of the ``main`` branch, create and push a tag with the semantic version:

.. code-block:: sh

git tag -s X.Y.Z -m "X.Y.Z"
git push --tags
git tag -s X.Y.Z -m "X.Y.Z"
git push --tags

Patch Releases and Release Branches
-----------------------------------
Expand Down
Loading