From 02113e29e86f6fdef5d53319ae09cbce67df68cb Mon Sep 17 00:00:00 2001 From: Irfan Alibay Date: Mon, 14 Aug 2023 19:22:37 +0100 Subject: [PATCH 1/2] Master to develop changes and completely fix the release docs (#297) * Changed several references of master and replaced them with text pointing to the develop branch. * Updated the release documents (actual examples of what needs to be done + links for Cirrus are missing, these will be added after the next release because I honestly don't even remember what needs to be done for them). * Changed / fixed some external links. --- .github/workflows/gh-ci-tests.yaml | 2 +- .github/workflows/gh-ci.yaml | 4 +- .github/workflows/pre-commit.yaml | 2 +- README.rst | 2 +- doc/source/contributing_code.rst | 28 ++-- doc/source/contributing_docs.rst | 2 +- doc/source/examples/README.rst | 2 +- .../preparing_releases_and_hotfixes.rst | 123 ++++++++++++++---- 8 files changed, 113 insertions(+), 52 deletions(-) diff --git a/.github/workflows/gh-ci-tests.yaml b/.github/workflows/gh-ci-tests.yaml index 8a7cdf683..f1d443024 100644 --- a/.github/workflows/gh-ci-tests.yaml +++ b/.github/workflows/gh-ci-tests.yaml @@ -12,7 +12,7 @@ on: concurrency: # Probably overly cautious group naming. - # Commits to develop/master will cancel each other, but PRs will only cancel + # Commits to develop will cancel each other, but PRs will only cancel # commits within the same PR group: "${{ github.ref }}-${{ github.head_ref }}-${{ github.workflow }}" cancel-in-progress: true diff --git a/.github/workflows/gh-ci.yaml b/.github/workflows/gh-ci.yaml index cb0c62345..4e3acb7e4 100644 --- a/.github/workflows/gh-ci.yaml +++ b/.github/workflows/gh-ci.yaml @@ -3,20 +3,18 @@ on: push: branches: - develop - - master tags: - "release-*" pull_request: branches: - develop - - master schedule: # 3 am Tuesdays and Fridays - cron: "0 3 * * 2,5" concurrency: # Probably overly cautious group naming. - # Commits to develop/master will cancel each other, but PRs will only cancel + # Commits to develop will cancel each other, but PRs will only cancel # commits within the same PR group: "${{ github.ref }}-${{ github.head_ref }}-${{ github.workflow }}" cancel-in-progress: true diff --git a/.github/workflows/pre-commit.yaml b/.github/workflows/pre-commit.yaml index c08ee186f..b9d74b0e0 100644 --- a/.github/workflows/pre-commit.yaml +++ b/.github/workflows/pre-commit.yaml @@ -10,7 +10,7 @@ on: concurrency: # Probably overly cautious group naming. - # Commits to develop/master will cancel each other, but PRs will only cancel + # Commits to develop will cancel each other, but PRs will only cancel # commits within the same PR group: "${{ github.ref }}-${{ github.head_ref }}-${{ github.workflow }}" cancel-in-progress: true diff --git a/README.rst b/README.rst index c7e29c37e..9346ef48d 100644 --- a/README.rst +++ b/README.rst @@ -15,7 +15,7 @@ This repository contains the `user guide`_ for MDAnalysis. The user guide origin .. _`user guide`: https://www.mdanalysis.org/UserGuide/index.html .. _`many popular formats`: https://www.mdanalysis.org/UserGuide/formats/index.html .. _MDAnalysis: https://www.mdanalysis.org -.. _LICENSE: https://github.com/MDAnalysis/mdanalysis/blob/master/LICENSE +.. _LICENSE: https://github.com/MDAnalysis/mdanalysis/blob/develop/LICENSE .. _`project for Google Season of Docs 2019-2020`: https://developers.google.com/season-of-docs/docs/participants/project-mdanalysis .. |usergroup| image:: https://img.shields.io/badge/Google%20Group-Users-lightgrey.svg diff --git a/doc/source/contributing_code.rst b/doc/source/contributing_code.rst index eb512b320..4397baa2f 100644 --- a/doc/source/contributing_code.rst +++ b/doc/source/contributing_code.rst @@ -270,9 +270,6 @@ At this point you should be able to import MDAnalysis from your locally built ve >>> mda.__version__ '2.6.0-dev0' -.. note:: - If your version number does not end in :code:`-dev0`, you may be on the ``master`` branch. In your ``mdanalysis/`` directory, switch to the ``develop`` branch: - :code:`git checkout develop`. With pip and virtualenv ----------------------- @@ -343,10 +340,6 @@ At this point you should be able to import MDAnalysis from your locally built ve >>> mda.__version__ '2.6.0-dev0' -.. note:: - If your version number does not end in :code:`-dev0`, you may be on the ``master`` branch. In your ``mdanalysis/`` directory, switch to the ``develop`` branch: - :code:`git checkout develop`. - .. _branches-in-mdanalysis: @@ -354,14 +347,9 @@ At this point you should be able to import MDAnalysis from your locally built ve Branches in MDAnalysis ---------------------- -There are two important branches in MDAnalysis: - - - ``master``: for production-ready code - - ``develop``: for development code +The most important branch of MDAnalysis is the ``develop`` branch, to which all development code for the next release is pushed. -The ``master`` branch is only for stable, production-ready code. Development code should *never* be committed to this branch. Typically, code is only committed by the release manager, when a release is ready. - -The ``develop`` branch can be considered an "integration" branch for including your code into the next release. Only working, tested code should be committed to this branch. Code contributions ("features") should branch off ``develop`` rather than ``master``. +The ``develop`` branch can be considered an "integration" branch for including your code into the next release. Only working, tested code should be committed to this branch. All code contributions ("features") should branch off ``develop``. At each release, a snapshot of the ``develop`` branch is taken, packaged and uploaded to PyPi and conda-forge. .. _create-code-branch: @@ -386,10 +374,16 @@ There are several special branch names that you should not use for your feature - ``master`` - ``develop`` - - ``release-*`` + - ``package-*`` + - ``gh-pages`` + + +``package`` branches are used to :ref:`prepare a new production release ` and should be handled by the release manager only. + +``master`` is the old stable code branch and is kept protected for historical reasons. +``gh-pages`` is where built documentation to be uploaded to github pages is held. -``release`` branches are used to :ref:`prepare a new production release ` and should be handled by the release manager only. .. _writing-new-code: @@ -673,7 +667,7 @@ in the Scientific Python community. They are nice to read as normal text and are This standard specifies the format of the different sections of the docstring. See `this document -`_ +`_ for a detailed explanation, or look at some of the existing functions to extend it in a similar manner. diff --git a/doc/source/contributing_docs.rst b/doc/source/contributing_docs.rst index 54a9252ec..62f0ba2f5 100644 --- a/doc/source/contributing_docs.rst +++ b/doc/source/contributing_docs.rst @@ -42,7 +42,7 @@ Here is an overview of the development workflow for the user guide, as expanded #. :ref:`Fork the MDAnalysis repository ` from the mdanalysis account into your own account #. :ref:`Set up an isolated virtual environment ` for your documentation - #. :ref:`Create a new branch off the master branch ` + #. :ref:`Create a new branch off the develop branch ` #. Add your new documentation. #. :ref:`Build and view the documentation `. #. :ref:`Commit and push your changes, and open a pull request `. diff --git a/doc/source/examples/README.rst b/doc/source/examples/README.rst index 118af8947..00929d298 100644 --- a/doc/source/examples/README.rst +++ b/doc/source/examples/README.rst @@ -38,4 +38,4 @@ each notebook on `Binder`_. :target: https://mybinder.org/v2/gh/MDAnalysis/binder-env-userguide/main?urlpath=git-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252FMDAnalysis%252FUserGuide%26urlpath%3Dtree%252FUserGuide%252Fdoc%252Fsource%252Fexamples%26branch%3Dmaster .. _Binder: https://mybinder.org/v2/gh/MDAnalysis/binder-env-userguide/main?urlpath=git-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252FMDAnalysis%252FUserGuide%26urlpath%3Dtree%252FUserGuide%252Fdoc%252Fsource%252Fexamples%26branch%3Dmaster -.. _GitHub: https://github.com/MDAnalysis/UserGuide/tree/master/doc/source/examples +.. _GitHub: https://github.com/MDAnalysis/UserGuide/tree/develop/doc/source/examples diff --git a/doc/source/preparing_releases_and_hotfixes.rst b/doc/source/preparing_releases_and_hotfixes.rst index 2e2256add..ac6c609f3 100644 --- a/doc/source/preparing_releases_and_hotfixes.rst +++ b/doc/source/preparing_releases_and_hotfixes.rst @@ -5,11 +5,10 @@ Preparing a release =================== -Rules for a release branch: +Rules for release branches: - - May branch from: ``develop`` - - Must be merged into: ``master`` (and ``develop`` if needed) - - Naming convention: ``release-*`` where ``*`` is a version number + - Branch from ``develop`` + - Naming convention: ``package-*`` where ``*`` is a version number Release policy and release numbering ==================================== @@ -44,24 +43,44 @@ so that people using the :ref:`develop branch ` from the Typical workflow for preparing a release ======================================== -#. Declare feature freeze on ``develop`` via the `developer mailing list`_ +Summary of tasks +---------------- -#. Create a release branch from ``develop``:: + * Declare a `feature freeze` on `develop` via discord and/or the developer mailing list + * Finalize the ``CHANGELOG`` file with the date of release + * Increment the version across ``MDAnalysis`` and ``MDAnalysisTests`` (4 places) + * `Merge` changes into `develop` + * Create a tag from `develop` named `release-` + * Create a `package-` branch from `develop` + * Add rebuilt `C / C++` files to `package-` + * Create a tag from `package-` + * Check automated testing of source distribution and wheel generation and upload + * Create a new release from newly created tag + * Check that deployment actions have adequately pushed dist and wheels to PyPi + * Manually upload Cirrus CI wheels (temporary) + * Update `conda-forge` packages + * Create a blog post outlining the release + * Increment develop branch files ready for the next version + * Clean up old dev docs builds - git checkout -b release-0.7.6 develop -#. Finalise the ``CHANGELOG`` with the release number and date. Summarize important changes and add all authors that contributed to this release. +Getting the develop branch ready for a release +---------------------------------------------- -#. Make sure the version number is right. The following files need to be updated: ``maintainer/conda/MDAnalysis/meta.yaml``, ``package/MDAnalysis/version.py``, ``package/setup.py``, ``testsuite/MDAnalysisTests/__init__.py``, and ``testsuite/setup.py``. +#. Declare feature freeze on ``develop`` via `discord` and the `developer mailing list`_ -#. Check that the documentation is up-to-date and tests pass. +#. Create a pre-release feature branch from ``develop`` -#. Commit the finalized state:: +#. Finalise the ``CHANGELOG`` with the release number and date. Ensure that the ``CHANGELOG`` summarizes important changes and includes all authors that contributed to this release. - git commit -m "release 0.7.6 ready" +#. Make sure the version number matches the release version. The following files need to be updated: ``maintainer/conda/MDAnalysis/meta.yaml``, ``package/MDAnalysis/version.py``, ``package/setup.py``, ``testsuite/MDAnalysisTests/__init__.py``, and ``testsuite/setup.py``. #. Create a pull request against ``develop`` from this branch. + +Packaging the release +--------------------- + #. Create a new tag from ``develop`` named ``release-`` where ```` is the release version number (this tag contains a snapshot of the Python source files as they were when the release was created): .. code-block: bash @@ -77,7 +96,7 @@ Typical workflow for preparing a release # MDAnalysis cd package/ - python setup.py sdist + pipx run build --sdist #. Once committed, create a new tag based on ``package-`` (this tag will contain a record of all the files as they were deployed to users for that version): @@ -86,7 +105,7 @@ Typical workflow for preparing a release git tag -m "package 0.7.5 of MDAnalysis and MDAnalysisTests" package-0.7.5 git push --tags origin -#. Upon creation of the new ``package-*`` tag, the `developer mailing list`_, will be triggered to create source/wheels, upload them to testpypi, and re-download them and run tests. +#. Upon creation of the new ``package-*`` tag, the `deploy github action`_ workflow will be automatically triggered to create source/wheels, upload them to testpypi, re-download them and run tests. #. If all the tests come back green, you are good to go for a full release. @@ -102,33 +121,83 @@ Typical workflow for preparing a release #. Add the new changes to ``develop`` and restart the release process. - #. Upon getting to the ``package-*`` tag creation, create a test tag with a different version number (bump by a minor release or add a ``-beta`` modifier), so as to trigger the release and tests against testpypi. + #. If the code had successfully uploaded to testpypi and failed later, you will need to create a test ``package-*`` tag which contains a different release number of in the source code (bumpy by a minor release or add a ``-beta`` modifier). Note: if the code had not successfully uploaded you can just continue the release process as normal. + + #. If CI comes back green then delete the test tag, and create a normal ``package-*`` tag with the correct version number. + + #. The github action will fail, but this is ok since we tested it with the test tag above. + + +Completing the release +---------------------- + +If everything works, you can now complete the release by: + +#. Creating a release on GitHub based on the newly created ``package-`` tag. + +#. Make sure you include relevant release notes, including any known issues and highlights for the release. + +#. Once published, the `deploy github action`_ will be triggered which will upload the source distributions and wheels to PyPI. + + #. If the `deploy github action`_ fails and no files have been uploaded, then restart the action. + + #. If the action fails and some files have been uploaded, then you will not be able to re-upload to PyPI. At this point you will need to yank the release from PyPI and create a new minor version and re-deploy it. + + +Manually upload Cirrus CI wheels (temporary) +-------------------------------------------- + + +.. todo:: actually add some examples & links of how to do this.` + + +Unfortunately the deployment of Cirrus CI generated wheels (for `osx-arm64` and `linux-aarch64`) does not get properly triggered by a release. + +#. Go to the ``package-*`` tag triggered Cirrus CI run results and download the generated wheels. + +#. Upload them to PyPi using ``twine``. + + +Update `conda-forge` packages +----------------------------- + +On push to PyPI, the conda-forge bot should automatically pick up the presense of a new version and create a pull request on the `MDAnalysis feedstock`_ and the `MDAnalysisTests feedstock`_. You will need to merge the MDAnalysis feedstock followed by the MDAnalysisTests feedstock in order for the new package to appear on conda-forge. + +To do this you will need to: + +#. Update the ``meta.yaml`` files as necessary, especially bumping up the python and dependency minimum versions as necessary. + +#. If NumPy pins differ from those conda-forge uses, you will need to update the ``conda_build_config.yaml`` accordingly. + +#. Ask the conda-forge bot to re-render, check that CI returns green, approve and merge the pull request. + - #. Delete the test tag, create a normal ``package-`` tag with the correct version number. +Create a blog post outlining the release +---------------------------------------- - #. The githuba action will fail, but this is ok since we tested it with the test tag above. +Create a blog post outlining the release notes and publicize it on the mailing list / discord / twitter/ etc...! -#. If everything works, you can now complete the release by createing a release on Github based on the newly created ``package- Date: Tue, 15 Aug 2023 18:13:38 +0100 Subject: [PATCH 2/2] Remove meta.yaml from list of files to update (#299) --- doc/source/preparing_releases_and_hotfixes.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/source/preparing_releases_and_hotfixes.rst b/doc/source/preparing_releases_and_hotfixes.rst index ac6c609f3..17fe36dfa 100644 --- a/doc/source/preparing_releases_and_hotfixes.rst +++ b/doc/source/preparing_releases_and_hotfixes.rst @@ -73,7 +73,7 @@ Getting the develop branch ready for a release #. Finalise the ``CHANGELOG`` with the release number and date. Ensure that the ``CHANGELOG`` summarizes important changes and includes all authors that contributed to this release. -#. Make sure the version number matches the release version. The following files need to be updated: ``maintainer/conda/MDAnalysis/meta.yaml``, ``package/MDAnalysis/version.py``, ``package/setup.py``, ``testsuite/MDAnalysisTests/__init__.py``, and ``testsuite/setup.py``. +#. Make sure the version number matches the release version. The following files need to be updated: ``package/MDAnalysis/version.py``, ``package/setup.py``, ``testsuite/MDAnalysisTests/__init__.py``, and ``testsuite/setup.py``. #. Create a pull request against ``develop`` from this branch. @@ -83,7 +83,7 @@ Packaging the release #. Create a new tag from ``develop`` named ``release-`` where ```` is the release version number (this tag contains a snapshot of the Python source files as they were when the release was created): - .. code-block: bash + .. code-block:: bash git tag -m "release 0.7.5 of MDAnalysis and MDAnalysisTests" release-0.7.5 git push --tags origin