Release Best Practices Guide #149

riverma · 2024-03-26T03:44:17Z

Purpose

A guide to help make release-making as consistent and automated as possible
PLEASE REVIEW LIVE VERSION HERE: https://github.com/NASA-AMMOS/slim/tree/issue-97/docs/guides/documentation/releases
- Help provide feedback in this PR by testing it out on your project!

Proposed Changes

[ADD] A draft guide text
[ADD] A template release.yml file to help make release notes look good and standardized with categories
[ADD] A python script to help aggregate release notes from multiple GitHub repositories into a single file

Issues

[New Process Improvement Need]: Consolidated Release Notes #97

Testing

Guide successfully rendered in SLIM docusaurus view (locally)
See template release.yml in action for the SLIM project here: https://github.com/riverma/slim/releases/tag/v1.4.0

Automation release notes aggregation script tested successfully on all SLIM repositories with the following output:

$ python gh_aggregate_release_notes.py gh_aggregate_release_notes_config.yml 
Downloading release notes: 100%|█████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  3.47it/s]
# slim-starterkit-python

## What's Changed
* Release Update of Python Starter Kit with Scanning by @ingyhere in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/14
* NASA-AMMOS/slim#89: Functional GitHub Actions-based secrets detection… by @ingyhere in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/10
* NASA-AMMOS/slim#25: Functional GitHub Actions-based SCRUB (CodeQL) analysis... by @ingyhere in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/11
* NASA-AMMOS/slim#110: Functional Pylint static code analysis. ... by @ingyhere in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/12
* Add Governance documentation by @jpl-jengelke in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/9

**Full Changelog**: https://github.com/NASA-AMMOS/slim-starterkit-python/compare/1.0.0...1.0.1

# slim-starterkit

## What's Changed
* Remove duplicate text. Standardize text for link insertion. by @jpl-jengelke in https://github.com/NASA-AMMOS/slim-starterkit/pull/2

## New Contributors
* @jpl-jengelke made their first contribution in https://github.com/NASA-AMMOS/slim-starterkit/pull/2

**Full Changelog**: https://github.com/NASA-AMMOS/slim-starterkit/commits/v1.0.0

riverma · 2024-03-26T03:57:08Z

@mike-gangl - would this help Unity make releases across many repositories? Let me know what you think.

Standardized release template for SLIM as described in #149

docs/guides/documentation/releases/README.md

nutjob4life

Token handling is probably the serious issue but see also my other comments which you can take or leave 😀

docs/guides/documentation/releases/README.md

nutjob4life · 2024-03-26T16:37:49Z

docs/guides/documentation/releases/README.md

+   2.1 Download our script 
+   - Copy/download our script to your local machine
+      ```
+      curl -o gh_aggregate_release_notes.py https://raw.githubusercontent.com/NASA-AMMOS/slim/issue-97/docs/guides/documentation/releases/gh_aggregate_release_notes.py


Personal taste: in documentation like this, I always prefer the --long format command-line arguments as they feel more "documentary" 😁 , so -o → --output

Please remember to switch that link to the main branch before releasing. I only mention it here because it is something that often slips through the cracks.

Thanks @nutjob4life - fixed in latest commit.

@jpl-jengelke - will keep this comment unresolved so I don't forget!

nutjob4life · 2024-03-26T16:38:19Z

docs/guides/documentation/releases/README.md

+
+   2.2 Create a configuration file with your project's release information. Save the file with extension `.yml` A sample is provided below:
+      ```
+      github_token: <INSERT YOUR GH PERSONAL TOKEN>


Could use a brief hint here on how to get a token—maybe down on the FAQ?

In retrospect, I think this is awfully risky. Your token could afford a lot of permissions unless it's carefully scoped—and putting it into a file makes it too easy to check into version control (yay detect-secrets!) Maybe it should be an environment variable?

My comment is to link to the GH Actions documentation directly -- I wouldn't put "" here, but instead I would make it explicit with ${{ secrets.GITHUB_TOKEN }}. Then in 2.2, something like, "Create a release notes template (.yml) that's authorized to publish your project."

I think that's OK because "secrets.GITHUB_TOKEN" is mnemonically explicit.

@nutjob4life - I was debating between an environment variable and a file. I felt that a file could be kept secure on a system with permission control, whereas the environment variable approach keeps a credential on the command-line history. Though I see the risk of committing this file and your point! Are these our only two choices, I'm curious about best practices here.

@jpl-jengelke - agreed to linking to GH directly and letting their docs handle the UI specifics.

docs/guides/documentation/releases/README.md

nutjob4life · 2024-03-26T16:39:53Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

@@ -0,0 +1,99 @@
+import requests


The script should probably include some comments at the top about dependencies. For example, it does import requests so needs a Python installation (or virtual environment) with Requests installed. Bonus points for mentioning a version number of requests too 👍 (same for tqdm, yaml).

Echoes my comment above... Also, Python 3+.

Also may want to add a little try-except error catching to explain to a casual user what's wrong. What if the template itself is munged up? That can be hard to catch.

Some minimal logging (even just print statements) may be helpful, too.

nutjob4life · 2024-03-26T16:53:08Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

+        if response.status_code == 200:
+            return response.json().get("body", "")
+        else:
+            print(f"Error fetching release notes for {url}: {response.status_code}. Retrying...")


Maybe move the retry_num += 1 up and add an if retry_num < max_retries to condition the sleep? The "edge" case is that GitHub gives non-200 every single time, but we end up doing a final sleep for 2 × 2⁴ + [0.0..1.0] seconds—up to 33 seconds—to then just abort without a final attempt.

In other words, change from attempt…sleep…attempt…sleep…attempt…sleep…attempt…sleep…abort → attempt…sleep…attempt…sleep…attempt…sleep…attempt…abort 🤷

Not a huge deal but I've been known to be rather pedantic 😂

Let me look into this thanks @nutjob4life

nutjob4life · 2024-03-26T16:57:38Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

+        print("Failed to read configuration.")
+        return
+
+    github_token = config.get("github_token")


See comment above about putting the token into the .yml (or .yaml where I come from 😁) file.

jpl-jengelke

Maybe some changes, but specifically with the token stuff, would be good.

jpl-jengelke · 2024-03-26T17:55:57Z

docs/guides/documentation/releases/README.md

+   2.1 Download our script 
+   - Copy/download our script to your local machine
+      ```
+      curl -o gh_aggregate_release_notes.py https://raw.githubusercontent.com/NASA-AMMOS/slim/issue-97/docs/guides/documentation/releases/gh_aggregate_release_notes.py


Please remember to switch that link to the main branch before releasing. I only mention it here because it is something that often slips through the cracks.

jpl-jengelke · 2024-03-26T18:08:57Z

docs/guides/documentation/releases/README.md

+
+   2.2 Create a configuration file with your project's release information. Save the file with extension `.yml` A sample is provided below:
+      ```
+      github_token: <INSERT YOUR GH PERSONAL TOKEN>


My comment is to link to the GH Actions documentation directly -- I wouldn't put "" here, but instead I would make it explicit with ${{ secrets.GITHUB_TOKEN }}. Then in 2.2, something like, "Create a release notes template (.yml) that's authorized to publish your project."

jpl-jengelke · 2024-03-26T18:09:53Z

docs/guides/documentation/releases/README.md

+
+   2.2 Create a configuration file with your project's release information. Save the file with extension `.yml` A sample is provided below:
+      ```
+      github_token: <INSERT YOUR GH PERSONAL TOKEN>


I think that's OK because "secrets.GITHUB_TOKEN" is mnemonically explicit.

jpl-jengelke · 2024-03-26T18:17:43Z

docs/guides/documentation/releases/README.md

+   2.3 Run the script to generate aggregated release notes
+      ```
+      $ python gh_aggregate_release_notes.py your_configuration_file.yml
+      ```


OK, I like the concept of building release notes based off the GH API and the script looks light enough. When I first read about this I though of using GH's built-in facilities to just copy it. Could it all be done in an action file without a separate Python component? What if the actual release notes stored in the 'releases' section of the repository are used or the macros that create them there are reused? I think if GH products are reused then GH handles maintenance, the notes are uniform and the maintenance footprint is reduced. (I can see Python versions and module releases being pesky as time goes forward, plus shouldn't there be a dependencies file (requirements.txt)?

jpl-jengelke · 2024-03-26T18:20:21Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

@@ -0,0 +1,99 @@
+import requests


Echoes my comment above... Also, Python 3+.

jpl-jengelke · 2024-03-26T18:23:10Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

+    }
+
+    max_retries = 5
+    retry_num = 0


You could use Tenacity here to automate retries over the function. It would also handle the manifest other potential exceptions that could occur.

jpl-jengelke · 2024-03-26T18:25:08Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

@@ -0,0 +1,99 @@
+import requests


Also may want to add a little try-except error catching to explain to a casual user what's wrong. What if the template itself is munged up? That can be hard to catch.

Some minimal logging (even just print statements) may be helpful, too.

jpl-jengelke · 2024-03-26T18:26:25Z

docs/guides/documentation/releases/release.yml

Having a YAML-based template, which is very pedantic on format, may be a source of issues for casual users who aren't very pedantic with spacing, etc.

pogi7 · 2024-03-27T00:09:15Z

docs/guides/documentation/releases/README.md

+1. **Customize Release Notes**:
+   - Create a `.github/release.yml` file in your repository.
+   - Download our [GitHub release notes template](release.yml) and place the contents into your `.github/release.yml` file.
+   - Customize our recommended template as a baseline and to configure how release notes are generated based on your project's issue labels.


@riverma Adding a picture(s) of how to this step generates release notes would be helpful for new users wanting to adopt SLIM.

Great suggestion @pogi7 - will do here 👍

riverma · 2024-04-03T18:19:37Z

Received a use case from the OPERA project regarding the following question: how do we safeguard that a super release will include the right versioned repositories?

In other words, can this best practice guide PR help ensure that a particular build or release of many repositories doesn't lead to a situation where the wrong version of a particular repository's code is mixed up with another repository during a super release?

CC @philipjyoon @hhlee445

mike-gangl · 2024-04-12T15:22:02Z

@mike-gangl - would this help Unity make releases across many repositories? Let me know what you think.

So in terms of unity, we keep a changelog which is where most release notes comes from; thought this can still be very useful- there is a bit of a difference in what goes into a changelog and what we would describe at a system level release (which, in my opinion, is slightly different than these super releases. I can see "super releases" on a service area basis. The System releases, however, would be manually currated and de-jargoned for a slightly different audience.

stirlingalgermissen · 2024-07-11T18:56:04Z

Recommend using AI/ML to process code commits in addition to commit and PR comments for generating release notes.

There is significant benefit if AI/ML can extract release note information from code from that has not already been captured as a developer comment.

Draft guide with release template and aggregator script automation

d9c3457

riverma self-assigned this Mar 26, 2024

riverma requested review from a team March 26, 2024 03:44

riverma added the information sharing Process improvements involving documentation, training, guides, communication, etc label Mar 26, 2024

Fixed text typos and other updates

04ce4ec

riverma added a commit that referenced this pull request Mar 26, 2024

Create release.yml

c87cab7

Standardized release template for SLIM as described in #149

nutjob4life reviewed Mar 26, 2024

View reviewed changes

docs/guides/documentation/releases/README.md Show resolved Hide resolved

nutjob4life suggested changes Mar 26, 2024

View reviewed changes

jpl-jengelke suggested changes Mar 26, 2024

View reviewed changes

pogi7 reviewed Mar 27, 2024

View reviewed changes

riverma added 2 commits March 29, 2024 16:40

Make curl args more readable

e5beb8e

Standard out language fix

243d920

riverma changed the title ~~[DRAFT] Release Best Practices Guide~~ Release Best Practices Guide Apr 4, 2024

riverma mentioned this pull request Jun 25, 2024

[New Website Feature]: Improve best practice guide template based on markdown recommendations #161

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release Best Practices Guide #149

Release Best Practices Guide #149

riverma commented Mar 26, 2024 •

edited

Loading

riverma commented Mar 26, 2024

nutjob4life left a comment

nutjob4life Mar 26, 2024

jpl-jengelke Mar 26, 2024

riverma Mar 29, 2024

nutjob4life Mar 26, 2024 •

edited

Loading

jpl-jengelke Mar 26, 2024

jpl-jengelke Mar 26, 2024

riverma Mar 29, 2024

nutjob4life Mar 26, 2024

jpl-jengelke Mar 26, 2024

jpl-jengelke Mar 26, 2024

nutjob4life Mar 26, 2024

riverma Mar 29, 2024

nutjob4life Mar 26, 2024

jpl-jengelke left a comment

jpl-jengelke Mar 26, 2024

jpl-jengelke Mar 26, 2024

jpl-jengelke Mar 26, 2024

jpl-jengelke Mar 26, 2024

jpl-jengelke Mar 26, 2024

jpl-jengelke Mar 26, 2024

jpl-jengelke Mar 26, 2024

jpl-jengelke Mar 26, 2024

pogi7 Mar 27, 2024

riverma Mar 29, 2024

riverma commented Apr 3, 2024

mike-gangl commented Apr 12, 2024

stirlingalgermissen commented Jul 11, 2024

Release Best Practices Guide #149

Are you sure you want to change the base?

Release Best Practices Guide #149

Conversation

riverma commented Mar 26, 2024 • edited Loading

Purpose

Proposed Changes

Issues

Testing

riverma commented Mar 26, 2024

nutjob4life left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nutjob4life Mar 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpl-jengelke left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

riverma commented Apr 3, 2024

mike-gangl commented Apr 12, 2024

stirlingalgermissen commented Jul 11, 2024

riverma commented Mar 26, 2024 •

edited

Loading

nutjob4life Mar 26, 2024 •

edited

Loading