Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add create release branch action #59

Merged
merged 16 commits into from
Oct 23, 2024
37 changes: 35 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,11 +255,44 @@ By default, all files in the S3 directory are uploaded. When the `dry_run` input
is set to anything other than `false`, no files are uploaded, but instead the
filename along with the resulting location in the bucket is printed.

## Create Release Branch

Use this action to create a release branch and populate it with metadata.
It will create a new Silk Asset Group, update the SBOM-lite file,
update the ``SILK_ASSET_GROUP`` and ``EVERGREEN_PROJECT`` env variables
in the release workflow file, bump the version to a
prerelease version, and push the changes.

> [!Note]
> You will need to wait overnight before making a release on
> the new branch to allow Silk to be populated, so it is recommended to
> make a minor/major release prior to creating a release branch, or create the
> release branch at least one day before a planned release.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm, this action is intended to be run in preparation for a release, not as part of the release process itself?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct


```yaml
- name: Setup
uses: mongodb-labs/drivers-github-tools/setup@v2
with:
...

- name: Create Release Branch
uses: mongodb-labs/drivers-github-tools/create-branch@v2
with:
# user inputs
branch: ...
version: ...
base_ref: <optional>
push_changes: <whether to push changes>
# other inputs
version_bump_script: <path/to/version/bump/script>
evergreen_project: <name of evergreen release project>
```

## Python Helper Scripts

These scripts are opinionated helper scripts for Python releases.

### Bump and Tag
### Pre-Publish

Bump the version and create a new tag. Verify the tag.
Push the commit and tag to the source branch unless `dry_run` is set.
Expand All @@ -270,7 +303,7 @@ Push the commit and tag to the source branch unless `dry_run` is set.
with:
...

- uses: mongodb-labs/drivers-github-tools/python/bump-and-tag@v2
- uses: mongodb-labs/drivers-github-tools/python/pre-publishv2
with:
version: ${{ inputs.version }}
version_bump_script: ./.github/scripts/bump-version.sh
Expand Down
54 changes: 54 additions & 0 deletions create-branch/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Create Release Branch
description: Create a release branch and update branch metadata
inputs:
# User provided inputs.
branch_name:
description: The name of the new branch
required: true
version:
description: The version to set on the branch
required: true
base_ref:
description: The base reference for the branch
push_changes:
description: Whether to push the changes
default: "true"
# Workflow provided inputs.
version_bump_script:
description: The script used to bump the version
required: true
evergreen_project:
description: The name of the evergreen project for the new branch
required: true
release_workflow_path:
description: The path to the release workflow file
default: .github/workflows/release.yml
sbom_file_path:
description: The path of the sbom-lite file
default: sbom.json
silk_group_prefix:
description: The prefix to use for the silk asset group, defaults to the repo name
Comment on lines +29 to +30
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense that we use a prefix here since we'll now have to include the branch name into the silk group. However, the sbom group takes a silk_asset_group, so if nothing else we should document how these two actions will play nice together in future.

artifactory_image:
description: Image to use for artifactory
default: artifactory.corp.mongodb.com/release-tools-container-registry-public-local

runs:
using: composite
steps:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I like the flow of creating a branch and updating version info. For example, on PHP we have v1.x as a development branch for the next minor version, and v1.19.x as a maintenance branch. However, v1.x already has version info for the next minor version (so 1.20.0-dev at the moment), and when we branch off for 1.20 we would update v1.x to 1.21.0-dev. Could you explain the branch management scheme that you envision for this script?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For release branches in Go and Python, we would release from the main branch which is targeting the next minor release, then create a release branch for that minor version. The release branch's version would target the next patch version, so 1.20.1-dev in your example.

- name: Create a release branch and update branch metadata
shell: bash
env:
BRANCH: ${{ inputs.branch_name }}
BASE_REF: ${{ inputs.base_ref }}
SBOM_FILE_PATH: ${{ inputs.sbom_file_path }}
RELEASE_WORKFLOW_PATH: ${{ inputs.release_workflow_path }}
EVERGREEN_PROJECT: ${{ inputs.evergreen_project }}
SILK_PREFIX: ${{ inputs.silk_group_prefix }}
ARTIFACTORY_IMAGE: ${{ inputs.artifactory_image }}
run: ${{ github.action_path }}/create-branch.sh
- uses: mongodb-labs/drivers-github-tools/bump-version@v2
with:
version: ${{ inputs.version }}
version_bump_script: ${{ inputs.version_bump_script }}
commit_template: "Prep branch ${{ inputs.branch_name }}"
push_commit: ${{ inputs.push_changes }}
76 changes: 76 additions & 0 deletions create-branch/create-branch.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#! /bin/bash
set -eu

echo "Create or checkout the branch."
OWNER_REPO="${GITHUB_REPOSITORY}"
git ls-remote --exit-code --heads https://github.com/${OWNER_REPO}.git refs/heads/$BRANCH || {
git branch $BRANCH $BASE_REF
}
git fetch origin $BRANCH || true
git checkout $BRANCH

echo "Get silk creds."
# shellcheck disable=SC2046
export $(grep -v '^#' $SILKBOMB_ENVFILE | xargs -0)

echo "Get a silk token."
SILK_JWT_TOKEN=$(curl -s -X POST "https://silkapi.us1.app.silk.security/api/v1/authenticate" \
-H "accept: application/json" -H "Content-Type: application/json" \
-d '{ "client_id": "'${SILK_CLIENT_ID}'", "client_secret": "'${SILK_CLIENT_SECRET}'" }' \
| jq -r '.token')

echo "Get the silk asset group prefix."
if [ -z "${SILK_PREFIX:-}" ]; then
REPO="${OWNER_REPO##*/}"
SILK_PREFIX=${REPO}
fi
SILK_GROUP="${SILK_PREFIX}-${BRANCH}"

echo "Create the silk asset group."
json_payload=$(cat <<EOF
{
"active": true,
"name": "${SILK_GROUP}",
"code_repo_url": "https://github.com/${OWNER_REPO}",
"branch": "${BRANCH}",
"metadata": {
"sbom_lite_path": "${SBOM_FILE_PATH}"
},
"file_paths": [],
"asset_id": "$SILK_GROUP"
}
EOF
)
Comment on lines +30 to +43
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that this config works for all of our repos; if not we can always make this customisable or use a template asset group that we fetch and modify.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This config works for all repos that I maintain, including libmonocrypt/bindings/python, but yeah, we could make it templatable if needed.

curl -X 'POST' \
'https://silkapi.us1.app.silk.security/api/v1/raw/asset_group' \
-H "accept: application/json" -H "Authorization: ${SILK_JWT_TOKEN}" \
-H 'Content-Type: application/json' \
-d "$json_payload"

echo "SILK_ASSET_GROUP=$SILK_GROUP" >> $GITHUB_STEP_SUMMARY

echo "Create a temp sbom."
TMP_SBOM=sbom-for-${BRANCH}.json
podman run --platform="linux/amd64" --rm -v "$(pwd)":/pwd \
${ARTIFACTORY_IMAGE}/silkbomb:1.0 \
update --sbom-out /pwd/${TMP_SBOM}

echo "Get the new timestamp and serial number."
set -x
SERIAL=$(jq -r '.serialNumber' ${TMP_SBOM})
TIMESTAMP=$(jq -r '.metadata.timestamp' ${TMP_SBOM})
rm ${TMP_SBOM}

cat ${SBOM_FILE_PATH}
echo "Replace the values in the existing sbom."
cat <<< "$(jq --indent 4 '.serialNumber = "'${SERIAL}'"' ${SBOM_FILE_PATH})" > ${SBOM_FILE_PATH}
cat <<< "$(jq --indent 4 '.metadata.timestamp = "'${TIMESTAMP}'"' ${SBOM_FILE_PATH})" > ${SBOM_FILE_PATH}
cat ${SBOM_FILE_PATH}
Comment on lines +52 to +68
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow exactly what's happening here. We fetch the SBOM for the new branch, then bump the serial number in the SBOM file path - why is that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SBOM needs to be unique per branch if I understand correctly.


echo "Update the workflow with the silk asset group and evergreen project."
sed -i 's/SILK_ASSET_GROUP:.*/SILK_ASSET_GROUP: '${SILK_GROUP}'/' ${RELEASE_WORKFLOW_PATH}
sed -i 's/EVERGREEN_PROJECT:.*/EVERGREEN_PROJECT: '${EVERGREEN_PROJECT}'/' ${RELEASE_WORKFLOW_PATH}
Comment on lines +70 to +72
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this relates to my earlier comment about the asset group prefix vs. hardcoded asset name. I'd personally change the release tooling to use the same prefix logic for the silk asset group.
As for evergreen project name, we don't use that in our PHP release workflow, but I can see how other drivers may need it. Wouldn't it make more sense to use the same <prefix>-<branch-name> for those names so we don't need to update this information for each release branch?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use the evergreen project name to tie the release to an Evergreen run url, which is included in the compliance report. Doesn't each branch need its own Evergreen project? Either way, we could make this part optional if no evergreen project is given.


echo "Add the changed files."
git --no-pager diff
git add ${SBOM_FILE_PATH} ${RELEASE_WORKFLOW_PATH}
Loading