Adds initial release automator #633

chuckha · 2019-03-07T21:06:39Z

Signed-off-by: Chuck Ha [email protected]

What this PR does / why we need it:

This PR adds a basic release tool to help us get started with automated releases. The idea is to run it to create a draft release that we edit and publish by hand when we're satisfied with the release notes.

Here is an example run:

 salazar:cluster-api-provider-aws cha$ go run ./cmd/release/main.go -user chuckha -remote chuckha -version v1.2.6
tagging repository 🐲
pushing tag 🐲
drafting a release 🐲
uploading "cluster-api-provider-aws-examples.tar" 🐲
uploading "clusterawsadm-darwin-amd64" 🐲
uploading "clusterawsadm-linux-amd64" 🐲
uploading "clusterctl-darwin-amd64" 🐲
uploading "clusterctl-linux-amd64" 🐲
The next steps are:

- Write the release notes. It is a manual process
- push the container images
- Email [email protected] to announce that a release happened

And this is what the generated release will look like:

There is opportunity to improve this script greatly and push it up to cluster-api to help with the releases there.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #122

Special notes for your reviewer:

Has a documented dependency on github.com/itchio/gothub that could be removed by using GitHub API. This was installed and used for expediency.

install with go get github.com/itchio/gothub

Release note:
No user facing changes

NONE

k8s-ci-robot · 2019-03-07T21:06:52Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chuckha

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [chuckha]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

vincepri

Nice start. One comment for a typo, otherwise lgtm. Have you considered a simple bash script instead of a Go file? I don't mind either way, more curious if you have a plan here.

vincepri · 2019-03-07T21:12:16Z

cmd/release/main.go

+)
+
+/*
+required parameteres


Suggested change

required parameteres

required parameters

vincepri · 2019-03-07T21:13:37Z

cmd/release/main.go

+const (
+	// TODO figure this out based on directory name
+	repository   = "cluster-api-provider-aws"
+	artifactsDir = "out"


These should probably be flags at some point

chuckha · 2019-03-07T21:48:11Z

@vincepri I'm really hoping to avoid this https://github.com/kubernetes/release/blob/master/anago with regards to go vs bash.

There is nothing wrong with that bash script, in fact it's one of the best bash scripts/libraries I've ever seen, but I find it harder to maintain a bash script than a go script. I do expect there to be more than just RunCommand called in future if that helps.

The other piece is Integrating this with a configuration language. As you pointed out, flags/config would be really helpful here. It will be a lot easier to ingest and reference values in a config file via go than bash. The goal here (eventually) would be that if we have a nice configuration file for providers to fill out, they get release tooling for free

cmd/release/main.go

detiber

lgtm outside of the question

detiber · 2019-03-07T22:25:25Z

cmd/release/main.go

+
+	// create draft release
+	// TODO: not everyone names the remotes after the username
+	RunCommand("gothub", "release", "--tag", version, "--user", remote, "--repo", repository, "--draft")


Would overriding the upstream for the working branch (using git branch --set-upstream-toaffect this? For example I override the upstream for my master branch to upstream/master?

Nope, that would be fine. Since this is a github specific release tool, it will assume the remote as "github.com//" regardless of local git settings. So I think you're pointing at a misconception that I had when writing this -- it actually is GitHub user and my variable name is bad. Will fix.

ncdc · 2019-03-08T17:06:46Z

cmd/release/main.go

+	RunCommand("make", "release-artifacts")
+
+	// Creates git tag
+	RunCommand("git", "tag", "--force", version)


I would advise against --force. If you need to wipe & recreate a tag, delete it manually first.

I would also recommend either doing an annotated tag or a GPG signed tag.

ah yes, this sounds great, thanks for the feedback

ncdc · 2019-03-08T17:06:59Z

/cc

ncdc · 2019-03-08T17:22:51Z

cmd/release/main.go

+	// TODO(chuckha): it would be ideal if we could release major/minor/patch and have it
+	// automatically bump the latest tag git finds
+	// until then, hard code it
+	remote := os.Args[1] // default to origin probably


How would you feel about these being flags instead of positional arguments?

ncdc · 2019-03-08T17:25:47Z

I'm curious (with no strong feelings necessarily either way) why did you choose to write this in Go instead of bash or make or python or whatever?

chuckha · 2019-03-08T17:26:47Z

I'm curious (with no strong feelings necessarily either way) why did you choose to write this in Go instead of bash or make or python or whatever?

#633 (comment)

ncdc · 2019-03-08T17:28:16Z

Thanks, SGTM. I'd recommend thinking about how you want to unit test this tool, if applicable. To me, that's one major advantage to writing something like this in NotBash

chuckha · 2019-03-08T17:31:14Z

Thanks, SGTM. I'd recommend thinking about how you want to unit test this tool, if applicable. To me, that's one major advantage to writing something like this in NotBash

Testing yeah, it's definitely an advantage. Standard DI testing is the way to go here. That will both make the code better and let use more easily get rid of the gothub dependency.

Looks like flags got requested twice, so that is enough signal that people really think flags should be in the initial release. I'll update this PR

randomvariable · 2019-03-08T17:54:22Z

lgtm for the initial take

chuckha · 2019-03-08T18:35:01Z

/hold for updates

chuckha · 2019-03-08T18:35:11Z

/hold

for updates*

ashish-amarnath · 2019-03-08T18:49:20Z

cmd/release/main.go

+
+	// attach tarball of yaml and binaries for systems to the release
+	for _, file := range expectedFiles {
+		RunCommand("gothub", "upload", "--tag", version, "--user", user, "--repo", repository, "--file", path.Join(artifactsDir, file), "--name", file)


What are the failure modes for this?
Should we perform any cleanup should this fail, before allowing a retry?

ashish-amarnath · 2019-03-08T18:50:16Z

cmd/release/main.go

+
+	// Build all the release binaries
+	RunCommand("make", "release-artifacts")
+


suggest adding a validate artifacts step to confirm all expected files are present.

ashish-amarnath · 2019-03-08T18:50:59Z

cmd/release/main.go

+	version := os.Args[3]
+
+	// Build all the release binaries
+	RunCommand("make", "release-artifacts")


Suggested change

RunCommand("make", "release-artifacts")

RunCommand("make", "clean", "release-artifacts")

ashish-amarnath · 2019-03-08T18:53:18Z

cmd/release/main.go

+	}
+}
+
+func mustSatisfyDependencies() {


nit: suggest rename ensureDependencies

justaugustus

Can we include documentation on usage as well in the docs/releasing.md, that includes any gotchas and hardcoded parameters?
I'll plan to copy this into capz and use it for our releases as well, but I want to make I do so in the way that it was intended.

detiber · 2019-03-08T20:41:30Z

/milestone v1alpha1

chuckha · 2019-03-08T21:06:42Z

@justaugustus I don't think it's ready for that. We'll start using it for this repo and make it fit other uses cases as we go forward.

chuckha · 2019-03-08T21:13:31Z

/hold cancel

Signed-off-by: Chuck Ha <[email protected]>

ashish-amarnath · 2019-03-08T21:29:31Z

docs/releasing.md

@@ -1,17 +1,25 @@
 # Release process

+## Semi-automatic
+
+1. make the release artifacts `make release artifacts`


Suggested change

1. make the release artifacts `make release artifacts`

1. make the release artifacts `make clean release-artifacts`

what's the benefit of adding clean? release-artifacts are a PHONY target and will always overwrite whatever exists.

Clean will simply clean up the dev environment a user may already have

clean will delete existing examples prior to running release-artifacts target. This will ensure that the release will have newly generated examples in it.

Oh.. I see. the release-artifactsdoesn't call other make targets. so clean not required.

ashish-amarnath · 2019-03-08T21:30:05Z

docs/releasing.md

-5. Attach the tarball to the drafted release
-6. Attach `clusterawsadm` and `clusterctl` to the drafted release (for darwin
+2. Tag the repository and push the tag `git tag -s $VERSION `
+3. Run `make release-artifacts`


Suggested change

3. Run `make release-artifacts`

3. Run `make clean release-artifacts`

docs/releasing.md

vincepri

/lgtm

Bumped golang to 1.20.2

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Mar 7, 2019

k8s-ci-robot requested review from randomvariable and vincepri March 7, 2019 21:06

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 7, 2019

vincepri reviewed Mar 7, 2019

View reviewed changes

detiber reviewed Mar 7, 2019

View reviewed changes

cmd/release/main.go Show resolved Hide resolved

detiber reviewed Mar 7, 2019

View reviewed changes

chuckha force-pushed the release-tool branch 3 times, most recently from 77eae78 to a9ce6bc Compare March 8, 2019 16:33

ncdc reviewed Mar 8, 2019

View reviewed changes

k8s-ci-robot requested a review from ncdc March 8, 2019 17:06

ncdc reviewed Mar 8, 2019

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 8, 2019

ashish-amarnath reviewed Mar 8, 2019

View reviewed changes

justaugustus suggested changes Mar 8, 2019

View reviewed changes

k8s-ci-robot added this to the v1alpha1 milestone Mar 8, 2019

chuckha force-pushed the release-tool branch from a9ce6bc to 0e0f8f1 Compare March 8, 2019 21:04

chuckha force-pushed the release-tool branch from 0e0f8f1 to 41d43df Compare March 8, 2019 21:13

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 8, 2019

chuckha force-pushed the release-tool branch from 41d43df to 38927ac Compare March 8, 2019 21:17

Adds initial release automator

b614978

Signed-off-by: Chuck Ha <[email protected]>

chuckha force-pushed the release-tool branch from 38927ac to b614978 Compare March 8, 2019 21:26

ashish-amarnath reviewed Mar 8, 2019

View reviewed changes

detiber reviewed Mar 8, 2019

View reviewed changes

docs/releasing.md Outdated Show resolved Hide resolved

Update docs/releasing.md

4494489

vincepri approved these changes Mar 8, 2019

View reviewed changes

k8s-ci-robot assigned vincepri Mar 8, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 8, 2019

k8s-ci-robot merged commit 1235f41 into kubernetes-sigs:master Mar 8, 2019

luthermonson pushed a commit to luthermonson/cluster-api-provider-aws that referenced this pull request Apr 3, 2023

Merge pull request kubernetes-sigs#633 from spectrocloud/securityScan

8200bfc

Bumped golang to 1.20.2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds initial release automator #633

Adds initial release automator #633

chuckha commented Mar 7, 2019 •

edited

Loading

k8s-ci-robot commented Mar 7, 2019

vincepri left a comment

vincepri Mar 7, 2019

vincepri Mar 7, 2019

chuckha Mar 7, 2019

chuckha commented Mar 7, 2019

detiber left a comment

detiber Mar 7, 2019

chuckha Mar 8, 2019

ncdc Mar 8, 2019

chuckha Mar 8, 2019

ncdc commented Mar 8, 2019

ncdc Mar 8, 2019

ncdc commented Mar 8, 2019

chuckha commented Mar 8, 2019

ncdc commented Mar 8, 2019

chuckha commented Mar 8, 2019

randomvariable commented Mar 8, 2019

chuckha commented Mar 8, 2019

chuckha commented Mar 8, 2019

ashish-amarnath Mar 8, 2019

ashish-amarnath Mar 8, 2019

ashish-amarnath Mar 8, 2019

ashish-amarnath Mar 8, 2019 •

edited

Loading

justaugustus left a comment

detiber commented Mar 8, 2019

chuckha commented Mar 8, 2019

chuckha commented Mar 8, 2019

ashish-amarnath Mar 8, 2019

chuckha Mar 8, 2019

ashish-amarnath Mar 8, 2019

ashish-amarnath Mar 8, 2019

ashish-amarnath Mar 8, 2019

vincepri left a comment


		// Build all the release binaries
		RunCommand("make", "release-artifacts")

	RunCommand("make", "release-artifacts")
	RunCommand("make", "clean", "release-artifacts")

	1. make the release artifacts `make release artifacts`
	1. make the release artifacts `make clean release-artifacts`

	3. Run `make release-artifacts`
	3. Run `make clean release-artifacts`

Adds initial release automator #633

Adds initial release automator #633

Conversation

chuckha commented Mar 7, 2019 • edited Loading

k8s-ci-robot commented Mar 7, 2019

vincepri left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chuckha commented Mar 7, 2019

detiber left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ncdc commented Mar 8, 2019

Choose a reason for hiding this comment

ncdc commented Mar 8, 2019

chuckha commented Mar 8, 2019

ncdc commented Mar 8, 2019

chuckha commented Mar 8, 2019

randomvariable commented Mar 8, 2019

chuckha commented Mar 8, 2019

chuckha commented Mar 8, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ashish-amarnath Mar 8, 2019 • edited Loading

Choose a reason for hiding this comment

justaugustus left a comment

Choose a reason for hiding this comment

detiber commented Mar 8, 2019

chuckha commented Mar 8, 2019

chuckha commented Mar 8, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vincepri left a comment

Choose a reason for hiding this comment

chuckha commented Mar 7, 2019 •

edited

Loading

ashish-amarnath Mar 8, 2019 •

edited

Loading