Publish tags that can be consumed by dependabot

[btlogy]

As today, merging pull request in main branch triggers the publication of the images in Docker Hub based on the short revision hash (e.g: 220a007 -> 220a007, then d38de20 -> d38de20).

However, our downstream project relies on tags which are created manually (e.g.: 220a007 = 20230214-1).
I believe that once created, pushing the tag triggers the delivery jobs once more to deliver the image in Docker Hub (e.g.: 20230214-1).

Thus we can end up in the current situation where we had a few image updates published on Docker Hub, but w/o the proper tag to be consumed by downstream (e.g.: d006725 != 20230215-1)!
Maybe there is a good reason for this. But I don't see any.

In addition, it might be required for those tags to be semantically formatted (semver) in order to be consumed downstream by dependabot.

[btlogy]

@wuan, does this make sense to you?

[wuan]

@wuan, does this make sense to you?

We selected the <date>-<increment> format originally as those releases combine versions from the upstream projects and semantic versioning will be not that straightforward. Given the low rate of releases we do here I'm not sure if we have an advantage automating the release process.

[btlogy]

Thanks you for feedback @wuan

We selected the - format originally as those releases combine versions from the upstream projects and semantic versioning will be not that straightforward.

It makes sense indeed. Maybe Dependabot will not mind if we keep using this format. I'll try.

Given the low rate of releases we do here I'm not sure if we have an advantage automating the release process.

That's a bit the point here, there was a few updates we did not yet propagated downstream:

• update dependencies #18
• Switch to Python slim-bullseye image w/o PyPy #21
• Switch base images to slim-bullseye #28

And we can expect more (also by design since #29):

• Update magic-wormhole image with latest upstream code #23
• Docker containers sould run in rootless mode #34
• Bump pypy from 3.9-7.3.9-slim-bullseye to 3.9-7.3.11-slim-bullseye in /relay #31
• Bump pypy from 3.9-7.3.9-slim-bullseye to 3.9-7.3.11-slim-bullseye in /mailbox #32

I don't mind keep it manual, but automation with something like gtihub-tag action might save us some time in the mid term already.

Said otherwise: do you (or anyone else) see any strong reason I would have missed to keep it manual?

[btlogy]

I've made a test elsewhere, and dependabot fails to find updates based on hash for sure:

updater | 2023/05/12 14:37:37 INFO <job_662034499> Checking if leastauthority/magic-wormhole e3f0480 needs updating
updater | 2023/05/12 14:37:37 INFO <job_662034499> Latest version is e3f0480

Manual or not, I propose to use a semver like <yy>.<m>.<inc> rather than <date>-<increment> (e.g.: 23.5.0).

[wuan]

That should be fine.

[btlogy]

At the end, my tests have shown that Dependabot is dealing correctly with <date>-<inc>:

Bump leastauthority/magic-wormhole-mailbox from 20221108-1 to 20230214-1 in /ansible/docker/mailbox

Thus, the only reason left to use semver is the automation because I suspect it will likely be much easier to automatically bump 23.5.0 than 20230512-1.

In the meantime, I've found an other annoying problem to consume thos tags downstream: no support for docker-compose (yet)!

[btlogy]

At the end, my tests have shown that Dependabot is dealing correctly with <date>-<inc>

I think I've missed something there: I bet it will not work for 20230512-1 to 20230512-2.
I suspect the <inc> to not be interpreted as a part of the version, but rather as a flavor (e.g.: debian vs alpine).

Anyway, semver is likely a better choice... If we do something with Dependabot...

[meejah]

Just FYI, but that's usually called "CalVer" (i.e versions like 23.5.0 -- as magic-folder uses too) see https://calver.org/

[btlogy]

I've remove the automation part in the title of the issue as it was secondary.

The most important part is to produce tags that can be consume by other processes downstream, starting with dependabot.

Though, once we manually publish one of those new tag (CalVer likely), we will have to pay attention to not brake those downstream process. This is where the automation could help I guess...

I believe the first new tag will have to be manual anyway.
I'm considering doing it with the following tags:

• 23.2.0 : as an alias of the existing 20230214-1
• 23.5.0 : as a new release with (some) of the latest optimizations

This should put us on the right path and allow us to verify how to automatically bump those version downstream.

Though I'm kind of sad we do not see anything about the upstream version here.
This would kind of look better:

• leastauthority/magic-wormhole:0.12.0-23.5.0-slim-bullseye
• leastauthority/magic-wormhole-mailbox:0.4.1-23.5.0-slim-bullseye
• leastauthority/magic-wormhole-relay:0.2.1-23.5.0-slim-bullseye

Although hard to test if this will work as I expect because I'm not allowed to delete a wrong tag in Docker Hub (if that's even possible)!

Let's make some baby steps I guess.

[btlogy]

I've just pushed 23.2.0 (logs), which did overwrite the latest (as expected from reading the workflow).
But then I've just pushed 23.5.0 (logs), and the latest went back the latest images.

Note for later: we could save some build time if the tag were automatically released (not re-building the images twice).

[btlogy]

Also, as for now the digests differs between the images published from main branch and from the tag.
Likely because the creation time differs too.
So I guess we are duplicating images per release...

I believe that automating the release when merging PR into the main branch is the only way we could avoid duplication.

[btlogy]

Anyway: I'm now testing if those tags can be consume downstream by dependabot as expected.

[btlogy]

Looks good:

updater | 2023/05/16 13:53:12 INFO <job_663549791> Latest version is 23.5.0
updater | 2023/05/16 13:53:12 INFO <job_663549791> Requirements to unlock own
updater | 2023/05/16 13:53:12 INFO <job_663549791> Requirements update strategy 
updater | 2023/05/16 13:53:12 INFO <job_663549791> Updating leastauthority/magic-wormhole-mailbox from 23.2.0 to 23.5.0
updater | 2023/05/16 13:53:12 INFO <job_663549791> Submitting leastauthority/magic-wormhole-mailbox pull request for creation
  proxy | 2023/05/16 13:53:13 [074] GET https://api.github.com:443/repos/la-test/sbx-actions/commits?per_page=100
  proxy | 2023/05/16 13:53:13 [074] * authenticating github api request with token for api.github.com
  proxy | 2023/05/16 13:53:13 [074] 200 https://api.github.com:443/repos/la-test/sbx-actions/commits?per_page=100
updater | 2023/05/16 13:53:14 INFO <job_663549791> Finished job processing
updater | 2023/05/16 13:53:14 INFO Results:
updater | +---------------------------------------------------------------------------+
updater | |                    Changes to Dependabot Pull Requests                    |
updater | +---------+-----------------------------------------------------------------+
updater | | created | leastauthority/magic-wormhole-mailbox ( from 23.2.0 to 23.5.0 ) |
updater | +---------+-----------------------------------------------------------------+

[btlogy]

I'd better update the documentation!

[btlogy]

I now have access to the Docker account and I've re-published 'real' tags for 23.2.0 = 20230214-1 images (the previous ones were actually broken due to #38.

But this does not seem to prevent Dependabot to propose PR for older version:

Bumps leastauthority/magic-wormhole-mailbox from 23.5.0 to 20230214-1.

Likely because from a SemVer point of view 20230214 > 23.0.5...

[btlogy]

I'd better update the documentation!

I do not see how/what to update after all.

[btlogy]

In order to fix the SemVer issue, I've manually tagged a few older versions and remove those last ones from Docker Hub:

• 221105-1 -> 22.11.0
• 221108-1 -> 22.11.1
• 230214-1 -> 23.2.0

I've also remove any every non-release tag released before #28

This repository should now be ready to be easily bumped from downstream.

[btlogy]

I'd better update the documentation!

I do not see how/what to update after all.

There is already an issue to improve the documentation (via the overview in Dokcer Hub):

• Add documentation to Docker Hub #5