Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support using regex capture groups to clean up the semver of a tag #407

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

djeebus
Copy link

@djeebus djeebus commented Mar 23, 2022

This allows argocd-image-updater to support finding the latest tag even when they don't really follow semver exactly. Some good examples of this:

  • plexinc/pms-docker example: 1.25.7.5604-980a13e02 (semver requires the final character to be a +, not -)
  • fireflyiii/core example: version-5.6.16 (leading chars should be v, not version-)

This PR would let us set the regexp to be (in pms-docker's case) ^(\d+\.\d+\.\d+)\.\d+-.*$', or just (\d+\.\d+\.\d+). The code checks for groups, and if none are found, assumes that the full tag is the semver (same as current behavior). If a capture group is found, however, then it parses the capture group as a semver, rather than the full thing.

Please let me know if there's anything I can do to help this along, or if I'm out-of-scope for this project, and I'll figure something out. This would help me keep a lot of my homelab images up to date though. Thanks!!

@CLAassistant
Copy link

CLAassistant commented Mar 23, 2022

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@jannfis jannfis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @djeebus !

This indeed looks useful. However, I'm not so sure whether it's a good idea to reuse allowed-tags for transforming the tag name to a version though, even if it may seem reasonable and elegant currently. But it would not allow us to use multiple regexps in the future for allow-tags, or introduce a simpler matcher (e.g. a glob) to be combined with allowed-tags in a semver update strategy.

I think we may want to introduce a new annotation, e.g. semver-transformer that will be used for this transformation.

I'm happy to discuss the pros and cons of both before you make a change, tho.

TagDate *time.Time
TagDigest string
TagName string
TagNamePart string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this field would need some additional comment on what it actually represents, it is not clear to me.

return tag
}

func tryParseSemVer(tag string) *semver.Version {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please have a unit test for this method?

Comment on lines +67 to +69
if len(tag) > 0 && tag[0] != 'v' {
tag = "v" + tag
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is necessary. I think Masterminds/semver treats "x.y.z" and "vx.y.z" the same.

Comment on lines +69 to +72
var (
shouldRemove = false
item = tuple{t, t}
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't use var() block for consistency with rest of the code. I'd suggest using := operator.

}

// MatchFuncRegexp matches the tagName against regexp pattern and returns the result
func MatchFuncRegexp(tagName string, args interface{}) bool {
func MatchFuncRegexp(tagName string, args interface{}) (string, bool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The methods' documentary comment should be updated to reflect the new behavior.

func MatchFuncNone(tagName string, args interface{}) bool {
return false
func MatchFuncNone(tagName string, args interface{}) (string, bool) {
return tagName, false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should return "" instead of tagName, for consistency with MatchFuncRegexp

@@ -88,7 +110,7 @@ func (endpoint *RegistryEndpoint) GetTags(img *image.ContainerImage, regClient R
} else if endpoint.TagListSort == TagListSortLatestLast {
ts = i
}
imgTag := tag.NewImageTag(tagStr, time.Unix(int64(ts), 0), "")
imgTag := tag.NewImageTagWithTagPart(tagStr.original, tagStr.found, time.Unix(int64(ts), 0), "")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I think tagStr should be renamed to tagInfo or something like that in the loop to better reflect the fact that it's not a string anymore. Using tagStr was a bad decision in the first place, but now it just becomes confusing :)

return NewImageTagWithTagPart(tagName, tagName, tagDate, tagDigest)
}

// NewImageTagWithTagPart initializes an ImageTag object and returns it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please be more specific what the difference to NewImageTag is. Also I was wondering if this should be rather something more along the lines of the following, instead of a new initializer:

func (t *ImageTag) WithTagNamePart(tagNamePart string) *ImageTag {
  t.TagNamePart = tagNamePart
  return t
}

tag := NewImageTag(...).WithTagNamePart(...)

WDYT?

@djeebus
Copy link
Author

djeebus commented Mar 31, 2022

@jannfis thanks for the review!

I'm certainly not opposed to creating a new tag. It would increase the scope a little bit, but if it's better for you guys, I can swap to that when I get a chance.

Just so we're on the same page, something like the following would match version-1.2.3, and not version-1.3.0, and neither require the other to exist in order to do their part, right?

argocd-image-updater.argoproj.io/fireflyiii.allow-tags="regexp:version-1\.2\..*"
argocd-image-updater.argoproj.io/fireflyiii.semver-transformer="version-(\d+\.\d+\.\d+)"

Edit: maybe w/ the regexp: prefix on semver-transformer as well?

@jannfis
Copy link
Contributor

jannfis commented Apr 4, 2022

The allow-tags is usually not required in a semver context, because the allowed versions are specified using semver constraint in the image-list annotation. In the semver strategy, we ignore any tag that doesn't parse into a semver anyway.

For example, if you want to stay within 1.2 minor version, you'd configure

argocd-image-updater.argoproj.io/image-list: fireflyiii=some/where/fireflyiii:1.2.x

I would suggest the following for this change:

  • Early in the tag processing, apply any transformation from the transformer annotation
  • Use existing semver constraint checking against the extracted tag from the transformer if applicable, otherwise use the original tag value

@djeebus
Copy link
Author

djeebus commented Apr 6, 2022

@jannfis sorry, that was a bad example, just wanted to make sure the names made sense. I've submitted a new PR (#413), as the new approach was different enough from this that I wanted to keep the diff clean. Feel free to close this one!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants