Unify image file format for version parsing

[hellt]

Currently, the version extraction logic uses the source image file as presented by the vendor with a sed expression parsing the version out of this sometimes long and complex file name.
This makes sane people sad when looking at these seds.

The proposal is to unify the way the source qcow/vmdk image files are provided to the make to reduce the ambiguity and drive the sed complexity to zero.

I propose to use the following file name format:

vendor_nos_[type]_(anyVersionString).qcow2

where:

• vendor is a lowercase vendor name, as used in the makefile's VENDOR variable
• nos is a name of the network os, as used in the makefile's NAME variable
• type is a hardware emulated type. An optional field

The version extraction logic then can be centralized in the core make file and removed from the individual make files of a NOS. The version extraction for most systems can use awk and fetch anything from the last _ char until it reaches .qcow2.

NOSes that can't rely on the centralized version extraction should define the VERSION variable in their own make file. This should avoid version detection logic in the centralized make file as it should check if the VERSION variable is set.

[vista-]

I'm not sure whether the saved development effort would offset the additional "support" effort created when we move to a different naming scheme altogether... not just talking about the initial migration here. Being able to copy the vendor-provided image file as-is and doing make is the lowest amount of effort we can get on the user side, and requiring users to rename to a specific format (and having to read the Makefile to figure out this format) might get annoying.

Instead, I would propose to move to the extended regular expressions supported by sed (sed -re 's/.../.../').
This is a much more friendly format and requires less escaping of control characters:

sed -e 's/PA-\(VNM\|VM\)-KVM-\(.*\).qcow2/\2/' becomes sed -re 's/PA-(VNM|VM)-KVM-(.*).qcow2/\2/'. Much better!

In my opinion, maintaining these is worth the effort we save for users (yes, this means I volunteer to write complex regexes if need be), and the regex needs to be very rarely touched, as vendors rarely change image naming conventions.

Your second point, centralizing the logic in the core Makefile, I fully support, however, I think this can be done while preserving the capability to extract version strings from vendor-provided image names.

Instead of putting the version extraction logic in the per-NOS Makefile, we could just make the Makefile read the image name regex pattern from a file, and substitute that into the sed command, like so:

# image_name.sed
s/PA-(VNM|VM)-(.*)/\2/

# ../Makefile
VERSION_SED_REGEX=$(shell tr -d '\n' < ./image_name.sed)
VERSION=$(shell echo ${IMAGE} | sed -re '${VERSION_SED_REGEX}')

[hellt]

Being able to copy the vendor-provided image file as-is and doing make is the lowest amount of effort we can get on the user side, and requiring users to rename to a specific format (and having to read the Makefile to figure out this format) might get annoying.

they only have to do it once, and no need to read the makefile, the readme is going to provide this information.

The problem is that vendors once in a while come up with different namings, or users get an image from another source that had it renamed. Dealing with this in vrnetlab is a work no one asked for, whereas renaming is an easy operation.

The cost of mv whatever.qcow vendor_nos_version.qcow2 is negligible, if users can't do that, they clearly won't be ready to consume containerlab.

[hellt]

But I am ok giving the sed approach a longer lifetime by switching to extended re engine. I don't think splitting out sed in its own file yields any substantial benefits...

[vista-]

I think if we do a pass on all of the Makefiles to convert them to the extended regex format, we might as well go ahead with changing over to the proposed standardized naming convention.

@kaelemc any thoughts?

[hellt]

In support of a unified naming scheme:

1. no more dealing with sed. There is a single expressions for almost all systems
2. disk image name matches the container name that the make produces (same vendor_nos:version pattern)
3. users get to choose how they want the version to appear. E.g. instead of some original version like 16.03.A-rc1.beta3 they may set it to 16.03-b3 if they want to

[kaelemc]

I agree with @vista-. I think making the barrier to entry as low as possible is the best way for beginners, while renaming is very self explanatory (to us) it still adds one extra step for someone new. I think most vendor images are static enough.

On the flip side unified naming is much nicer as you can now deduce the kind in clab from the image name itself.

I also agree with @hellt. If you can't figure this out from the README, then maybe you'll need a bit of support regardless.

I think we can solve 90% of the pain points with the naming by having a better log message for the naming convention if the extraction fails. Maybe in red text or something so it stands out to the user.

and yeah, I think if we are going to have to change all the Makefiles anyway, it's better to just go ahead with the unified naming and solve the pain points earlier rather than later.

This is a perfect opportunity to also go ahead and group images under vendor directories, it sort of compliments the unified naming in structure as well.

For example there is currently a cisco dir with only iol inside. I will move all other Cisco images under the cisco dir.