blueprints: (optionally) include blueprint repos in 2nd depsolve

[mvo5]

This commit allows to (optionally) include the custom repositories from the blueprints as part of the depsolving when installing the user packages.

I'm a bit unsure about the best way to go about a full end-to-end test, I'm not super familiar with the existing integration test scripts in "images", I was thinking to do the full end-to-end test with image-builder-cli but maybe that is silly, ideas very welcome.

The amount of duplication in between rhel/fedora in the pkg/distro is also something I'm not happy about but it seems that is the current design but I would love to get back to this and see we can do something about it.

Thanks to Ondrej for raising this.

[achilleas-k]

The amount of duplication in between rhel/fedora in the pkg/distro is also something I'm not happy about but it seems that is the current design but I would love to get back to this and see we can do something about it.

Indeed, the ImageType.Manifest() function is probably identical for all implementations of the interface (in each distro). In cases where it's not, it's probably by accident, or because a specific distro doesn't support something (like RHEL 7), but that latter issue could be handled differently.

[achilleas-k]

I'm a bit unsure about the best way to go about a full end-to-end test

Yeah, it's a bit tricky. The obvious thing to do is to add a configuration with a repository in the blueprint to test/configs/, but it would have to be a repository that's not used already. The build tests all use the repositories from test/data/repositories/ and there's no straightforward way to drop one.

We could do something a bit "silly". If you look at the test repository configurations, some of them have these image_type_tags, which limits the use of a repository to just those image types that are listed. This is done so that we don't use, for example, google-cloud-sdk repositories when building an image for any other platform. So one thing we could do is add a build config (blueprint) that adds the google-cloud-sdk repository to a qcow2 build (or any other build, really) and also selects one of the google-cloud-sdk packages for installation. One issue with this though is that the configuration wont be updated automatically when our repository configurations are updated after new snapshots are created.
If we do this for a GA distro version (RHEL 9.5 or older), it should be okay though.

[thozza]

The amount of duplication in between rhel/fedora in the pkg/distro is also something I'm not happy about but it seems that is the current design but I would love to get back to this and see we can do something about it.

Indeed, the ImageType.Manifest() function is probably identical for all implementations of the interface (in each distro). In cases where it's not, it's probably by accident, or because a specific distro doesn't support something (like RHEL 7), but that latter issue could be handled differently.

+1

It used to be much worse - duplication for every RHEL major version 😅

We can unify Fedora and RHEL. The code can be written to support the super-set of all features. It does not matter if there are bits specific to RHEL-7, that Fedora does not use.

[thozza]

I'm a bit unsure about the best way to go about a full end-to-end test

Indeed, it's a bit tricky. The obvious thing to do is to add a configuration with a repository in the blueprint to test/configs/, but it would have to be a repository that's not used already. The build tests all use the repositories from test/data/repositories/ and there's no straightforward way to drop one.

I have two proposals:

1. Snapshotting and using the EPEL repository for this. We never install packages from EPEL.
2. An alternative would be our own COPR repository with a dummy package.

Both are pretty straightforward. EPEL repo would be usable only for a specific version of RHEL major version, so COPR is the most universal approach.

[achilleas-k]

2. An alternative would be our own COPR repository with a dummy package.

I was re-thinking my comment and had a similar thought. A custom repo with 2-3 dummy packages would be useful for all sorts of tests actually. We could use it for quick and dirty depsolve tests for example in all our projects, like in osbuild-composer where we create one locally each time.

EDIT: Elaborating on the "2-3 packages" part: If we make packages with dependencies, it would be a nice, simple depsolver test repo too.