Copr packages not installed in Fedora 41 and Rawhide

[jchecahi]

In the testing-farm runs of fedora-41 and fedora-rawhide, tmt is installing LLVM 18 packages coming from the system repos rather than LLVM 20 packages coming from the copr repo. Examples from snapshot 20240825:

• fedora-rawhide-x86_64
• fedora-41-x86_64

[jchecahi]

@kwk It turns out this issue is because of the testing-farm-tag repo priority, related to #395. In our test plan we decrease the priority of the repo in this line. However, we call dnf config-manager using the old dnf4 interface, which dnf5, default package manager for Fedora 41 and newer, don't understand . Since we're piping the output to a true command, the adjustment fails unadvertently, the priority change is not done, but tmt's prepare stage continues.

            dnf config-manager --save --setopt="testing-farm-tag-repository.priority=999" || true
[ cropped output, not relevant here ]
10:59:00             cmd: /var/ARTIFACTS/work-snapshot-gatingzljcusxv/tests/snapshot-gating/tree/tmt-prepare-wrapper.sh-default-1-default-0
10:59:00         environment
10:59:00             out: Updating and loading repositories:
[ cropped output, not relevant here ]
10:59:05             out: Complete!
10:59:05             err: Unknown argument "--save" for command "config-manager". Add "--help" for more information about the arguments.

I will work on a PR to solve this issue.

[nikic]

I wonder whether https://github.com/fedora-llvm-team/llvm-snapshots/blob/main/tests/snapshot-gating.fmf could inject an additional test that only checks that clang -v (or similar) returns the expected version for snapshots...

[kwk]

I wonder whether https://github.com/fedora-llvm-team/llvm-snapshots/blob/main/tests/snapshot-gating.fmf could inject an additional test that only checks that clang -v (or similar) returns the expected version for snapshots...

It can. And that's what I meant by "trustworthiness" in #666. I plan to inject the Version similar to $COPR_PROJECT.

[kwk]

@jchecahi how about this quick fix to first try the old dnf command, and if it fails, try the new one and if that fails, bail out.

dnf config-manager --save --setopt="testing-farm-tag-repository.priority=999" || dnf config-manager setopt "testing-farm-tag-repository.priority=999" || true

[jchecahi]

I wonder whether https://github.com/fedora-llvm-team/llvm-snapshots/blob/main/tests/snapshot-gating.fmf could inject an additional test that only checks that clang -v (or similar) returns the expected version for snapshots...

Yes that should be doable. However, I 'd go with a full script based an exact match between the installed package, and the one available in the repo. I'll make an attempt and update here what I find.

@jchecahi how about this quick fix to first try the old dnf command, and if it fails, try the new one and if that fails, bail out.

dnf config-manager --save --setopt="testing-farm-tag-repository.priority=999" || dnf config-manager setopt "testing-farm-tag-repository.priority=999" || true

I'd rather avoid the command || true approach to prevent masked failures in the future. IMHO, it's safer to increase the copr repo priority instead, and let tmt fail if the operation is not successful.

[jchecahi]

@nikic @kwk I implemented a script that checks the installed llvm rpm and aborts the run on failure: jchecahi@4705549

Right now it checks only for llvm rpm, but it's only a first iteration. I ran it on testing-farm on a Fedora 41 compose and the result looks promising: https://artifacts.dev.testing-farm.io/e6ea60be-9210-461a-a0a8-024f13900b8f/

[kwk]

@jchecahi okay, here's the most simple version of all that uses only rpm:

$ rpm -qi llvm-libs
Name        : llvm-libs
Version     : 20.0.0~pre20240825.g3ef64f7ab5b865
Release     : 1.fc41
Architecture: x86_64
Install Date: Tue Aug 27 07:44:51 2024
Group       : Unspecified
Size        : 134526998
License     : Apache-2.0 WITH LLVM-exception OR NCSA
Signature   : RSA/SHA256, Mon Aug 26 02:20:45 2024, Key ID d571ca252d9de8df
Source RPM  : llvm-20.0.0~pre20240825.g3ef64f7ab5b865-1.fc41.src.rpm
Build Date  : Sun Aug 25 02:10:23 2024
Build Host  : aws-x86-64-powerful-normal-prod-09057687-20240825-020719
Vendor      : Fedora Copr - group @fedora-llvm-team
URL         : http://llvm.org
Summary     : LLVM shared libraries
Description :
Shared libraries for the LLVM compiler infrastructure.

Please note, that we have the version 20.0.0~pre20240825.g3ef64f7ab5b865. This is very specific down to the git commit. But only if you install a snapshot you'll have a ~pre version. Therefore this should do it:

$ rpm -qi llvm-libs | grep -P "Version[^:]+:.*~pre.*"

If this succeeds you know that you'll have llvm installed from the snapshots copr repo. We don't need to care so much about anything else.

[jchecahi]

@kwk ok, fair enough let's keep it simple. I'd say that the simplest we can go is just printing the NVR and match pre:

# rpm -q llvm-libs | grep '~pre'
llvm-libs-20.0.0~pre20240825.g3ef64f7ab5b865-1.fc42.x86_64

[kwk]

@kwk ok, fair enough let's keep it simple. I'd say that the simplest we can go is just printing the NVR and match pre:

# rpm -q llvm-libs | grep '~pre'
llvm-libs-20.0.0~pre20240825.g3ef64f7ab5b865-1.fc42.x86_64

We should match the version though.