Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make test image caching saner #3534

Merged
merged 3 commits into from
Jan 24, 2025

Conversation

dmnks
Copy link
Contributor

@dmnks dmnks commented Jan 23, 2025

  • Use build-local "tags" instead of tagging all images as "rpm".
  • Fix constant base image rebuilds when switching back and forth between branches

Details in commit messages.

dmnks added 3 commits January 23, 2025 18:20
Don't use the single "rpm" tag, it causes multiple build directories to
step on each others' toes, such as when one of them tries to remove the
previous version of the image on a rebuild and falsely assumes that the
"rpm" tag still points to it.  That results in an error if that image is
in use by some other build directory, e.g. by a "make shell" session.

Instead, just cache the image ID locally in the build directory which
produced it and use that to refer to the image later.  It's like having
a unique tag for each build directory but without polluting the user's
image store with tagged images that are never cleaned up.

Since removing a build directory now means we lose the image ID, delete
the image as soon as we're done using it.  Only do that when running
mktree via cmake (the normal case), though, otherwise keep the image so
that one can still use mktree directly to prototype a test without
having to rebuild each time:

    ./mktree atshell mytest.sh

In that case, the user is expected to clean up the image by themselves
when they're done, of course.

When in CI mode, store the ID in the source tree as we may not have a
build tree.
Sometimes, you may want to use $PODMAN to run your own containers on top
of the image (or even base a custom image on it), in which case tagging
is actually desired, so add a shorthand for that.

Make sure to *not* clean the image if it's tagged, though.  Also update
the README which still talks about the "rpm" tag.
Base images sometimes used to be pruned as part of a rebuild when they
became dangling.  That could happen when the "rpm" image was based on a
different version of the base image.

This was super annoying since you had to wait for the whole package set
to be downloaded and installed again.

Luckily, base images don't change as often so we can perhaps afford to
keep each version around.  Do that by simply adding the Dockerfile hash
to the tag name.
@dmnks dmnks requested a review from a team as a code owner January 23, 2025 17:31
@dmnks dmnks requested review from pmatilai and removed request for a team January 23, 2025 17:31
@pmatilai
Copy link
Member

pmatilai commented Jan 24, 2025

/me pretends to understand and nods agreeably 😅

@pmatilai pmatilai merged commit 766192a into rpm-software-management:master Jan 24, 2025
1 check passed
@dmnks dmnks added the test Testsuite-related label Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Testsuite-related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants