Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: Use source root for repos if set #5284

Merged
merged 2 commits into from
Feb 10, 2025

Conversation

cgwalters
Copy link
Member

@cgwalters cgwalters commented Feb 7, 2025

core: Use source root for repos and dnf vars

The goal here is to have the new (more container-native) flow
for experimental compose rootfs to always operate on a "repos container"
that we've mounted.

I noticed that rpmostree_find_and_download_packages() has
similar behavior, but that's apparently only used in override
flows?

The only ugly workaround we need now is for the rpm-gpg stuff.

Signed-off-by: Colin Walters [email protected]


compose-rootfs: Add --source-root-rw

This is a hacky workaround for two distinct bugs:

  • Needing /usr/share/rpm (because that's what rpm-ostree wants)
    in the repos container
  • gpgkey= not working with --source-root (see below)

Closes: #5285

It's called --source-root-rw because the way we expect this
to work is in a Dockerfile, the rw flag will be set in RUN --mount=type=bind,rw
to make a temporary overlayfs, so we can go ahead and mutate that
root with our workarounds.

Signed-off-by: Colin Walters [email protected]


@cgwalters cgwalters force-pushed the source-root-releasever branch from 8f82c3b to 242bef3 Compare February 7, 2025 20:14
@cgwalters
Copy link
Member Author

Cool, this makes things way less hacky for the cross build scenario!

@cgwalters cgwalters force-pushed the source-root-releasever branch from 242bef3 to 9209ef9 Compare February 7, 2025 20:18
@cgwalters
Copy link
Member Author

OK I updated this with more crimes workarounds and we now sweep a whole lot ugliness under the rug so that --source-root-rw=/repos Just Works.

@cgwalters
Copy link
Member Author

cgwalters commented Feb 7, 2025

error: Installing packages: Unknown rpm-md repository: fedora

OK right of course this breaks the other compose paths, but hooray for CI. Digging...

EDIT: I think I fixed that

@cgwalters cgwalters force-pushed the source-root-releasever branch from 0a99369 to d70698d Compare February 7, 2025 21:53
@cgwalters
Copy link
Member Author

cgwalters commented Feb 7, 2025

Feb 07 22:20:07 qemu0 kola-runext-upgrades[1796]: Checking out tree 60da9bb...done
Feb 07 22:20:07 qemu0 kola-runext-upgrades[1796]: error: No enabled repositories 

Okay...now legacy treecompose works, but we somehow broke client side upgrades? Fun.

EDIT: I think I figured that out

@cgwalters cgwalters force-pushed the source-root-releasever branch 2 times, most recently from 9960da1 to 557734d Compare February 8, 2025 01:39
src/app/rpmostree-compose-builtin-tree.cxx Show resolved Hide resolved
src/app/rpmostree-compose-builtin-tree.cxx Show resolved Hide resolved
# In theory this can skew from the builder
FROM quay.io/fedora/fedora:41 as repos
# Demonstrate skew from the builder
FROM quay.io/centos/centos:stream10 as repos

# You must run this build with `-v /path/to/rpm-ostree:/run/build/rpm-ostree:ro`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, how about instead we mount an install tree and then rsync that in the container env? Similar to #5286.

Copy link
Member Author

@cgwalters cgwalters Feb 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the CI sprawl is a huge pain. Lately what I've been thinking a lot about is getting really strict and saying: all build stuff is always in a container.

OK to just leave this bit for now and circle back to the larger CI problem...well, not sure when 😢 ?

rust/src/compose.rs Show resolved Hide resolved
rust/src/compose.rs Show resolved Hide resolved
The goal here is to have the new (more container-native) flow
for `experimental compose rootfs` to always operate on a "repos container"
that we've mounted.

I noticed that `rpmostree_find_and_download_packages()` has
similar behavior, but that's apparently only used in override
flows?

The only ugly workaround we need now is for the rpm-gpg stuff.

Signed-off-by: Colin Walters <[email protected]>
This is a hacky workaround for two distinct bugs:

- Needing /usr/share/rpm (because that's what rpm-ostree wants)
  in the repos container
- gpgkey= not working with --source-root (see below)

Closes: coreos#5285

It's called `--source-root-rw` because the way we expect this
to work is in a Dockerfile, the `rw` flag will be set in `RUN --mount=type=bind,rw`
to make a temporary overlayfs, so we can go ahead and mutate that
root with our workarounds.

Signed-off-by: Colin Walters <[email protected]>
@cgwalters cgwalters force-pushed the source-root-releasever branch from 557734d to 1c6fae8 Compare February 8, 2025 18:24
@cgwalters cgwalters enabled auto-merge February 9, 2025 13:12
@cgwalters cgwalters mentioned this pull request Feb 10, 2025
@cgwalters cgwalters merged commit a9e49f5 into coreos:main Feb 10, 2025
18 checks passed
@cgwalters
Copy link
Member Author

For the record, this one has some nontrivial regression risk. I was surprised that I initially broke package layering. Fortunately, our CI coverage is pretty good around this area.

But, something to watch out for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

--source-root doesn't work with gpgkeypath in repo files
3 participants