extend-payload-to-esp: extends any src_input_dir to esp #935

say-paul · 2025-05-27T11:34:22Z

should fix: #651
fixes : #766
To support raspberry pi 4, which works well for rpm-ostree, but in bootc.
This PR add command extend-payload-to-esp which will enable bootup install to move the files to esp.
usage:

RUN dnf install -y uboot-images-armv8

# rpm -ql uboot-images-armv8 | grep rpi
# /usr/share/uboot/rpi_arm64
# /usr/share/uboot/rpi_arm64/u-boot.bin

RUN bootupctl backend extend-payload-to-esp /usr/share/uboot/rpi_arm64

what extend-payload-to-esp does:

stages the files to /usr/lib/efi/firmware/{name}/{version}/EFI/
stores rpm details and timestamp in /usr/lib/efi/firmware/{name}/{version}/EFI.json

bash-5.2# tree /usr/lib/efi/firmware/uboot/
/usr/lib/efi/firmware/uboot/
`-- 1:2025.04-1
    |-- EFI
    |   `-- u-boot.bin
    `-- EFI.json

bootupd install and bootupd update is updated to read from the above path and install to esp.
use the metadata(EFI.json) to update the bootupd-state

rpi4 -usecase: fedora-iot/iot-distro#82 (comment)

say-paul · 2025-05-27T11:38:00Z

I am still trying to build an image with this feature , but with no luck,
My Containerfile looks like this:

FROM quay.io/fedora/fedora-iot:42-x86_64
RUN dnf install -y make cargo rust glib2-devel openssl-devel ostree-devel git
    
RUN git clone -b firmware-copy https://github.com/say-paul/bootupd.git && \
    mkdir -p /boot/efi && \
    touch /boot/efi/dummy.{dtb,dat,efi,elf,bin}

WORKDIR bootupd
RUN export CARGO_HOME=/tmp/.cargo && \
    mkdir -p $CARGO_HOME && \
    cargo build --release
RUN make install

After building the image with bib, I don't see the files in /boot/efi.

cgwalters

Thanks for working on this!

cgwalters · 2025-05-28T12:51:55Z

src/efi.rs

+
+    /// Copy firmware files from /boot/efi to target directory
+    fn copy_firmware_blobs(&self, src_root: &openat::Dir, dest_root: &str) -> Result<()> {
+        let src_efi = "boot/efi";


A core design principle that bootupd tries to maintain is that the operating system payload is underneath /usr as much as possible.

Today in bootc (and prior ostree) we encourage /boot to be empty in the container image. bootupd takes care at build time of moving the payloads to /usr.

It also has infrastructure to keep track of the versions of things it installs so it knows when to update.

So the problem domain is just more complex than this.

There's more discussion in #766 (let's make that the canonical issue).

Ran: #936 on my current implementation with an additional step of creating the files in /boot:

# Build from the current git into a c9s-bootc container image. # Use e.g. --build-arg=base=quay.io/fedora/fedora-bootc:41 to target # Fedora or another base image instead. # ARG base=quay.io/centos-bootc/centos-bootc:stream9 FROM $base as build # This installs our package dependencies, and we want to cache it independently of the rest. # Basically we don't want changing a .rs file to blow out the cache of packages. RUN <<EORUN set -xeuo pipefail dnf -y install cargo git openssl-devel EORUN RUN mkdir -p /boot/firmware/efi && \ touch /boot/firmware/efi/dummy.{dtb,dat,efi,elf,bin} # Now copy the source COPY . /build WORKDIR /build # See https://www.reddit.com/r/rust/comments/126xeyx/exploring_the_problem_of_faster_cargo_docker/ # We aren't using the full recommendations there, just the simple bits. RUN --mount=type=cache,target=/build/target --mount=type=cache,target=/var/roothome \ make && make install-all DESTDIR=/out # FROM $base # # Clean out the default to ensure we're using our updated content # RUN rpm -e bootupd # COPY --from=build /out/ / # # Sanity check this too # RUN bootc container lint --fatal-warnings

From #755

Content shipped in an RPM with files in /boot that are present in the initial base image build where bootupctl backend generate-update-payload is invoked

The build container has the dummy files:

bash-5.2# ls /boot/firmware/efi/ dummy.bin dummy.dat dummy.dtb dummy.efi dummy.elf

but the final image that is built using the COPY --from=build /out/ / doesn't have the files, 😕

I think this function can be modified into a generic one to copy files, but I am not sure how to trigger it from the container build process

but the final image that is built using the COPY --from=build /out/ / doesn't have the files, 😕

The build image is only intended to hold the result of building the bootupd binary itself. If you want to modify the final image for other reasons (e.g. injecting kernel arguments, adding debug tools or other configuration, and as in this case inject arbitrary other content) then you should put it on the last stage.

Or better even - don't modify the dockerfile, make a new one and have it be FROM localhost/bootupd.

cgwalters · 2025-05-28T13:23:10Z

My Containerfile looks like this:

Can you try out #936 ? It has several advantages, but the biggest will be that you can just podman build it and it will do incremental builds from what's in local git, instead of hardcoding your repo.

cgwalters · 2025-06-05T15:02:50Z

Thank you for working on this. I propose that we continue design discussion (e.g. what the interface should look like) in #766 (comment) and I'll comment on the PR for code review.

But a specific downside of what you're doing here is that it would require all tools calling bootupd to be changed (particularly bootc) and to have specific knowledge of what should be installed.

Whereas the goal of bootupd is to abstract that from callers (like bootc) - the person building the container extends the payload, and bootc shouldn't need to know it's copying more files to the ESP.

Also, crucially, my proposal there would also transparently handle updates, which is an important goal.

say-paul · 2025-06-20T12:00:17Z

It should now stage the updates for any user input path,
I think we have to additionally chain this in efi::{install,run_update}
Its very similar to generate_update_metadata but with an user_input.

Addressed

travier · 2025-06-20T13:05:04Z

A lot of questions are unanswered here:

Is those firmware files versioned?
Is it always installed under the same name?
Is it tied to a kernel version?
What happens when we rollback?

Overall, I would suggest we work toward #926 which should get us to #766.

say-paul · 2025-06-20T14:14:13Z

The goal has changed with iterations of the idea and other effort on: #938 , So I updated the PR's main goal on my first comment

Is those firmware files versioned?

Yes, i can keep checks (rpm -qa) if we want to harden it.

Is it always installed under the same name?

Yes

Is it tied to a kernel version?

Those are firmware files and uboot binary.

What happens when we rollback?
We rollback the firmwares too, it will be tracked through bootup-state.json

Overall, I would suggest we work toward #926 which should get us to #766.

Here's how it plays in with #926 : bootupctl backend extend-payload-to-esp /usr/lib/uboot/rpi4 can put the files in the necessary format as laid out by #926 so that bootupd install and update can read from it.~~(some modifications required in the PR)~~

say-paul · 2025-06-20T16:25:19Z

This should now be inline with to what is required by #926

say-paul · 2025-06-20T16:48:48Z

We need to teach generate-update-metadata in #938 to read from /firmware/
and stage it for updates:
bootupd extend-payload-to-esp /boot/efi to extend firmware to /usr usecase - > files installed in /bbot/efi by bcm2711-firmware.rpm
and also:
bootupd extend-payload-to-esp /usr/share/uboot/rpi_arm64 -> use-case: copy /usr/share/uboot/rpi_arm64/rpi-u-boot.bin installed by uboot-images-armv8.rpm

say-paul · 2025-07-21T05:13:02Z

Curious about a mode dealing with a rpi config.txt. There a way to have it land one and then not update it if its there already or alternately only if unchanged?

@kfox1111 I can possibly modify upgrade to do things like not to remove the old file if no new file.

HuijingHei · 2025-07-21T07:21:47Z

So does this mean that you only want to extend single file /usr/share/uboot/rpi_arm64/u-boot.bin, not files under /boot/efi like bcm2711-firmware, is this right?

@HuijingHei On high level i want to copy content of a dir to /boot/efi that includes both containers: /boot/efi and /usr/share/uboot/rpi_arm64/ to esp of the image.

this is the complete usecase : fedora-iot/iot-distro#82 (comment)

Agree with Colin and Timothee that there are partly duplication with #926, e.g. stages the input files to /usr/lib/efi/firmware/{name}/{version}/EFI/, maybe we should make them independent functions and let both can reuse it. Would you like to change this (or I will do the change)? And it would be better to add test for comment in the PR. WDYT?

say-paul · 2025-07-22T07:03:09Z

src/efi.rs

    }

+    fn extend_payload(&self, sysroot_path: &str, src_input: &str) -> Result<Option<bool>> {
+        let dest_efidir_base = Path::new(sysroot_path).join("usr/lib/efi").join("firmware");


@HuijingHei do you suggest overloading the function to put files in usr/lib/efi/(firmware|shim|grub)

bam80 · 2025-07-30T10:41:01Z

Anything left?

say-paul · 2025-07-30T13:53:25Z

Nope, just rebased it.

extend_payload_to_esp <path> will move the all contents of the <path> to /usr This features gives user ability to stage payloads so that install and update can install the contents to esp.It will move all the files and sub dirs under the path provide by user to /usr/lib/efi/firmware/<name>/<version>/EFI/ Incase of an exiting path extend-payload will only keep the latest version of the payload. example usage: `bootupctl backend extend_payload_to_esp /usr/lib/mydata` will move the content of the dir to `/usr/lib/efi/firmware/<pkg-name>/<version>/EFI/` stores metadata at `/usr/lib/efi/firmware/<pkg-name>/<version>/EFI.json`. Added test_extend_payload_to_esp unit test.

Added additional flow in efi::install to read from the path created by extend_payload_to_esp command and install to /boot/efi

efi::update is updated to read from path created by extend_payload_to_esp and update the firmware.

cgwalters

Sorry for the delay in review and thanks for working on this.

I have some initial comments.

cgwalters · 2025-09-02T12:46:32Z

.github/workflows/ci.yml

+        run: |
+          set -xeuo pipefail
+          sudo truncate -s 5G myimage-extend.raw
+          sudo podman run --rm --privileged -v .:/target --pid=host --security-opt label=disable \


This is not at all a new problem, and I won't say we should block on this but...

This is greatly exacerbating the problem that it's just a pain to run these tests outside of GHA (short of https://github.com/nektos/act etc.)

There's an effort to adapt our tests to TMT, but that gets into larger CI issues.

Anyways again not a blocker, but just noting.

Actually thinking about this a bit more, why can't this test work how we'd expect things to work in production?

Basically if we change this to be a podman build that installs the files instead that should all Just Work, right?

cgwalters · 2025-09-02T12:47:19Z

.github/workflows/ci.yml

+              
+              # Create a fake RPM database for testing
+              mkdir -p /usr/lib/sysimage/rpm
+              echo "fake rpm database" > /usr/lib/sysimage/rpm/Packages


Hmmm....this seems highly likely to confuse other code in bootupd at some point.

As is right now, I am pretty sure it will result in bootupd not installing shim and grub, right?

cgwalters · 2025-09-02T15:23:43Z

src/cli/bootupctl.rs

    Generate(super::bootupd::GenerateOpts),
    #[clap(name = "install", hide = true)]
    Install(super::bootupd::InstallOpts),
+    #[clap(name = "extend-payload-to-esp", hide = true)]


I don't think we should hide = true this - it's a user visible feature.

cgwalters · 2025-09-02T15:26:09Z

src/efi.rs

+                panic!("extend_payload returned None - expected success");
+            }
+            Err(e) => {
+                panic!("extend_payload failed when it should have succeeded: {}", e);


It'd avoid all this nesting to just use .unwrap() or .expect() on the result which seems preferable to me.
And we may as well do the same for the Option

cgwalters · 2025-09-02T15:27:16Z

src/efi.rs

+
+        let original_path = std::env::var("PATH").unwrap_or_default();
+        let new_path = format!("{}:{}", mock_rpm_dir.path().display(), original_path);
+        std::env::set_var("PATH", &new_path);


This is fully unsafe for good reason in Rust 2024: https://doc.rust-lang.org/edition-guide/rust-2024/newly-unsafe-functions.html#stdenvset_var-remove_var

It's more painful but basically if we wanted to mock RPM we'd have to explicitly pass it as a parameter.

cgwalters · 2025-09-02T15:28:14Z

src/efi.rs

+                anyhow!("RPM query returned an empty or malformed version string")
+            })?;
+
+        let parts: Vec<&str> = version_string_part.split('-').collect();


Hmm @HuijingHei is reworking the version parsing in #977 etc.

cgwalters · 2025-09-02T15:28:56Z

src/efi.rs

+                format!(
+                    "{}-{}",
+                    parts[parts.len() - 2],
+                    parts[parts.len() - 1]


Try using e.g.

match parts.as_slice() { [name, version, release] => { ... } _ => anyhow::bail!()

cgwalters · 2025-09-02T15:29:34Z

src/efi.rs

            filetree::apply_diff(&updated, &destdir, &diff, None)
                .context("applying filesystem changes")?;
+            // update firmware
+            let firmware_base_dir_path = Path::new("usr/lib/efi/firmware");


Where does this path come from? Is it something you made up here?

cgwalters · 2025-09-02T15:30:10Z

src/efi.rs

+        let mut found_firmware = BTreeMap::new();
+        // Scan and install supplemental firmware
+        let firmware_base_dir_path = Path::new("usr/lib/efi/firmware");
+        if src_root.exists(firmware_base_dir_path)? {


The code past this is getting really deeply nested, can we factor out the inner part here into a helper function?

travier · 2025-09-09T16:15:23Z

We had a chat a while back and this should now be possible with what we have in #938.

Creates a test variant to evaluate bootupd PR #935's extend-payload-to-esp command as an alternative to the bootupctl shim approach. Key changes: - New pi-pr935 variant using Fedora 43 (Rawhide) - Builds bootupd from say-paul/bootupd firmware-copy branch - Uses bootupctl backend extend-payload-to-esp to stage firmware - No bootupctl shim required - Added task definitions for building container and disk images This will help test and provide feedback on coreos/bootupd#935 and #766 to move the ARM firmware support ecosystem forward. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

say-paul changed the title ~~Copy firmware blobs to boot/efi to ensure that the aarch64~~ Copy firmware blobs to boot/efi of the disk image May 28, 2025

say-paul changed the title ~~Copy firmware blobs to boot/efi of the disk image~~ Copy firmware blobs to /boot/efi of the disk image May 28, 2025

cgwalters previously requested changes May 28, 2025

View reviewed changes

openshift-merge-robot added the needs-rebase label Jun 3, 2025

say-paul force-pushed the firmware-copy branch from b79a009 to 8df5030 Compare June 5, 2025 14:29

openshift-merge-robot removed the needs-rebase label Jun 5, 2025

say-paul changed the title ~~Copy firmware blobs to /boot/efi of the disk image~~ Copy files using sub command install-to-filesystem Jun 5, 2025

say-paul force-pushed the firmware-copy branch from 3ec5405 to c26320f Compare June 5, 2025 14:32

say-paul requested a review from cgwalters June 5, 2025 14:33

say-paul force-pushed the firmware-copy branch from c26320f to f40d439 Compare June 5, 2025 14:36

say-paul mentioned this pull request Jun 13, 2025

Bootc image does not boot on rpi fedora-iot/iot-distro#82

Open

say-paul force-pushed the firmware-copy branch from f40d439 to e1f8322 Compare June 18, 2025 23:59

say-paul changed the title ~~Copy files using sub command install-to-filesystem~~ extend-payload-to-esp: extends any src_input_dir to esp Jun 19, 2025

say-paul force-pushed the firmware-copy branch 3 times, most recently from 634cf2f to 9bac770 Compare June 19, 2025 12:09

openshift-merge-robot added the needs-rebase label Jun 20, 2025

say-paul force-pushed the firmware-copy branch from d8e22ee to 3256867 Compare June 20, 2025 16:34

openshift-merge-robot removed the needs-rebase label Jun 20, 2025

say-paul force-pushed the firmware-copy branch from 3256867 to e3276df Compare June 20, 2025 16:39

say-paul force-pushed the firmware-copy branch from e3276df to b0a2a53 Compare June 20, 2025 16:52

say-paul commented Jul 22, 2025

View reviewed changes

say-paul force-pushed the firmware-copy branch from c0bf635 to 15d36a3 Compare July 25, 2025 13:15

openshift-merge-robot added the needs-rebase label Jul 25, 2025

say-paul force-pushed the firmware-copy branch 3 times, most recently from 3781502 to 0412f72 Compare July 25, 2025 14:31

say-paul force-pushed the firmware-copy branch from 0412f72 to a9e0187 Compare July 30, 2025 13:42

openshift-merge-robot removed the needs-rebase label Jul 30, 2025

say-paul force-pushed the firmware-copy branch 2 times, most recently from b97ea7f to cf10fc1 Compare July 30, 2025 13:52

say-paul force-pushed the firmware-copy branch from cf10fc1 to 9bea9e4 Compare July 30, 2025 14:03

say-paul added 3 commits July 31, 2025 17:30

update(efi::install)install from extend-payload-path

292237b

Added additional flow in efi::install to read from the path created by extend_payload_to_esp command and install to /boot/efi

update(efi::update)update extended payload path

b40a1f2

efi::update is updated to read from path created by extend_payload_to_esp and update the firmware.

say-paul force-pushed the firmware-copy branch 6 times, most recently from 78ed7eb to fc199aa Compare August 4, 2025 14:44

ci: added tests for install and update for extended payload

9afa751

say-paul force-pushed the firmware-copy branch from fc199aa to 9afa751 Compare August 4, 2025 15:03

verbose

fdf4b31

cgwalters reviewed Sep 2, 2025

View reviewed changes

extend-payload-to-esp: extends any src_input_dir to esp #935

Are you sure you want to change the base?

extend-payload-to-esp: extends any src_input_dir to esp #935

Conversation

say-paul commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

say-paul commented May 27, 2025

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

say-paul May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cgwalters commented May 28, 2025

Uh oh!

cgwalters commented Jun 5, 2025

Uh oh!

say-paul commented Jun 20, 2025

Uh oh!

travier commented Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

say-paul commented Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

say-paul commented Jun 20, 2025

Uh oh!

say-paul commented Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

say-paul commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HuijingHei commented Jul 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bam80 commented Jul 30, 2025

Uh oh!

say-paul commented Jul 30, 2025

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

travier commented Sep 9, 2025

Uh oh!

Reviewers

Assignees

Labels

say-paul commented May 27, 2025 •

edited

Loading

say-paul May 30, 2025 •

edited

Loading

travier commented Jun 20, 2025 •

edited

Loading

say-paul commented Jun 20, 2025 •

edited

Loading

say-paul commented Jun 20, 2025 •

edited

Loading

say-paul commented Jul 21, 2025 •

edited

Loading