Kiwi seems to generate an incompatible configuration for Grub 2.12 on Ubuntu 23.10

[IsaacJT]

Problem description

Images generated by Kiwi with a distribution using Grub 2.12 do not boot in EFI mode. When the image is booted, it immediately restarts after loading Grub. I'm guessing that this is likely due to the configuration that Kiwi generates not being compatible with the new changes in Grub 2.12.

An example of where this happens is in Ubuntu Mantic 23.10 using QEMU and the OVMF EFI firmware.

The Grub EFI binary is executed, but seems to bail out and then restart:

BdsDxe: loading Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
BdsDxe: starting Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
error: terminal `gfxterm' isn't found.
BdsDxe: loading Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
BdsDxe: starting Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
error: terminal `gfxterm' isn't found.
BdsDxe: loading Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
BdsDxe: starting Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
error: terminal `gfxterm' isn't found.

If you add nvram to the VM to persist the EFI variables, it restarts and loads the EFI setup app (assumedly because Grub exits):

BdsDxe: loading Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x4,0x0)
BdsDxe: starting Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x4,0x0)
error: terminal `gfxterm' isn't found.
error: no suitable video mode found.
error: no video mode activated.
BdsDxe: loading Boot0000 "UiApp" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(462CAA21-7614-4503-836E-8AB6F4662331)
BdsDxe: starting Boot0000 "UiApp" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(462CAA21-7614-4503-836E-8AB6F4662331)

Expected behaviour

The image boots correctly using Grub 2.12.

Steps to reproduce the behaviour

Build the ubuntu-jammy Kiwi description with the following patch applied to change it to be based on Ubuntu 23.10: https://github.com/OSInside/kiwi-descriptions/blob/master/ubuntu/x86_64/ubuntu-jammy/config.xml

--- ubuntu-jammy/config.xml     2023-10-26 11:57:40.819348036 +0200
+++ ubuntu-mantic/config.xml    2023-10-26 12:35:41.611519796 +0200
@@ -1,10 +1,10 @@
 <?xml version="1.0" encoding="utf-8"?>

-<image schemaversion="6.8" name="Ubuntu-22.04_appliance">
+<image schemaversion="6.8" name="Ubuntu-23.10_appliance">
     <description type="system">
         <author>Marcus Schaefer</author>
         <contact>[email protected]</contact>
-        <specification>Image description for Ubuntu 22.04</specification>
+        <specification>Image description for Ubuntu 23.10</specification>
     </description>
     <profiles>
         <profile name="Live" description="Live image" import="true"/>
@@ -42,14 +42,14 @@
     <repository type="apt-deb" repository_gpgcheck="false" package_gpgcheck="false" imageinclude="true">
         <source path="obs://Virtualization:Appliances:Builder/xUbuntu_22.04"/>
     </repository>
-    <repository type="apt-deb" alias="Jammy-security" distribution="jammy-security" components="main multiverse restricted universe" repository_gpgcheck="false">
+    <repository type="apt-deb" alias="Mantic-security" distribution="mantic-security" components="main multiverse restricted universe" repository_gpgcheck="false">
         <source path="http://security.ubuntu.com/ubuntu"/>
     </repository>
-    <repository type="apt-deb" alias="Jammy-updates" distribution="jammy-updates" components="main multiverse restricted universe" repository_gpgcheck="false">
-        <source path="http://us.archive.ubuntu.com/ubuntu/"/>
+    <repository type="apt-deb" alias="Mantic-updates" distribution="mantic-updates" components="main multiverse restricted universe" repository_gpgcheck="false">
+        <source path="http://de.archive.ubuntu.com/ubuntu/"/>
     </repository>
-    <repository type="apt-deb" alias="Jammy" distribution="jammy" components="main multiverse restricted universe" repository_gpgcheck="false">
-        <source path="http://us.archive.ubuntu.com/ubuntu/"/>
+    <repository type="apt-deb" alias="Mantic" distribution="mantic" components="main multiverse restricted universe" repository_gpgcheck="false">
+        <source path="http://de.archive.ubuntu.com/ubuntu/"/>
     </repository>
     <packages type="image">
         <package name="grub2-themes-ubuntu-mate"/>

Build the image using the "Virtual" profile and boot it with QEMU:

cp /usr/share/OVMF/OVMF_VARS.fd /tmp/nvram
qemu-system-x86_64 \
    -m 1G --enable-kvm --smp 2 --cpu host -nographic -M q35 \
    -drive file="/usr/share/OVMF/OVMF_CODE.fd",readonly=on,if=pflash,format=raw,unit=0 \
    -drive file="/tmp/nvram",if=pflash,format=raw,unit=1 \
    -netdev user,id=user0,hostfwd=tcp::10022-:22 -device virtio-net-pci,netdev=user0 \
    -drive file=./Ubuntu-23.10_appliance.x86_64-1.16.4-0.raw,if=virtio
rm /tmp/nvram

OS and Software information

• KIWI version: 9.25.17
• Operating system host version: Ubuntu 23.10
• Operating system target version: Ubuntu 23.10
• Open Build Service version (N/A if not using OBS): N/A
• Koji version (N/A if not using Koji): N/A

[IsaacJT]

Note: I don't think it's related to #2354 as Grub doesn't even get to that step any more before bailing out

[schaefi]

For the virtual profile there is firmware="uefi" set. In this mode kiwi first searches for a pre-built (distribution provided) EFI binary and only if it can't find one it compiles its own EFI image using the grub tooling. My assumption is that in the kiwi build log some grub-mkimage call will be found and that the resulting EFI image is for some reason not a good one. Do you have the build log available somehow and can check that ?

The part that is really strange is the reboot that happens immediately after loading the grub EFI module. The error message about "gfxterm" means it was able to at least read the config file from the associated device. A restart after that seems to point to an exception inside some compiled in grub code.

If you find a grub-mkimage call in the build log you also see how many grub modules we compile into this module. Maybe this gives a hint ? Other than that kiwi generates the grub bits using the grub tooling, in case of some config error we usually land in a grub shell but a reboot of the machine is new to me too.

Thus I'm sorry I have no immediate solution at hand

[IsaacJT]

Hi @schaefi, unfortunately I haven'd had time to look into this more - I'll let you know when I have more info

[schaefi]

ok no problem, thanks for the feedback. I drop the question label and I'm sure issues with grub 2.12 will come along our way without asking :)

[schaefi]

@IsaacJT several changes have been made that touches the problems you found:

• Follow up fix for drop of hybrid boot snippets #2432
• Drop hybrid boot snippets from the GRUB 2 configuration template #2424
• Fixed grub terminal setup #2420

We are building many integration test images using grub version="2.12" and they seem to work correctly. e.g for TW

• https://build.opensuse.org/package/show/Virtualization:Appliances:Images:Testing_x86:tumbleweed/test-image-disk

Would you mind to double check if the latest version fixes things on your end too ?

There is a Staging build for testing available here:

• https://build.opensuse.org/project/show/Virtualization:Appliances:Staging

If you need another target distro to be enabled on Staging please let me know

Thanks much

[IsaacJT]

Hi @schaefi, thanks for the update!

Unfortunately I am no longer working on the project but @ahresse should be able to check this out and see if it's been resolved.

[schaefi]

ok, I guess no further feedback on this topic means it either works or is not so much of interest anymore ;) For the we built integration tests for grub 2.12 worked. I'm closing