Manual install without Fedora Kinoite #1
Comments
|
oh yea you have to specify it if you're not running within the ostree system already: ( So exporting |
|
You're sharp @M1cha, thanks a ton Strangely this command made no difference: $ export OSTREE_REPO=/ostree/repoI did it this way instead: $ mkdir -p /ostree/repo
$ ostree init --repo=/ostree/repo --mode=bare
# Then executed this after updating the last line to: ostree commit -v -b archlinux/latest --tree=dir="$rootfs" --repo=/ostree/repo
$ update
OT: Preparing transaction in repository 0x56235963c080
OT: Pushing lock non-blocking with timeout 30
OT: Opening repo lock file
OT: Push lock: state=unlocked, depth=0
OT: Locking repo shared
OT: Using new tmpdir staging-48c0fb3b-8830-4a48-9557-4305e755d132-e1LPyT
OT: Committing transaction in repository 0x56235963c080
OT: txn commit staging-48c0fb3b-8830-4a48-9557-4305e755d132-e1LPyT
OT: Popping lock non-blocking with timeout 30
OT: Pop lock: state=shared, depth=1
OT: Unlocking repoSomething is still missing though: |
|
It looks like it's assuming the sysroot is in the current directory(ostree/deploy instead of /sysroot/ostree/deploy). I guess that's because you didn't boot via ostree. But that's fine because there's a Also, I see that you created that in |
Thanks for the insights!
$ ostree init --repo=/ostree/repo --mode=bare
error: Creating repo: mkdirat: No such file or directory
Edit: My bad, just realized that
Here's a new attempt with $ mkdir -p /sysroot/ostree
$ ostree init --repo=/sysroot/ostree/repo --mode=bare
$ ostree commit -v -b archlinux/latest --tree=dir=/root/archlinux-ostree/out/rootfs --repo=/sysroot/ostree/repo
$ mkdir /sysroot/ostree/deploy
$ ostree admin --sysroot=/sysroot/ostree/deploy os-init archlinux
error: loading sysroot: opendir(ostree/repo): No such file or directory
# Another attempt
$ export OSTREE_REPO=/sysroot/ostree/repo
$ ostree admin --sysroot=$OSTREE_REPO os-init archlinux
error: loading sysroot: opendir(ostree/repo): No such file or directory |
|
|
|
Thanks a ton for the fast replies. $ ostree admin --sysroot=/sysroot os-init archlinux
ostree/deploy/archlinux initialized as OSTree root
$ ostree admin --sysroot=/sysroot deploy --os=archlinux --no-merge archlinux/latest
Relabeling /var (no stamp file 'var/.ostree-selabeled' found)
Bootloader updated; bootconfig swap: yes; bootversion: boot.1.1, deployment count change: 1Although I've been stuck on bootloader for a while, I fail to understand the structure: |
|
is In fedora they use keep in mind that if you have |
I compared $ find /sysroot/boot
/sysroot/boot
/sysroot/boot/loader.1
/sysroot/boot/loader.1/entries
/sysroot/boot/loader.1/entries/ostree-1-archlinux.conf
/sysroot/boot/ostree
/sysroot/boot/ostree/archlinux-f7246e5ebdf5182e6157bacdbead3bf9eef5aa55517fa882fb05583b4e70ec25
/sysroot/boot/ostree/archlinux-f7246e5ebdf5182e6157bacdbead3bf9eef5aa55517fa882fb05583b4e70ec25/vmlinuz-6.4.12-zen1-1-zen
/sysroot/boot/ostree/archlinux-f7246e5ebdf5182e6157bacdbead3bf9eef5aa55517fa882fb05583b4e70ec25/initramfs-6.4.12-zen1-1-zen.img
/sysroot/boot/boot
/sysroot/boot/loader
$ find /boot ! -name "*.png" ! -name "*.pf2" ! -name "*.mo" ! -name "*.mod" ! -name "*.lst"
/boot
/boot/vmlinuz-linux
/boot/initramfs-linux.img
/boot/initramfs-linux-fallback.img
/boot/grub
/boot/grub/x86_64-efi
/boot/grub/x86_64-efi/modinfo.sh
/boot/grub/x86_64-efi/core.efi
/boot/grub/x86_64-efi/grub.efi
/boot/grub/locale
/boot/grub/themes
/boot/grub/themes/starfield
/boot/grub/themes/starfield/COPYING.CC-BY-SA-3.0
/boot/grub/themes/starfield/README
/boot/grub/themes/starfield/theme.txt
/boot/grub/fonts
/boot/grub/grubenv
/boot/grub/grub.cfg
/boot/EFI
/boot/EFI/BOOT
/boot/EFI/BOOT/BOOTX64.EFI
It sounds like
That's the case! I tried to put it together like this: $ lsblk -o +UUID
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
UUID
sda 8:0 0 127G 0 disk
├─sda1 8:1 0 512M 0 part /boot D668-25F1
└─sda2 8:2 0 126.5G 0 part / 6a36a809-abcd-40a5-b0c9-b9a53905e75e# /etc/grub.d/00_boot.cfg
search --no-floppy --fs-uuid --set=dev 6a36a809-abcd-40a5-b0c9-b9a53905e75e
set prefix=($dev)/grub
search --no-floppy --fs-uuid --set=root 6a36a809-abcd-40a5-b0c9-b9a53905e75e
search --no-floppy --fs-uuid --set=boot D668-25F1
export $prefix
configfile $prefix/grub.cfg# /etc/grub.d/10_ostree.cfg
# Contents of: https://github.com/ostreedev/ostree/blob/v2023.6/src/boot/grub2/grub2-15_ostree$ grub-mkconfig -o /boot/grub/grub.cfg
Found linux image: /boot/vmlinuz-linux
Found initrd image: /boot/initramfs-linux.img
Found fallback initrd image(s) in /boot: initramfs-linux-fallback.img
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
$ reboot
$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-linux root=UUID=6a36a809-abcd-40a5-b0c9-b9a53905e75e rw rootfstype=xfs loglevel=3 quietI'm probably missing something since I can't grasp how grub is supposed to load contents from Sorry I'm really confused about the bootloader part |
ostree follows the GNU boot loader specification. The files in
|
|
Thanks, that makes sense I moved ### BEGIN /etc/grub.d/15_ostree ###
### END /etc/grub.d/15_ostree ###I think this condition is failing: # https://github.com/ostreedev/ostree/blob/v2023.6/src/boot/grub2/grub2-15_ostree#L23C6-L23C26
test -d /ostree/repoNot really sure where to go from here :/ |
|
I don't know if mkconfig has a verbose flag but if they keep stdout connected you could add What my scripts do is based on this: https://ostreedev.github.io/ostree/adapting-existing/ |
Hehehe it's classic to have docs which only covers some parts of the implementation 💦
That's really clever considering # grub-mkconfig -o /boot/grub/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-linux
Found initrd image: /boot/initramfs-linux.img
Found fallback initrd image(s) in /boot: initramfs-linux-fallback.img
+ which ostree
+ test -d /ostree/repo
+ exit 0
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
doneYeah the issue seems to be that
Just to recap all the commands, I'm building $ mkdir -p /sysroot/ostree
$ ostree init --repo=/sysroot/ostree/repo --mode=bare
$ ostree commit -v -b archlinux/latest --tree=dir=/root/archlinux-ostree/out/rootfs --repo=/sysroot/ostree/repo
$ mkdir /sysroot/ostree/deploy
$ ostree admin --sysroot=/sysroot os-init archlinux
$ ostree admin --sysroot=/sysroot deploy --os=archlinux --no-merge archlinux/latestIn this case, maybe it makes more sense to do it directly in [root@arch ~]# mkdir /ostree
[root@arch ~]# ostree init --repo=/ostree/repo --mode=bare
[root@arch ~]# ostree commit -v -b archlinux/latest --tree=dir=/root/archlinux-ostree/out/rootfs --repo=/ostree/repo
OT: Preparing transaction in repository 0x55d991ddd080
OT: Pushing lock non-blocking with timeout 30
OT: Opening repo lock file
OT: Push lock: state=unlocked, depth=0
OT: Locking repo shared
OT: Using new tmpdir staging-a9b9c0dd-fc52-491a-aaea-346d924c7c53-cjqXhc
OT: Committing transaction in repository 0x55d991ddd080
OT: txn commit staging-a9b9c0dd-fc52-491a-aaea-346d924c7c53-cjqXhc
OT: Popping lock non-blocking with timeout 30
OT: Pop lock: state=shared, depth=1
OT: Unlocking repo
f3561001d80523d1d1162376ab54003f3827da2b55ca743a346350573293c656
[root@arch ~]# mkdir /ostree/deploy
[root@arch ~]# ostree admin --sysroot=/ os-init archlinux
ostree/deploy/archlinux initialized as OSTree root
[root@arch ~]# ostree admin --sysroot=/ deploy --os=archlinux --no-merge archlinux/latest
Relabeling /var (no stamp file 'var/.ostree-selabeled' found)
error: Preparing final bootloader swap: symlinkat: Operation not permittedSeems like it's not a good idea on a running system, instead with chroot: $ mkdir -p /mnt/arch/
$ mount /dev/sda2 /mnt/arch/
$ mount /dev/sda1 /mnt/arch/boot
$ arch-chroot /mnt/arch
$ ostree admin --sysroot=/ deploy --os=archlinux --no-merge archlinux/latest
error: opendir(var): No such file or directory
$ pwd
/
$ ls -d /var
/var
$ ls -d /ostree/deploy/archlinux/deploy/*/var
/ostree/deploy/archlinux/deploy/55f3f25edbe2ca0233c8b4bba6f61c110f21364c7bcbd6cfe1eac58c34331e5e.0/varOh man, feels like we're so close! |
|
On fedora, # ls -lah /ostree
lrwxrwxrwx 3 root root 14 Jul 29 21:22 /ostree -> sysroot/ostree |
|
Thanks! $ ln -s /sysroot/ostree /ostree
$ grub-mkconfig -o /boot/grub/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-linux
Found initrd image: /boot/initramfs-linux.img
Found fallback initrd image(s) in /boot: initramfs-linux-fallback.img
+ which ostree
+ test -d /ostree/repo
+ . /etc/default/grub
++ GRUB_DEFAULT=0
++ GRUB_TIMEOUT=5
++ GRUB_DISTRIBUTOR=Arch
++ GRUB_CMDLINE_LINUX_DEFAULT='loglevel=3 quiet'
++ GRUB_CMDLINE_LINUX=rootfstype=xfs
++ GRUB_PRELOAD_MODULES='part_gpt part_msdos'
++ GRUB_TIMEOUT_STYLE=menu
++ GRUB_TERMINAL_INPUT=console
++ GRUB_GFXMODE=auto
++ GRUB_GFXPAYLOAD_LINUX=keep
++ GRUB_DISABLE_RECOVERY=true
+ test -f /boot/grub2/.grub2-blscfg-supported
+ test -n /dev/sda2
+ set -e
+ . /usr/share/grub/grub-mkconfig_lib
++ prefix=/usr
++ exec_prefix=/usr
++ datarootdir=/usr/share
++ datadir=/usr/share
++ bindir=/usr/bin
++ sbindir=/usr/bin
++ '[' x/usr/share/grub = x ']'
++ test x = x
++ grub_probe=/usr/bin/grub-probe
++ test x = x
++ grub_file=/usr/bin/grub-file
++ test x = x
++ grub_mkrelpath=/usr/bin/grub-mkrelpath
++ command -v gettext
++ :
++ grub_tab=' '
+ DEVICE=/dev/sda1
++ grub_get_device_id /dev/sda1
++ old_ifs='
'
++ IFS='
'
++ device=/dev/sda1
++ '[' x '!=' xtrue ']'
+++ /usr/bin/grub-probe --device /dev/sda1 --target=fs_uuid
++ fs_uuid=D668-25F1
++ echo D668-25F1
++ IFS='
'
+ GRUB2_BOOT_DEVICE_ID=D668-25F1
+ export GRUB2_BOOT_DEVICE_ID
++ prepare_grub_to_access_device /dev/sda1
++ old_ifs='
'
++ IFS='
'
+++ /usr/bin/grub-probe --device /dev/sda1 --target=partmap
++ partmap=gpt
++ for module in ${partmap}
++ case "${module}" in
++ echo 'insmod part_gpt'
+++ /usr/bin/grub-probe --device /dev/sda1 --target=abstraction
++ abstraction=
+++ /usr/bin/grub-probe --device /dev/sda1 --target=fs
++ fs=fat
++ for module in ${fs}
++ echo 'insmod fat'
++ '[' x = xy ']'
+++ /usr/bin/grub-probe --device /dev/sda1 --target=compatibility_hint
++ fs_hint=hd0,gpt1
++ '[' xhd0,gpt1 '!=' x ']'
++ echo 'set root='\''hd0,gpt1'\'''
++ '[' x '!=' xtrue ']'
+++ /usr/bin/grub-probe --device /dev/sda1 --target=fs_uuid
++ fs_uuid=D668-25F1
+++ /usr/bin/grub-probe --device /dev/sda1 --target=hints_string
++ hints='--hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 '
++ '[' 'x--hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 ' '!=' x ']'
++ echo 'if [ x$feature_platform_search_hint = xy ]; then'
++ echo ' search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 D668-25F1'
++ echo else
++ echo ' search --no-floppy --fs-uuid --set=root D668-25F1'
++ echo fi
++ IFS='
'
+ GRUB2_PREPARE_ROOT_CACHE='insmod part_gpt
insmod fat
set root='\''hd0,gpt1'\''
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 D668-25F1
else
search --no-floppy --fs-uuid --set=root D668-25F1
fi'
+ export GRUB2_PREPARE_ROOT_CACHE
+ exec ostree admin instutil grub2-generate
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
doneOut of curiously I tried to run the command from $ ostree admin instutil grub2-generate
**
OSTree:ERROR:src/libostree/ostree-bootloader-grub2.c:164:_ostree_bootloader_grub2_generate_config: assertion failed: (grub2_boot_device_id != NULL)
Bail out! OSTree:ERROR:src/libostree/ostree-bootloader-grub2.c:164:_ostree_bootloader_grub2_generate_config: assertion failed: (grub2_boot_device_id != NULL)
Aborted (core dumped)There's a piece missing in this lego set 👀 |
|
|
|
Need more symlinks to somewhere, do you have these? $ ostree admin instutil grub2-generate --verbose
OT: Didn't find $sysroot/boot/loader symlink; assuming bootversion 0
OT: Didn't find $sysroot/ostree/boot.0 symlink; assuming subbootversion 0
**
OSTree:ERROR:src/libostree/ostree-bootloader-grub2.c:164:_ostree_bootloader_grub2_generate_config: assertion failed: (grub2_boot_device_id != NULL)
Bail out! OSTree:ERROR:src/libostree/ostree-bootloader-grub2.c:164:_ostree_bootloader_grub2_generate_config: assertion failed: (grub2_boot_device_id != NULL)
Aborted (core dumped) |
|
I don't think running that outside of the grub script is the right way if you get errors when previously you didn't. My |
|
Hmm, took a few steps back found out that $ ls -lah /sysroot/boot
total 0
drwxr-xr-x 4 root root 62 Aug 29 09:12 .
drwxr-xr-x 4 root root 32 Aug 29 09:12 ..
lrwxrwxrwx 1 root root 1 Aug 29 09:12 boot -> .
lrwxrwxrwx 1 root root 8 Aug 29 09:12 loader -> loader.1
drwxr-xr-x 3 root root 21 Aug 29 09:12 loader.1
drwxr-xr-x 3 root root 88 Aug 29 09:12 ostree
$ ls -lah /sysroot/ostree/
total 0
drwxr-xr-x 5 root root 62 Aug 29 09:12 .
drwxr-xr-x 4 root root 32 Aug 29 09:12 ..
lrwxrwxrwx 1 root root 8 Aug 29 09:12 boot.1 -> boot.1.1
drwxr-xr-x 3 root root 23 Aug 29 09:12 boot.1.1
drwxr-xr-x 3 root root 23 Aug 29 09:12 deploy
drwxr-xr-x 7 root root 102 Aug 29 09:12 repoI came across this also: https://wiki.gnome.org/Projects/OSTree/DeploymentModel2#Redeploying Seems like files in $ cp -rf /sysroot/boot/* /boot/
cp: cannot create symbolic link '/boot/boot': Operation not permitted
cp: cannot create symbolic link '/boot/loader': Operation not permittedWell, that's not the right way of doing it 😂 |
|
Well, this explains why files are in the wrong location ( |
|
oh yea that makes sense. Using |
M1cha
changed the title
|
I got way too happy when I saw that issue report and the resolution, I jinxed it: [root@arch ~]# mkdir -p /sysroot/ostree
[root@arch ~]# ostree init --repo=/sysroot/ostree/repo --mode=bare
[root@arch ~]# ostree commit -v -b archlinux/latest --tree=dir=/root/archlinux-ostree/out/rootfs --repo=/sysroot/ostree/repo
OT: Preparing transaction in repository 0x56183e5cb070
OT: Pushing lock non-blocking with timeout 30
OT: Opening repo lock file
OT: Push lock: state=unlocked, depth=0
OT: Locking repo shared
OT: Using new tmpdir staging-aba45dca-9f39-4f3b-9154-9d877bdfff61-bmFwhf
OT: Committing transaction in repository 0x56183e5cb070
OT: txn commit staging-aba45dca-9f39-4f3b-9154-9d877bdfff61-bmFwhf
OT: Popping lock non-blocking with timeout 30
OT: Pop lock: state=shared, depth=1
OT: Unlocking repo
fa7daa856e1cff367d8a3a081bf3db981463c4eb552a71aa2f1c2f1f6b138865
[root@arch ~]# mkdir /sysroot/ostree/deploy
[root@arch ~]# ostree admin --sysroot=/ os-init archlinux
error: loading sysroot: opendir(ostree/repo): No such file or directory
[root@arch ~]# ostree admin --sysroot=/sysroot os-init archlinux
ostree/deploy/archlinux initialized as OSTree root
[root@arch ~]# ostree admin --sysroot=/ deploy --os=archlinux --no-merge archlinux/latest
error: No such file or directory
[root@arch ~]# ostree admin deploy --os=archlinux --no-merge archlinux/latest
error: No such file or directory
[root@arch ~]# ostree admin deploy --verbose
error: No such file or directoryAlso tried updating OSTree from |
|
Turns out $ ln -s /sysroot/ostree /ostree
$ ostree admin --sysroot=/sysroot os-init archlinux
$ ostree admin --sysroot=/ deploy --os=archlinux --no-merge archlinux/latest
Relabeling /var (no stamp file 'var/.ostree-selabeled' found)
error: Bootloader write config: No KERNEL argument found after LABELNow this is new. [root@arch ~]# ln -s /sysroot/ostree /ostree
[root@arch ~]# mkdir /sysroot/ostree/deploy
[root@arch ~]# ostree admin --sysroot=/ os-init archlinux
ostree/deploy/archlinux initialized as OSTree stateroot
[root@arch ~]# ostree admin --sysroot=/ deploy --os=archlinux --no-merge archlinux/latest
Relabeling /var (no stamp file 'var/.ostree-selabeled' found)
error: Preparing final bootloader swap: symlinkat: Operation not permitted |
Sounds like this approach might need some adjustments to work with a (UEFI) FAT32 |
|
Turns out you can use a separate (
$ lsblk -o +FSTYPE
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS FSTYPE
sdb 8:16 0 127G 0 disk
|-sdb1 8:17 0 512M 0 part /boot/efi vfat
|-sdb2 8:18 0 20G 0 part /home ext4
`-sdb3 8:19 0 106.5G 0 part / ext4# /etc/fstab
# Generated with: $ genfstab -U /mnt/arch >> /mnt/arch/etc/fstab
# <file system> <dir> <type> <options> <dump> <pass>
# /dev/sdb3 LABEL=SYS_ROOT
UUID=661f46d0-e6de-4df5-8e0c-2b6f92016f32 / ext4 rw,relatime 0 1
# /dev/sdb2 LABEL=SYS_HOME
UUID=c8f8e836-5b0c-4f52-a36c-9d04717243fc /home ext4 rw,relatime 0 2
# /dev/sdb1 LABEL=SYS_BOOT
UUID=B20C-7F9C /boot/efi vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 2There's a lot of ways to use a ESP partition (e.g. various paths, by adding $ pacman -S grub
# --removable makes it detectable without having to use efibootmgr
$ grub-install --target=x86_64-efi --efi-directory=/boot/efi --removable --boot-directory=/boot/efi/EFI --bootloader-id=archlinux
$ grub-mkconfig -o /boot/efi/EFI/grub/grub.cfgThe current OSTree setup looks like this: # OSTree DEPS
$ pacman -S ostree which wget
# OSTREE REPO INITALIZATION
$ mkdir -p /sysroot/ostree
$ ostree init --repo=/sysroot/ostree/repo --mode=bare
$ ostree commit -v -b archlinux/latest --tree=dir=/root/archlinux-ostree/out/rootfs --repo=/sysroot/ostree/repo
$ ln -s /sysroot/ostree /ostree
# OSTREE REPO ACTIVATION
$ mkdir /sysroot/ostree/deploy
$ ostree admin --sysroot=/ os-init archlinux
$ ostree admin --sysroot=/ deploy --os=archlinux --no-merge archlinux/latest
# OSTREE BOOTLOADER
$ wget https://raw.githubusercontent.com/ostreedev/ostree/v2023.6/src/boot/grub2/grub2-15_ostree -O /etc/grub.d/15_ostree
$ chmod +x /etc/grub.d/15_ostree
$ grub-mkconfig -o /boot/efi/EFI/grub/grub.cfgTouch down 😎
This still needs a bit more work but it feels like it's getting close: |
|
Sorry for the late answer.
That's similar to what fedora does:
This sounds similar to what I had before I set |
|
In that case grub might need a few more tweaks! But firstly, just realized something funny is going on. With a clean install (before adding any ostree related steps from above): # ls /boot
efi initramfs-linux.img vmlinuz-linuxThen when following OSTree setup with commands from above (up to $ ostree admin --sysroot=/ deploy --os=archlinux --no-merge archlinux/latest
Bootloader updated; bootconfig swap: yes; bootversion: boot.1.1, deployment count change: 1
# ls /boot
boot efi initramfs-linux.img loader loader.1 ostree vmlinuz-linux
# ls /boot/boot
boot efi initramfs-linux.img loader loader.1 ostree vmlinuz-linux
# ls /boot/boot/boot
boot efi initramfs-linux.img loader loader.1 ostree vmlinuz-linux
# ls /boot/boot/boot/boot/boot/boot/boot/boot/boot/boot/boot/boot/boot/boot
boot efi initramfs-linux.img loader loader.1 ostree vmlinuz-linuxContents of |
That's because it's a symbolic link. I recommend |
|
My bad, thanks for pointing it out!
If I inspect the It looks like it's searching the right path:
Since those exist here: $ find /boot -iname *fcad05e9f4bbe2a8e94763dda2542a4bf53ee9af812afb02de6d28f6051e9d1b*
/boot/ostree/archlinux-fcad05e9f4bbe2a8e94763dda2542a4bf53ee9af812afb02de6d28f6051e9d1b
$ ls /boot/ostree/archlinux-fcad05e9f4bbe2a8e94763dda2542a4bf53ee9af812afb02de6d28f6051e9d1b
initramfs-6.4.12-zen1-1-zen.img vmlinuz-6.4.12-zen1-1-zen
$ find /sysroot -iname fcad05e9f4bbe2a8e94763dda2542a4bf53ee9af812afb02de6d28f6051e9d1b
/sysroot/ostree/boot.1.1/archlinux/fcad05e9f4bbe2a8e94763dda2542a4bf53ee9af812afb02de6d28f6051e9d1b
$ find /ostree -iname fcad05e9f4bbe2a8e94763dda2542a4bf53ee9af812afb02de6d28f6051e9d1b
$ find /ostree/ -iname fcad05e9f4bbe2a8e94763dda2542a4bf53ee9af812afb02de6d28f6051e9d1b
/ostree/boot.1.1/archlinux/fcad05e9f4bbe2a8e94763dda2542a4bf53ee9af812afb02de6d28f6051e9d1b
$ ls -l /ostree/boot.1
lrwxrwxrwx 1 root root 8 Sep 1 10:49 /ostree/boot.1 -> boot.1.1
$ ls /ostree/boot.1/archlinux/fcad05e9f4bbe2a8e94763dda2542a4bf53ee9af812afb02de6d28f6051e9d1b/0
bin boot dev etc home lib lib64 mnt opt ostree proc root run sbin srv sys sysroot tmp usr varBut I'm not sure if grub is able to see them or if something is wrong with the symlink (
Hmm, I gave it a shot anyway since I'm stuck: $ lsblk -o +UUID
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS UUID
sdb 8:16 0 127G 0 disk
|-sdb1 8:17 0 512M 0 part /boot/efi B20C-7F9C
|-sdb2 8:18 0 20G 0 part /home c8f8e836-5b0c-4f52-a36c-9d04717243fc
`-sdb3 8:19 0 106.5G 0 part / 661f46d0-e6de-4df5-8e0c-2b6f92016f32
$ ls /boot/efi/EFI/grub
fonts grub.cfg grub.cfg.old grubenv locale themes x86_64-efi
# nano /etc/grub.d/40_custom
search --no-floppy --fs-uuid --set=dev 661f46d0-e6de-4df5-8e0c-2b6f92016f32
set prefix=($dev)/boot/efi/EFI/grub
search --no-floppy --fs-uuid --set=root 661f46d0-e6de-4df5-8e0c-2b6f92016f32
search --no-floppy --fs-uuid --set=boot B20C-7F9C
export $prefix
configfile $prefix/grub.cfg
$ grub-mkconfig -o /boot/efi/EFI/grub/grub.cfgLeads to:
The file is there though! |
|
grub has a shell as well (press escape). That might help exploring the issue. |
Here's how it looks inside:
Right now I have If I understand this correctly Fedora is able to find the right files with I suppose I don't have a choice but to make |
|
While researching some problems here I stumbled upon Gnome's QEMU VM setup which skips bootloader creation altogether and just copies the right files to They also have a working Raspberry Pi 4 setup with U-Boot here:
Solved!
Solved! $ ostree config --repo=/ostree/repo set sysroot.bootprefix 'true'
Solved! |
|
FINALLY IT WORKS !! 🎉 🎉 🎉 I decided to scratch I also figured out how to add custom kernel boot flags, use e.g. Here's the final setup in a CI/CD fashion: # OSTREE: DEPS
$ pacman --noconfirm -S wget which
# OSTREE: DEPS (workaround missing --with-builtin-grub2-mkconfig)
$ pacman --noconfirm -S sudo git
$ git clone https://gitlab.archlinux.org/archlinux/packaging/packages/ostree.git /home/build/ostree
$ cd /home/build/ostree && chown -R nobody $(pwd)
$ source PKGBUILD && pacman -S --noconfirm --needed --asdeps base-devel "${makedepends[@]}" "${depends[@]}"
$ sed -i '/--with-builtin-grub2-mkconfig/a \ \ \ \ --with-modern-grub' PKGBUILD
$ sudo -u nobody makepkg -Cf --nocheck
$ pacman --noconfirm -U *.pkg.tar.zst
# OSTREE: REPO INITALIZATION
$ ostree admin init-fs --sysroot=/ --modern /
$ ostree init --repo=/ostree/repo --mode=bare
$ ostree config --repo=/ostree/repo set sysroot.bootprefix 'true'
$ ostree commit --repo=/ostree/repo --tree=dir=/root/archlinux-ostree/out/rootfs --branch=archlinux/latest -v
# OSTREE: REPO ACTIVATION
$ ostree admin os-init --sysroot=/ archlinux
$ ostree admin deploy --sysroot=/ --karg="root=LABEL=SYS_ROOT" --karg="rw" --os=archlinux --no-merge --retain archlinux/latest
# OSTREE: BOOTLOADER (GRUB2)
$ wget https://raw.githubusercontent.com/ostreedev/ostree/v2023.6/src/boot/grub2/grub2-15_ostree -O /etc/grub.d/15_ostree && chmod +x /etc/grub.d/15_ostree
$ grub-mkconfig -o /boot/efi/EFI/grub/grub.cfg
# OSTREE: EXTRA
cp /etc/{fstab,crypttab,shadow} /ostree/deploy/archlinux/deploy/*/etc/For reference: $ lsblk -o +FSTYPE,LABEL
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS FSTYPE LABEL
sdb 8:16 0 127G 0 disk
|-sdb1 8:17 0 512M 0 part /boot/efi vfat SYS_BOOT
|-sdb2 8:18 0 20G 0 part /home ext4 SYS_HOME
`-sdb3 8:19 0 106.5G 0 part / ext4 SYS_ROOTAll of this translates to: |
|
that's awesome news, great work 🥳 Also thanks for the hint about At this point I'm wondering what we could do with all that information. Make an ostree installation guide in the arch wiki? Create a tool to automate all those steps? Also, would it be worth to create tools for generating the rootfs or did the overlap between my scripts and yours end up small anyway? |
|
🚀 🚀 🚀
Great idea!
Ideally I think it would be wiser to let the user assemble their OSTree installation from scratch rather than rely on scripts as it gives more flexibility especially once systemd-boot support rolls in
I'm still using your script to build a rootfs since I haven't come up with a easier approach yet Have you looked into Perhaps it be better to try to do something similar to UBlue/OSTree Container by making a Docker/Image which creates the correct filesystem layout (with For reference, Elemental-Toolkit (https://github.com/rancher/elemental-toolkit/blob/main/examples/tumbleweed/Dockerfile) and KairOS (https://github.com/kairos-io/kairos/tree/master/examples) are alternative projects to OSTree which already upgrades the base system through OCI images. (For context, I'm not sure where |
That's actually how this whole project started out but I had so many issues that I gave that up. The tool that can do that is called bootc, and it uses functionality from ostree-rs-ext. I didn't manage to find a way to build a container in a way or convert it in a way that ends up in a usable ostree image
Additionally the whole process was painfully slow. I probably could have improved this a lot by doing all the work in a single
here is my old code for that: https://github.com/M1cha/bootc-archlinux |
|
Thanks for sharing, your effort with BootC definitely saved me from pursuing another rabbit hole 🙂 In this case, I think the only missing puzzle piece is a minimal set of commands for building a rootfs |
|
The updated ostree package broke my current setup 😅 I'll migrate to the mkconfig setup since it's better anyway but it's sad that something trivial like this is defined at compile time rather than detected at runtime within the ostree tool. 🤷
I'll try to create a tool and/or a bash library to simplify the whole thing. For now this will be archlinux only since there isn't much code in this repo that would help other distros right now. |
Even Gentoo's EBuild is hardcoded to use mkconfig, I don't know if doing it this way 4451680c is counterproductive or if it warrants a bug request. I suppose Arch Linux was never about freedom of choice but rather simplicity
Awesome, thanks a ton 🙏🏼 |
I've been playing around with building OSTree onto a empty partition (meaning rootfs only contains a bootloader and has no system outside of OSTree) to keep it super minimal, it almost works 🙂 $ find /mnt -maxdepth 2
/mnt
/mnt/ostree
/mnt/ostree/repo
/mnt/ostree/boot.1
/mnt/ostree/deploy
/mnt/ostree/boot.1.1
/mnt/boot
/mnt/boot/loader.1
/mnt/boot/efi
/mnt/boot/ostree
/mnt/boot/loader
/mnt/boot/boot
$ lsblk -o NAME,TYPE,MOUNTPOINTS
NAME TYPE MOUNTPOINTS
sda disk
├─sda1 part /boot
├─sda2 part /home
└─sda3 part /
sdb disk
├─sdb1 part /mnt/boot/efi
├─sdb2 part
└─sdb3 part /mntExcept for generation of boot entries.. Turns out
In other words: the idea is to build a OSTree system from a "rescue cd" onto a mountpoint, I'm trying to think of a clean way of doing this:
I don't fully understand it but OSTree skips |
That'd be great. Having a guide that let's you quickly set up a new ostree system using the official live ISO would be ideal. I think what you need is a chroot so you can enter a system that is similar enough to the ostree system that the bootloader setup works. Either you can just enter the rootfs directory you already have from the builder or you can use
BLS is an unofficial patchset that allows grub to load loader entries directly. Fedora seems to be applying those patches to their grub package but it doesn't look like they're actually using it. While this might be the technical ideal it means that we would have to maintain a grub-bls package. |
Definitely! Leaving a summary here of the findings:
Here's a working approach which uses the rootfs from the builder to keep the amount of steps to the minimum # Install OSTree boot entries
$ ostree admin deploy --sysroot=/mnt --karg="root=LABEL=SYS_ROOT" --karg="rw" --os=archlinux --no-merge --retain archlinux/latest
# Install ESP
$ grub-install --target=x86_64-efi --efi-directory=/mnt/boot/efi --removable --boot-directory=/mnt/boot/efi/EFI --bootloader-id=archlinuxezez /dev/sdb1
# Enable OSTree boot entries [workaround grub-mkconfig inability to change rootfs]
$ pacman --noconfirm -S arch-install-scripts
$ export OSTREE_SYS_PATH=$(ls -d /mnt/ostree/deploy/archlinux/deploy/*|head -n 1)
$ rm -rfv ${OSTREE_SYS_PATH}/boot/*
$ mount --mkdir --rbind /mnt/boot ${OSTREE_SYS_PATH}/boot
$ mount --mkdir --rbind /mnt/ostree ${OSTREE_SYS_PATH}/sysroot/ostree
$ arch-chroot ${OSTREE_SYS_PATH}
$ grub-mkconfig -o /boot/efi/EFI/grub/grub.cfg
$ exit && cd && umount -R /mntThe commands in the workaround section are entirely unnecessary if there is a way to do a pseudo chroot for Surely there must be a better way to do this which isn't as convoluted 🤔 |
|
You should open a discussion with the grub maintainers. My fear is that it's not easy to fix on their side though because most things including the extension scripts will probably assume that the current system is the target system. The arch wiki even warns about running grub-mkconfig and even grub-install outside of the final rootfs. Options like boot-directory seem to be an afterthought. Given all of that I'd actually say that the most compatible solution is a chroot into an ostree rootfs. I'm not sure if doing that on the deployment is safe in terms of unwanted modifications to it though. This solution would also work with other bootloaders. Keep in mind that ostree supports many. The cleanest solution would probably be using a grub fork with bls support. The loader specification is supposed to be generic so any bootloader with support for it should be able to boot into ostree without generating any configs. Given all of the options I'd probably prefer this one even if it complicates the initial setup due to having to install an AUR helper first. |
|
Thinking more about my second suggestion: Technically you'd still want to do that from within the chroot:
Given that train of thought I think it would be clean, generic and useful to extend the ostree cli with a subcommand that takes an image name or hash and a path as arguments and then creates and chroots into a temporary deployment. Deployments are fast to create because they use hardlinks. |
|
Thanks for sharing your thoughts! I think the first step could be to open a dialog with GRUB maintainers to see what they think, even if the final solution doesn't use grub-mkconfig, it'd be beneficial for users who'd like to generate a grub.cfg without chroot (since I encountered some users having this issue while doing ephemeral root) Grub with BLS sounds like a great idea even if it requires AUR (since Arch Wiki often refers to AUR packages anyway) and I like the idea about integrating it with ostree cli! I have a hard time wrapping my head around BLS though, not sure I can be much of help in that regard. I went ahead and summarized all the commands which I'm currently using to automatically build a OSTree system. Ended up making the code into a repository instead of a gist since this way it's easier to share/clone into a LiveCD with QEMU/VM. I might Not sure how the final tools/scripts/usage instructions will look like but hopefully this should make it easier to figure that out. I ended up building the entire OSTree in Podman with layer caching, only part which isn't cached is OSTree repo setup bits and partitioning. Feel free to take a look |
|
Sorry for the very late answer. |
No worries
This definitely needs to be addressed for desktop usage Someone recently added this wiki entry https://wiki.archlinux.org/title/Install_Arch_Linux_via_Docker Nonetheless, I toyed around with various approaches. This is probably the worst as it can't use Pacman cache while creating rootfs (if there is one mounted) thus it's the slowest (unless you want to keep a local copy of # time podman build --file="Containerfile.bootstrap" --no-cache
FROM docker.io/library/archlinux AS rootfs
RUN curl https://mirror.bahnhof.net/pub/archlinux/iso/2023.09.01/archlinux-bootstrap-x86_64.tar.gz -o archlinux-bootstrap-x86_64.tar.gz && \
tar -C /mnt -xzf archlinux-bootstrap-x86_64.tar.gz --numeric-owner && \
curl -s -L "https://archlinux.org/mirrorlist/?country=SE&protocol=http&ip_version=4" | sed -e 's/^#Server/Server/' -e '/^#/d' > /mnt/root.x86_64/etc/pacman.d/mirrorlist
FROM scratch
COPY --from=rootfs /mnt/root.x86_64 /
RUN pacman-key --init && \
pacman-key --populate && \
pacman --noconfirm -Syu
Out of curiously I tried reinstalling every package (without a pkg cache) as unlike Debian's FROM docker.io/library/archlinux AS rootfs
RUN sed -i -e "s|^NoExtract.*||g" /etc/pacman.conf && \
pacman --noconfirm -Sy $(pacman -Qq)
This can be even faster if you use a package cache for consecutive builds (
Here's the cleanest alternative code wise but it's just as slow as bootstrap: # time podman build --file="Containerfile.pacstrap" --cap-add="SYS_ADMIN" --no-cache
FROM docker.io/library/archlinux AS rootfs
# pacstrap uses Pacman config from the host
RUN sed -i -e "s|^NoExtract.*||g" /etc/pacman.conf && \
pacman --noconfirm -Sy arch-install-scripts && \
pacstrap -K /mnt base
FROM scratch AS base
COPY --from=rootfs /mnt /
Using a package cache made no difference here In summary:
I guess the increase in built time with Pacstrap approach is neglectable once the result is cached on disk as a tagged- and reusable Podman (base) image. Do you think there is any other way of going about this? |
Yes, that's why I do this: archlinux-ostree/scripts/makeroot Line 349 in 03829bc
There is a
NoExtract is not the only change. Sure, we can try to revert all of their changes but we'd have to stay updated on their dockerfile changes so we can check if there's new stuff that we don't want. Personally, I find that risky and too much work.
I agree as that's essentially exactly the same what my current bash scripts do, but utilizing podman containers instead of arch-chroot which is WAY cleaner. Good idea 👍
For me it makes a huge difference because The package download takes like 1-2 minutes for me. Admittedly it could probably be optimized by selecting a better mirror but the more packages you install the more of a difference it makes. Also, if you update often(e.g. during phases of many security updates) then you save time since in most cases it's just gonna download like 1-5 packages. We might actually be able to prevent the slow rootfs copy by using 2 dockerfiles instead of a single multi-stage dockerfile:
|
Just to expand on this, I ended up finding another method It's possible to use a script which fetches packages directly and unpacks them into a rootfs, this way you don't need to have Pacman or Pacstrap installed to build a new system. Here's a up to date example from Artix's repo: For Arch Linux here's two examples:
Good points, thanks for bringing them up! Here's another reason for using Pacstrap. I'm working on a CachyOS image right now, first using Arch Linux
Yupp, that's exactly what I meant by re-usable base image 🙂 On side note |
Just a quick update:
I failed to find anything else which could help us with getting rid of
Well, the COPY instruction add +8.5s (on highend hardware). It's not that bad at the end of the day if there's no other alternative. After the base image has been built, the next image where customization takes place wont be slowed down by this unless the user chooses to rebuild the base. |
No, but I never have more than two grub entries though that might be an artifact of me not having migrated to the new grub setup yet. Fedora did have like 3 or 4 entries but that logic might have come from their rpm-ostree wrapper. Another thing I kinda forgot about through all these discussions is that after you built the rootfs in a container you still need to do a commit. It looks like that's what you're doing here: So you have to copy the base rootfs, then when building the final rootfs podman has to copy it to wherever they store images, then when you want to export it, podman has to tar it, you untar it and copy it to some disk location. I don't think I like that process. I'd probably prefer a combination of both our approaches: keep using podman instead of arch-chroot, but without Containerfiles and using |
Maybe this can be hidden with BLS?
This indeed affects the performance, I did a experiment last week. # [TAR] export rootfs
$ time sh -c "rm -rf ./rootfs.tar && podman build -f Containerfile -o dest=./rootfs.tar,type=tar"
real 0m4.892s
user 0m4.021s
sys 0m7.704s
# [TAR] commit to OSTree
$ time sh -c 'ostree commit --repo=/ostree/repo --tree=tar=rootfs.tar --tar-autocreate-parents --branch=archlinux/latest -v && ostree admin deploy --sysroot=/ --karg="root=LABEL=SYS_ROOT" --karg="rw" --os=archlinux --no-merge --retain archlinux/latest'
real 0m10.669s
user 0m5.120s
sys 0m4.598svs # [DIR] export rootfs (this breaks file permissions) *invalid*
# $ time sh -c "rm -rf ./rootfs && podman build -f Containerfile -o dest=./rootfs,type=local"
# real 0m5.811s
# user 0m5.839s
# sys 0m10.875s
# [DIR] time sh -c 'rm -rf rootfs && podman export $(podman create rootfs bash) | tar -xC rootfs'
real 0m6.795s
user 0m2.011s
sys 0m9.950s
# [DIR] commit to OSTree
$ time sh -c 'ostree commit --repo=/ostree/repo --tree=dir=rootfs --branch=archlinux/latest -v && ostree admin deploy --sysroot=/ --karg="root=LABEL=SYS_ROOT" --karg="rw" --os=archlinux --no-merge --retain archlinux/latest'
real 0m10.185s
user 0m4.118s
sys 0m5.133sIn summary:
This would result in cleaner code for sure! If we want to Pacstrap each new deployment in
Just removing the bottlenecks saves around 15.3s (but the real world number would be higher as creating cache for each Well from experience, layer caching is quite a big drain on storage space and you can't use it forever (usually after 1-2 weeks Arch's repository has a complete new set of packages which breaks new installs and this cache is useless if you want to update (without Just food for thought ^
Yepp, the script is designed to be used in a very specific way with a small room for customization |
|
Personally, I always run a full pacstrap and never let pacman do an update because that removes the risk for leaving behind pacnew/pacdiff. For the number and kind of packages I have it rebuilds pretty fast when using a package cache. We seem to have different expectations for the speed though. For me 1 minute is a good time for building and committing from scratch. I also don't have a need for testing the resulting system in a container since my changes are minimal. I install packages, add config files and that's it. That being said, it's fine if we have different ways to build a rootfs for different use cases. We'll just have to figure out which ways are the best fit for which use cases and then document them properly. |
|
@GrabbenD I know that you have continued to turn your demo into a generic tool, but I've done the same. I think I've come up with something that can fit both our usecases and even improves upon both of them. Here's a few key points that allowed for that:
An issue I encountered: You can find a first version of this on the branch wip/generic of the repository (currently at a10cdbb). Please share your thoughts. |
|
Here's also some benchmarks with a fully populated pacman cache and while installing # time ./run --pacman-cache cache build_rootfs_container
real 0m59.026s
user 0m30.336s
sys 0m36.108s
# time ./run --pacman-cache cache commit_rootfs_container -- -b test
real 0m17.179s
user 0m0.066s
sys 0m0.079s
# time ./run --pacman-cache cache pacstrap_rootfs_directory base amd-ucode intel-ucode linux-zen btrfs-progs
real 0m26.654s
user 0m0.087s
sys 0m0.101s
# time ./run --pacman-cache cache commit_rootfs_directory -- -b test
real 0m5.760s
user 0m0.059s
sys 0m0.079s |
|
Great job! I took a peak at your new branch and the design of your wrapper makes a lot of sense for AUR and Arch Linux Wiki! I think you should go ahead and submit it when you feel like you're happy with the codebase to get more users onboard the OSTree train :) |
|
I think this issue is very much solved now. The tool is now very generic and we have an (unofficial) guide on the arch wiki. @GrabbenD A few insights that you might be interested in:
|
|
Neat |
On a fresh archinstall system I cloned this repository and executed:
Once
podman runfinishes in./updateit complains about:Have I done something wrong?
The text was updated successfully, but these errors were encountered: