alfs_docker - ALFS (Automated Linux From Scratch) with docker

参考:

reinterpretcat/lfs
lfs roadmap

alfs_docker is based on LFS-10.1-systemd for x86_64 only.

env:

ubuntu20.04/deepin v20/debian 10
docker 19.03.8

linux standards followed by LFS

note:

need a machine which support uefi because grub-install use efibootmgr, Otherwise use tag 1.0 which support bios+gpt.

setp

1. update version-check.sh

update lfs_root/scripts/version-check.sh

2. download linux kernel build config

download .config need match lfs_root/sources/linux-*.tar.xz's version.

$ export ALFSDockerRoo=~/git/alfs_docker
$ pushd /tmp
$ wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.10.17/amd64/linux-headers-5.10.17-051017-generic_5.10.17-051017.202102170631_amd64.deb
$ dpkg -x linux-headers-*.deb tmp
$ cp tmp/usr/src/linux-headers-*-generic/.config ${ALFSDockerRoo}/config

custom .config is in here, recommand

3. download all packages and patches

a tarball of all the needed files can be downloaded from one of the LFS files mirrors listed at lfs files mirror.

$ wget --continue https://mirror-hk.koddos.net/lfs/lfs-packages/lfs-packages-10.1.tar
$ wget https://mirror-hk.koddos.net/lfs/lfs-packages/SHA1SUMS
$ cat SHA1SUMS |grep "lfs-packages-10.1.tar" > lfs-sum
$ sha1sum -c lfs-sum
$ tar -xvf lfs-packages-10.1.tar
$ mv 10.1 ${ALFSDockerRoo}/lfs_root/sources
$ popd

or mirror for china: http://mirrors.ustc.edu.cn/lfs/lfs-packages/lfs-packages-10.1.tar

kernel mirror: wget https://mirrors.tuna.tsinghua.edu.cn/kernel/v5.x/linux-5.10.17.tar.xz

4. update args

glibc compatible : use host's glibc version
1. ${ALFSDockerRoo}/lfs_root/scripts/prepare/gcc.sh:--with-glibc-version=2.31 # docker base image(ubuntu 20.04)'s glibc version
kernel compatible for glibc
1. ${ALFSDockerRoo}/lfs_root/scripts/prepare/glibc.sh : --enable-kernel=4.19 # my host(debian 10) kernel version
2. ${ALFSDockerRoo}/lfs_root/scripts/build/glibc.sh : --enable-kernel=4.19

5. build docker image and build rootfs

$ ~~wget --continue --directory-prefix=lfs_root/sources https://mirrors.edge.kernel.org/pub/linux/utils/boot/syslinux/syslinux-6.03.tar.gz~~
$ wget --continue --directory-prefix=lfs_root/sources https://ftp.gnu.org/gnu/cpio/cpio-2.13.tar.gz
$ curl https://git.savannah.gnu.org/cgit/cpio.git/patch/?id=641d3f489cf6238bb916368d4ba0d9325a235afb -o lfs_root/sources/cpio-2.13.patch
$ wget https://codeload.github.com/lz4/lz4/tar.gz/v1.9.2 -O lfs_root/sources/lz4-1.9.2.tar.gz
$ sudo docker build . -t "lfs_builder"
$ cp -fv config/.config lfs_root/sources # can replace my custome .confing
$ sudo docker run --name lfs --privileged -d -it -v ${PWD}/rootfs:/mnt/lfs -v ${PWD}/lfs_root:/mnt/lfs/lfs_root --entrypoint /bin/bash lfs_builder # --privileged for mount in container
$ sudo docker exec -it lfs bash
root@401ccde8d881:/# $LFSRoot/scripts/version-check.sh   # for check env
root@401ccde8d881:/# vim ~/.bashrc                       # for MAKEFLAGS, LFS_DOCS, LFS_TEST, BackupBeforRealInstall, LFSVersion
root@401ccde8d881:/# source ~/.bash_profile
root@401ccde8d881:/# $LFSRoot/scripts/run-all.sh ~/build.log 2>&1 # start build lfs, 可以使用tmux实现nohup效果
root@401ccde8d881:/# exit

restore

# root@ddcd5d6dc98d:/# nohup $LFSRoot/scripts/restart-backup.sh > build.log 2>&1 & # in docker

note, that extended privileges are required by docker container in order to execute some commands (e.g. mount, mount -v --bind /dev $LFS/dev).

6. change lfs-rootfs-*.tar.xz -> bootable qcow2 image

qcow2 image account: root/root.

bootable qcow2 image v1 with efi is here, auth code is 1x3a. bootable qcow2 image v3 with efi is here, auth code is 4xjt, recommend.

see qemu.md, support bios/gpt and efi/gpt

use docker

$ qemu-img create -f qcow2 lfs.img 8G # qemu-img create -f <fmt> <image filename> <size of disk>
$ sudo modprobe -v nbd
$ sudo qemu-nbd -c /dev/nbd0 lfs.img
$ sudo lfs_root/scripts/gdisk.sh
$ sudo docker run --name image --privileged -d -it -v ${PWD}/lfs_root:/mnt/lfs_root --entrypoint /bin/bash lfs_builder
$ sudo docker exec -it image bash
root@401ccde8d881:/# /mnt/lfs_root/scripts/image/mount_lfs.sh # for partition
root@401ccde8d881:/# mount                                    # check mount, $LFS{,boot,boot/efi} is ok?
root@401ccde8d881:/# /mnt/lfs_root/scripts/image/run-image.sh # umount /dev/nbd0pN
root@401ccde8d881:/# vim $LFS/etc/fstab                  # set right fstab, see qemu.md
root@401ccde8d881:/# vim $LFS/boot/efi/EFI/lfs/grub.cfg  # set right /boot uuid, see qemu.md
root@401ccde8d881:/# vim $LFS/boot/grub/grub.cfg         # fix rootfs when generate grub.cfg, see qemu.md
root@401ccde8d881:/# /mnt/lfs_root/scripts/image/done.sh # umount /dev/nbd0pN
root@401ccde8d881:/# mount # check mount
$ sudo qemu-nbd -d /dev/nbd0
$ tar --zstd -cvf lfs.img.zstd lfs.img # maybe you will move it, `apt install zstd`
$ tar --zstd -xvf lfs.img.zstd
$ cp /usr/share/ovmf/OVMF.fd .
$ qemu-system-x86_64 -M q35 -pflash OVMF.fd -enable-kvm -m 1024 -hda lfs.img -vga qxl

no docker(废弃, 问题多, 见issue)

# sudo su root
# pwd
/home/chen/test/uefi/v3
# ls # lfs.img is connected to /dev/nbd0
lfs.img  lfs-rootfs-10.0-systemd.tar.xz  OVMF.fd
# export LFS=/home/chen/test/uefi/v3/lfs
# export KernelVersion=5.8.7
# cp -r ~/git/alfs_docker/lfs_root/scripts/image .
# cp ~/git/alfs_docker/lfs_root/scripts/build/prepare-vkfs-again.sh image
# image/mount_lfs.sh
# tar -xpf lfs-rootfs-10.0-systemd.tar.xz -C $LFS
# rsync -av /usr/lib/grub/x86_64-efi ${LFS}/usr/lib/grub
# cp /usr/bin/efibootmgr $LFS/usr/bin
# rsync -av /lib/x86_64-linux-gnu/{libefivar.so.1*,libefiboot.so.1*} $LFS/lib
# ln -sv $LFS/lib/modules/${KernelVersion}  /lib/modules/${KernelVersion}
# update-initramfs -c -k ${KernelVersion} -v -b $LFS/boot
# rm /lib/modules/${KernelVersion}
# prepare-vkfs-again.sh
# mkdir -pv ${LFSRoot}
# rsync -av image/qemu-image.sh ${LFS}
# rsync -av /usr/share/grub/unicode.pf2 $LFS/usr/share/grub
# chroot "$LFS" /usr/bin/env -i          \
    HOME=/root TERM="$TERM"            \
    PS1='(lfs chroot) \u:\w\$ '        \
    PATH=/bin:/usr/bin:/sbin:/usr/sbin \
    /bin/bash --login # --- into chroot
# qemu-image.sh
# exit --- exit chroot
# rm ${LFS}/qemu-image.sh
# image/done.sh
# sudo qemu-nbd -d /dev/nbd0
# exit # --- exit su
$ qemu-system-x86_64 -M q35 -pflash OVMF.fd -enable-kvm -m 1024 -hda lfs.img -vga qxl

cmd(maybe to use)

sudo efibootmgr -c -w -L "lfs" -d /dev/nbd0 -p 1 -l \\EFI\\boot\\bootx64.efi, /dev/nbd0 是EFI分区所在的磁盘, -p是EFI分区位置（默认为1），-l是启动efi文件的路径
sudo blkid
tar -tvf gcc-*.tar.xz # list files in tar.xz
watch -n 10 "ps -ef |grep bash" # watch processing when lost direct log with ssh broken
sudo docker run --rm -it ubuntu:20.04 bash # for debug dockerfile

update lfs

downlaod latest lfs from http://linuxfromscratch.org/lfs/downloads/
delete content before "Chapter 4. Final Preparations" to reduce interference for compare
compare current version with latest by Beyond Compare
update script`
build lfs again for test

log

offical log for compare: http://www.linuxfromscratch.org/lfs/build-logs/10.1/

log order not match because may be MAKEFLAGS

issue

hold "GCC plugins (GCC_PLUGINS) [Y/n/?] (NEW)" when run scripts/build/kernel.sh, and CONFIG_HAVE_GCC_PLUGINS=n does not work and make report "Restart config..."
```
  初步排查是`scripts/kconfig/conf.c`提示了`Restart config`, 待查???.

  kernel 5.8.7编译未出现该情况.
```

qemu启动image时, 在显示uefi logo后进入了uefi shell而不直接显示grub:

  原因: UEFI NVRAM启动项未设置且EFI下没有uefi启动备份用的boot文件夹(但也可能设置了boot文件夹也没用)
  解决方法: 先通过uefi shell手动选择启动项登入系统, 再执行`grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=lfs --recheck --debug`修正uefi配置, 最后重启即可.

使用rootfs在非docker环境下构建lfs, qemu image在grub到出现终端登录界面过程中出现了花屏, 但显示终端登录界面后花屏消失, 原因未知.

  起先推测:
  - 1. ~~kernel 5.8.3正常, kernel 5.8.7后不正常, kernel升级导致.~~
  - 2. 在非uefi环境下构建忽略了grub-install错误导致
  - 3. 可能是构建rootfs的docker(4.19)和构建lfs的docker(5.4)不是使用同一个版本的kernel导致.

  已验证:
  - 在docker+非uefi环境下构建的lfs不花屏
  - 在docker+uefi环境下构建的lfs也可能花屏.

  最新进展:
  问题出现在grub2引导界面至终端登录界面之间, 应该是grub获取分辨率失败导致, 理由: **qemu启动未使用`-vga`参数或使用`-vga cirrus|std`时, 在grub console输入videoinfo发现卡住; 使用参数`-vga qxl`后videoinfo正常输出**.

  解决方法: qemu追加参数`-vga qxl`即可.

  ps:
  - `qemu-system-x86_64 -enable-kvm -m 512 -kernel vmlinuz-5.8.7-lfs-10.0-systemd -initrd initrd.img-5.8.7` 未出现花屏.
  - 通过videoinfo输出所知, grub使用的分辨率由`EFI GOP driver`获取, 可以尝试更新OVMF固件解决(未测试).
  - 用了能找到的最新[OVMF固件](https://www.kraxel.org/repos/jenkins/edk2/edk2.git-ovmf-x64-0-20200515.1440.gcbccf99592.noarch.rpm), `-vga cirrus|std`时仍然花屏

  参考:
  - [How do I safely change grub2 screen resolution?](https://askubuntu.com/questions/54067/how-do-i-safely-change-grub2-screen-resolution)

使用rootfs在非docker环境下构建lfs, qemu image启动进入登入界面后发现, 输入出现卡顿, 延迟情况, 原因未知.
```
  验证: 在docker+非uefi环境下构建的lfs输入不卡顿, 因此废弃`no docker`构建方式
```

roadmap

see changelog.md

changelog

see changelog.md

upgrade lfs version

no forget to update x.xx. Stripping Again's save_lib and save_usrlib for issue 1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

alfs_docker - ALFS (Automated Linux From Scratch) with docker

setp

1. update version-check.sh

2. download linux kernel build config

3. download all packages and patches

4. update args

5. build docker image and build rootfs

restore

6. change lfs-rootfs-*.tar.xz -> bootable qcow2 image

use docker

no docker(废弃, 问题多, 见issue)

cmd(maybe to use)

update lfs

log

issue

roadmap

changelog

upgrade lfs version

Files

README.md

Latest commit

History

README.md

File metadata and controls

alfs_docker - ALFS (Automated Linux From Scratch) with docker

setp

1. update version-check.sh

2. download linux kernel build config

3. download all packages and patches

4. update args

5. build docker image and build rootfs

restore

6. change lfs-rootfs-*.tar.xz -> bootable qcow2 image

use docker

no docker(废弃, 问题多, 见issue)

cmd(maybe to use)

update lfs

log

issue

roadmap

changelog

upgrade lfs version