Get rid of debootstrap

[schaefi]

Replace debootstrap with an apt-get based pre-download of packages followed by a dpkg-deb extraction.
This Fixes #2599

[schaefi]

@Conan-Kudo I'm unsure about the usr-merge stuff and did not yet add that. Is this manual usrmerge really needed ? Maybe @DaanDeMeyer could provide a hint ?

Thanks

[Conan-Kudo]

Note this has the consequence of breaking Debian builds on openSUSE hosts since apt is not available there.

[Conan-Kudo]

@Conan-Kudo I'm unsure about the usr-merge stuff and did not yet add that. Is this manual usrmerge really needed ? Maybe @DaanDeMeyer could provide a hint ?

We do need it as that package is what forces usrmerge for Debian systems.

[DaanDeMeyer]

Note that what mkosi does might be too extreme, I don't think all the per architecture symlinks are actually required, I just never bothered with dropping them again.

[schaefi]

Note this has the consequence of breaking Debian builds on openSUSE hosts since apt is not available there.

I think that never worked because if you call apt-get on suse you get

apt-get --version
zypper 1.14.59

How funny, so I don't care.

[schaefi]

Note that what mkosi does might be too extreme, I don't think all the per architecture symlinks are actually required, I just never bothered with dropping them again.

yeah I also thought it might be too drastic :) I think in the kiwi case we try first without it and see where it causes issues and if it does try to find a solution...

[schaefi]

I set the don't merge label as I want to create a Staging build and see if our debian integration tests still builds

[Conan-Kudo]

Note that what mkosi does might be too extreme, I don't think all the per architecture symlinks are actually required, I just never bothered with dropping them again.

yeah I also thought it might be too drastic :) I think in the kiwi case we try first without it and see where it causes issues and if it does try to find a solution...

Do we have debian tests on all arches?

[schaefi]

we have

• Testing_arm:ubuntu/
• Testing_x86:ubuntu
• Testing_x86:debian

no power no s390

[Conan-Kudo]

That's probably fine for this...

[Conan-Kudo]

Do we tell apt to disable signature checking? Because if we don't, it probably isn't going to work.

[schaefi]

@Conan-Kudo @DaanDeMeyer ok I found why it does not work on an obs worker. So look at this

[   42s] [ DEBUG   ]: 15:16:49 | EXEC: [apt-get -q -c /usr/src/packages/KIWI-oem/build/image-root/kiwi_apt.conf -y --no-install-recomm
ends install -oDebug::pkgDPkgPm=1 -oDPkg::Pre-Install-Pkgs::=cat >/var/tmp/kiwi_debs_2dq76tkw ?essential ?exact-name(usr-is-merged) ap
t]
[   42s] [ DEBUG   ]: 15:16:49 | Apt download: Reading package lists...
[   42s] Building dependency tree...
[   42s] apt is already the newest version (2.6.1).
[   42s] base-files is already the newest version (12.4+deb12u6).
[   42s] dpkg is already the newest version (1.21.22).
[   42s] base-passwd is already the newest version (3.6.1).
[   42s] bash is already the newest version (5.2.15-2+b7).
[   42s] debianutils is already the newest version (5.7-0.5~deb12u1).
[   42s] bsdutils is already the newest version (1:2.38.1-5+deb12u1).
[   42s] coreutils is already the newest version (9.1-1).
[   42s] dash is already the newest version (0.5.12-2).
[   42s] init-system-helpers is already the newest version (1.65.2).
[   42s] diffutils is already the newest version (1:3.8-4).
[   42s] tar is already the newest version (1.34+dfsg-1.2+deb12u1).
[   42s] findutils is already the newest version (4.9.0-4).
[   42s] grep is already the newest version (3.8-5).
[   42s] gzip is already the newest version (1.12-1).
[   42s] hostname is already the newest version (3.23+nmu1).
[   42s] libc-bin is already the newest version (2.36-9+deb12u7).
[   42s] perl-base is already the newest version (5.36.0-7+deb12u1).
[   42s] login is already the newest version (1:4.13+dfsg1-1+b1).
[   42s] ncurses-base is already the newest version (6.4-4).
[   42s] ncurses-bin is already the newest version (6.4-4).
[   42s] sed is already the newest version (4.9-1).
[   42s] sysvinit-utils is already the newest version (3.06-4).
[   42s] util-linux is already the newest version (2.38.1-5+deb12u1).
[   42s] The following NEW packages will be installed:
[   42s]   usr-is-merged
[   42s] 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
[   42s] Need to get 0 B/5616 B of archives.
[   42s] After this operation, 13.3 kB of additional disk space will be used.

This is funny. On the worker itself the packages that we want for bootstrap are already installed. This leads to a number of is already the newest version and apt does not download them. Resulting in a Pre-Install-Pkgs list that consists out of one entry :) From that one entry we unpack it into the new root tree and of course that is by far not enough :)

So can the download be forced even if the system already has that package installed ?

Thanks much

[DaanDeMeyer]

Add --reinstall?

[Conan-Kudo]

That means it's using the host dpkg status file, is there a way to make apt not do that?

[DaanDeMeyer]

Sure apt has options for everything, should be somewhere in the man pages

[schaefi]

Dir::State::status solved it

[schaefi]

This is now working:

• https://build.opensuse.org/project/show/Virtualization:Appliances:Images:Testing_x86:debian

[schaefi]

The ubuntu integration needs some adaptions such that apt-get after bootstrap succeeds. This is now done. The integration tests for x86_64 and aarch64 succeeds here:

• https://build.opensuse.org/project/show/Virtualization:Appliances:Images:Testing_x86:ubuntu
• https://build.opensuse.org/project/show/Virtualization:Appliances:Images:Testing_arm:ubuntu

[schaefi]

<package name="libpam-runtime"/>

was needed to fix the pam issue. Its script code has to run such that we have to install this package in the proper system installation because just unpacking it in the bootstrap is not enough and it is not picked up by the dependency solver later. Thus needs to be explicitly added to the package list and that made the OS to function properly