Official docker image

[ghost]

Please configure docker image creation in CI/build machine after which it upload the official image to docker hub for each release.

Our current option is some unofficial/unmaintained channel docker run -it jprjr/sabotagelinux sh and nothing work in that box (butch update && butch install curl fails).

https://hub.docker.com/r/jprjr/sabotagelinux/

while official one can go under https://hub.docker.com/r/library/

[ghost]

Why would people want to run sabotage linux in docker? Sabotage doesn't have any release cycle.
Its made with self-hosting in mind, which is what people ususally dont want inside docker containers.

Judging from the description, jprjr/docker-misc is an personal experiment and shouldn't be used in production, probably.

[ghost]

@nero, that makes the build system guys lives easier. For example you can build and test your binaries on multiple musl-based systems without paying the cost of setting up an extra VM/physical hardware.

Docker is one command away and you can spin up [m]any distros, build bins, run tests and shut it down. You can even hook it up in TravisCI without paying cost of setting up your own CI server, using Jenkins for example.

cc @rofl0r

[rofl0r]

i can see it being useful. we recently had a generous donor sponsoring a second build server so i may dedicate some of its cycles to building a docker image.
@kasper3 please provide info on how to make an "official" image and getting upstreamed.

[ghost]

https://docs.docker.com/docker-hub/official_repos/#how-do-i-create-a-new-official-repository

@rofl0r, it will require sending PR to https://github.com/docker-library/official-images
Example of Alpine, https://github.com/gliderlabs/docker-alpine which produces https://github.com/docker-library/official-images/blob/master/library/alpine.

[rofl0r]

if i provide a rootfs tarball, can you help out in getting the docker scripts ?

[ghost]

Sure i will give it a shot 👍

[ghost]

Can we seek sponsorship from fastly like https://github.com/gliderlabs/docker-alpine#sponsors?

[rofl0r]

well, maybe if you ask nicely ? but i guess we should first getting step one done...

[ghost]

I tried to build it:

mkdir sabotage-build ; cd $_
curl -o rootfs.tar.xz http://ftp.barfooze.de/pub/sabotage/sabotage-1.1.14-x86_64-rootfs-core-2f51c8d0.tar.xz

cat > Dockerfile <<-"EOF"
FROM scratch
ADD rootfs.tar.xz /
CMD ["/bin/sh"]
EOF

docker build -t sabotage-builder .
docker run -it sabotage-builder

build succeeds, but docker run throws:

Error response from daemon: OCI runtime create failed: container_linux.go:296: starting container process caused "exec: "/bin/sh": stat /bin/sh: no such file or directory": unknown.

any ideas how to produce the working docker image from rootfs tarball?

cc @tianon, @andyshinn for help :)

[andyshinn]

The tarball has an extra path at root:

$ tar tf rootfs.tar.xz | head -n 5
sabotage-1.1.14-rootfs-core-2f51c8d0/
sabotage-1.1.14-rootfs-core-2f51c8d0/sbin
sabotage-1.1.14-rootfs-core-2f51c8d0/etc/
sabotage-1.1.14-rootfs-core-2f51c8d0/etc/crypttab
sabotage-1.1.14-rootfs-core-2f51c8d0/etc/wpa_supplicant.conf

You likely want to go into sabotage-1.1.14-rootfs-core-2f51c8d0 before tar like:

tar -cJf mytarfile.tar.xz -C sabotage-1.1.14-rootfs-core-2f51c8d0 .

[ghost]

@andyshinn, thanks. I am getting:

tar: can't change directory to 'sabotage-1.1.14-rootfs-core-2f51c8d0': No such file or directory

[ghost]

I tried with more selfcontained approach

cat > Dockerfile <<-"EOF"
FROM scratch
ADD http://foss.aueb.gr/mirrors/linux/sabotage/sabotage-1.1.14-x86_64-rootfs-core-2f51c8d0.tar.xz /
CMD ["/bin/sh"]
EOF

docker build -t sabotage-builder .
docker run -it sabotage-builder

and

cat > Dockerfile <<-"EOF"
FROM scratch
ADD http://foss.aueb.gr/mirrors/linux/sabotage/sabotage-1.1.14-x86_64-rootfs-core-2f51c8d0.tar.xz /
CMD ["/sabotage-1.1.14-rootfs-core-2f51c8d0/bin/sh"]
EOF

docker build -t sabotage-builder .
docker run -it sabotage-builder

same error. :(

[andyshinn]

The tarball is built wrong. It needs to be rebuilt without the sabotage-1.1.14-rootfs-core-2f51c8d0 prefix. If you extract it, then you can rebuild it with my previous command, then use that new tarball with the ADD.

[ghost]

man tar -> http://xkcd.com/1168/

[rofl0r]

the tarball is not "built wrong", it was built for general purpose, i.e. not specifically for docker.
i doubt docker is so inflexible it doesnt allow you to mv a directory's contents before booting it...

[ghost]

lets find out https://github.com/moby/moby/issues/36605. btw, scratch docker support limited commands. we can create an .sh which untar->tar the whole thing without the encapsulating directory as a last resort, but first try to find out if there is an elegant way.

[ghost]

@rofl0r, based on the comment https://github.com/moby/moby/issues/36605#issuecomment-373794369, would it be possible to permanently produce a tarball without a wrapper directory, so to avoid mismatch with other distro's tarballs? Perhaps an additional tarball, incurring ~40MBs on the servers (if changing the current rootfs tarball format is considered as a breaking change).

[tianon]

That, or simply have whatever process is going to be downloading that tarball to commit it to this repository perform the necessary repacking.

[rofl0r]

that may certainly be possible, but as you may recall my suggestion was you use the previous rootfs tarball for testing while i'm working on a new version. you also said you had it working at some point. so, what came out of that ?

[ghost]

the previous rootfs download->untar->re-tar way works:

# sitting in ubuntu

# setup
apt update
apt install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common < /dev/null
curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
apt update
apt install -y docker-ce < /dev/null
service docker start

# create an empty directory
mkdir test; cd $_

# fetch and curate archive
curl -O http://foss.aueb.gr/mirrors/linux/sabotage/sabotage-1.1.14-x86_64-rootfs-core-2f51c8d0.tar.xz
mkdir temp
tar -xf sabotage-1.1.14-x86_64-rootfs-core-2f51c8d0.tar.xz --strip-components=1 -C temp
tar -cJf rootfs.tar.xz -C temp .
rm -rf temp

# create a docker file
cat > Dockerfile <<-"EOF"
FROM scratch
ADD rootfs.tar.xz /
CMD ["/bin/sh"]
EOF

# docker build
docker build -t sabotage-builder .

# then run the built docker, enter sabotage
docker run -it sabotage-builder

btw, etc /proc/version in sabotage shows Linux version 4.9.60-linuxkit-aufs (root@4a42478ffb9a) (gcc version 6.3.0 (Alpine 6.3.0) ) #1 SMP Mon Nov 6 16:00:12 UTC 2017. alpine instead of sabotage..

[rofl0r]

i don't where docker picks up the kernel, but it seems to use one that was compiled on alpine. the rootfs does not contain a kernel.

[tianon]

The way containers work, the kernel comes from the host, so it'll never report "sabotage" unless you're running Docker on a native sabotage host.

[rofl0r]

http://ftp.barfooze.de/pub/sabotage/sabotage-1.1.19-x86_64-rootfs-stage2-e0b6285a.tar.xz

sha512 4b653382b366afdc0a9f0954cfd7b532e1c1e927fdc47464a9f306f6cbdcf73399a267eca8a8876bb4a5f684403d31897124c70d4a88e638aa9267ddcd389fe4

it's possible for anyone to reproduce the exact same packages inside by following these steps:

git checkout e0b6285a
./build-stage0
./enter-chroot
# inside chroot
butch install stage2
butch rebuild libz gmp mpfr mpc libelf gcc630 musl jobflow patch busybox binutils make sabotage-core join

the builds can be verified using

for p in `butch list | grep -v 'stage*' ` ; do echo -ne "$p " ; butch checksum "$p" ; done | tee pkgs.hash

[rofl0r]

FTR here's the output of the above butch checksum command:

services filelist /src/filelists/services.txt does not exist, probably the program is not installed
kernel-headers 715-760634a301d391caae6f35587741aee6081f9a3b8c2f769081f3805de934c78a57f9f11b791b2fd502ebc2f5b68eabc2154df339bbd31323e392cc9e974e98f3
kbd 34-31ba6c88efa74ccc717a27966e77d528ccb244f74cfe6d56a839098c6cc8c5f86d4febbf3c5b4b80cbf0f3aec765598f01929c5a9fac194d55f6de97a2295dbd
man 2-fd8c24ade6d75af9110396e50f410416b3f4672fb5d1b20e8c8ef352bd10fcd8252ca30f35e018856e369da5b298ba5257e0f2d181100069ec9796cf7f899598
m4 6-0e4216703aa908d30e4409da7d357a2fecebed0ac1b4534f9a7467a57a8937c3fd4b80d1ff4542af5924c278bc3db4901380f8628b2d471be4e842f09f2664c6
zlib filelist /src/filelists/zlib.txt does not exist, probably the program is not installed
gcc6 filelist /src/filelists/gcc6.txt does not exist, probably the program is not installed
dynamic-toolchain filelist /src/filelists/dynamic-toolchain.txt does not exist, probably the program is not installed
9base 226-482a540efbddc47e7514f650d9a42462083de51896a9b401858a8269fc82b7d81b73eb313837b517e49b2f20776a8f4edbe9ca77c1d0674ec9321b1e7bbfaee8
pkgconf 3-bac248f5f679e0823b028f1acf65e9a570f9daad90b64b8fefd3da3dfd79d050ad23034cc3ac541e5ad90d80fcd5b3dac33666cd70b6f60e2626e137bd6affb9
libblkid 4-b46a572d1eae7762efdac29a01e86e499e2c09ed81c3f2db3d6848e51603b575ed40ef1b479ba4be4610b11f3883b154c8f2c127662128715406396593819d35
e2fsprogs 112-d0ae8c1e194425f4b17ceb6469ec1e9cb1cd7bdb52b4080767d7558276ecbf4214704ccc1581e355ca7bc34f45bc3287f96242500f89f7234e28ce88b3c351e7
libressl 1567-f02e9f808781994f222b05c97f9eff54482def77da306c317f38e18313558e10c8f36e1fd75d7781c42011cf0daa9a3a03ab7caf044310869edd1a58af550791
ca-certificates 663-1ef6147ba29cfb83d26613082a4429ed5070f48ae5cff81d02e70fb57ef84a4c980f28a67a0cde1b081a1cffabf17eae3ae508bf25f1a13096da44d7105bc09d
libz 9-86ec79369b713f257d78edfb8aa9953634234d2ffdfe20f747f58f2efa5b3dbf59b386cf416da34bbf8ae975261c13230acf05bb3b9f902bd0d0452784d23da3
sabotage-core 42-7bae49e80ab3890d15621918c8f9756c599063e714706fecc7add3641f491e441098ae8036f99e93c58eebf2e351574e80b908734f5075fcc7c9e27c6e627a57
join 1-56bf20c139e8812e41ca84f45fe71364d80d76fea5a01c9d5a2c9f66014a85b24c8d760e26078c849c2920877fec03b2b91f57c70e21b001b5b6ea663d98c106
gmp 7-cbd75286622b44796ec04b402847a2f3fb525996e5db294234c004a428311035b501b4ff25b8dbc77d385c531cee09e8f84ce661f25157e5d4503d7b5bd1a0ad
mpfr 19-677a08a84a608925bc316d6eb323a748d87fe41d2a255f1fbdd21860709605506fa705cc4def733cfe7d7543099fa5acf523c4877edfa6807a5e0c7c695c0f74
mpc 6-8fcbe9fd4a0178514c5880051919f839ce06c4805db7a33851fe640156c00487fa6a25815a5f1504687be34467d9dccd7d8f0c4e1acf18f6293ad0a55bc92e12
libelf 3-3e9381dcbf19c3a353fb22705c96c23684edcc3db3593596df92a4e994cc80b822a3793bbbe2e2b451f42668b42c2dc53084f79e4fa7460bb319e0c2a9772202
gcc630 950-a35215996947a9a27d59b1e1b77a03ff4873fc25ea45e919dcdd78b75ba4096bae6eabfde9601ecf8c9f67757ebf1d14ea2609061e47967236ed281e6bbd1680
musl 230-7a51137987a52faa2b805e635f0534a01398d70a8029a04a9832076e9f018dd9e2c34ea0889bdac3f53836af4965444b50cc588acea64de1247ea77e03d14b78
jobflow 1-c3de1bf7ce3a6e2b02ad51f20ef2118ee6297c03dd9656b58a2b0b59b8dbddf148b626f16ed6d52eab796e89ceacf6de11e07f5df78b8487fb9cea54d7de608a
patch 3-466a8fe9874a9c19a37d1b303d22899fca821ba6b0a8ccb51d9892a69d22e6734c4f7163073425001563561fca47973c49c3f7ab58922d57baa9f3a28e864402
busybox 324-d32a0c92af465d17b4b9602de32c40c29e61fd59c2436a88a3dc2f3560250371cc5f023aed8a6e0a475badca48529754f07320a9f1643ecfb08fc87f777d9609
binutils 116-6abe6de9cd6597ac0433a4b2bf5f9459870b1c36824fae4b9665b7de2d920989c717ad049c0208bad223c8412f6d5f189d4dfd21ae900b0b3a172d0dff270d44
make 5-548ba92bfbe7210c9d3495b6702c76562f4027d7cb6e10f2f961b7e9f8b427c64dba42733918baa578d9e1f8ffde0ebb19175c01d14363b63d9a86c84c9e8324

[ghost]

Pushed a commit to https://github.com/sabotage-linux/docker.

Size comparison ⚖️

ubuntu:~/docker/builder$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
sabotage-builder    latest              c92724d8fc6d        15 minutes ago      158MB
debian              latest              2b98c9851a37        11 days ago         100MB
ubuntu              latest              f975c5035748        2 weeks ago         112MB
alpine              latest              3fd9065eaf02        2 months ago        4.15MB

Compared to alpine, sabotage box is 38x bigger. There must be lots of stuff that can be dropped from tarball.

[rofl0r]

thanks. sabotage does not (yet) feature binary packages, so the only way to install additional packages is by supplying a full toolchain (equivalent to debian base + build-essentials). that means sabotage install without toolchain would be worthless.
unfortunately, GCC 6.x is quite big (>100 MB).
a (useless) stripped down sabotage system would be around 800KB, the size of the busybox binary.

[montao]

could you please add some instructions how to start and use the docker image? i could run it before but i lost it. i can confirm that it did work, but now i don't know how to do it again.

it was useful in a way to try sabotage linux without dedicating an entirety.

[ghost]

Based on https://docs.docker.com/docker-hub/official_repos/#how-do-i-create-a-new-official-repository, I think the next (remaining) work is related to documentation for 'official docker image'. We can start a PR in docker-library/official-images repo (e.g. with title [wip] Add Sabotage Linux docker) simultaneously. It would make it very simple to obtain sabotage via docker.

could you please add instructions

Instructions added https://github.com/sabotage-linux/docker/blob/master/README.md