Going rootless

[haampie]

Hi all,

I've experimented a bit with runc's rootless containers + additional linux capabilities using sarus. I think rootless containers will get quite popular with the next major release of Docker, and I think it provides the perfect trade-off between flexibility and security. (rootless in this context = dropping privileged setuid before executing the container command)

The main reason to look into this is being able to build images inside of a container running in the sarus runtime, which is currently impossible (#10). It's also impossible to run package manager commands like apt-get [...] inside of an ubuntu container with sarus currently.

To solve these two problems, it seems we need a few Linux capabilities, to be precise: CAP_CHOWN, CAP_SETUID, CAP_SETGID, CAP_FOWNER, and CAP_DAC_OVERRIDE.

In the current situation we cannot have those capabilities in sarus because they are too powerful. E.g. a user can chown a root-owned file from a mounted directory to make him/herself owner, and there's probably more issues.

With user namespaces however, this is not an issue anymore. We can drop the seteuid and seteguid privileges right before executing the container command so that the container is executed as the current user, and then use namespaces with a user mapping to map the current user to root inside the container. This solves at least the obvious issues with mounting root-owned files (even when the user has CAP_CHOWN permissions):

# create a file owned by root that cannot be read by others, and verify it cannot be chown'ed when mounted inside the container
test-escalation-sarus $ echo "hi" > root-owned-file.txt
test-escalation-sarus $ sudo chown root:root root-owned-file.txt
test-escalation-sarus $ sudo chmod go-rw root-owned-file.txt
test-escalation-sarus $ sarus run --mount=type=bind,src=`pwd`,destination=/workspace -t ubuntu:18.04 /bin/bash
root@harmen-desktop:/# cd workspace/
root@harmen-desktop:/workspace# id
uid=0(root) gid=0(root) groups=0(root),65534(nogroup)
root@harmen-desktop:/workspace# cat root-owned-file.txt
cat: root-owned-file.txt: Permission denied
root@harmen-desktop:/workspace# chown harmen:harmen root-owned-file.txt
chown: changing ownership of 'root-owned-file.txt': Operation not permitted

Another great feature of namespaces is that files created as root inside of a mounted directory are in fact owned by the current user outside of the container.

The only potential issue at the moment seems to be that cgroups are not yet handled well with rootless containers, but the runc folks seem to have a workaround using cgroups v2, which is nearly finished.

Also note that it seems like a step in the direction of making the sarus not a setuid binary. Because we have to mount things, we can probably never entirely get rid of that, but with rootless containers we can at least drop the privileges before executing container commands.

I have a working example of everything here: develop...haampie:rootless not too many changes. If you want to compile it, you need to copy some hard-coded values from /etc/subuid and /etc/subgid.

With the above I can make sarus do all the things I would wish to do :) e.g.

$ sarus run ubuntu:18.04 /bin/bash -c 'apt-get update -qq && apt-get install --no-install-recommends -qq cowsay && /usr/games/cowsay "Hello from rootless sarus"'
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package perl-modules-5.26.
(Reading database ... 4046 files and directories currently installed.)
Preparing to unpack .../0-perl-modules-5.26_5.26.1-6ubuntu0.3_all.deb ...
Unpacking perl-modules-5.26 (5.26.1-6ubuntu0.3) ...
Selecting previously unselected package libgdbm5:amd64.
Preparing to unpack .../1-libgdbm5_1.14.1-6_amd64.deb ...
Unpacking libgdbm5:amd64 (1.14.1-6) ...
Selecting previously unselected package libgdbm-compat4:amd64.
Preparing to unpack .../2-libgdbm-compat4_1.14.1-6_amd64.deb ...
Unpacking libgdbm-compat4:amd64 (1.14.1-6) ...
Selecting previously unselected package libperl5.26:amd64.
Preparing to unpack .../3-libperl5.26_5.26.1-6ubuntu0.3_amd64.deb ...
Unpacking libperl5.26:amd64 (5.26.1-6ubuntu0.3) ...
Selecting previously unselected package perl.
Preparing to unpack .../4-perl_5.26.1-6ubuntu0.3_amd64.deb ...
Unpacking perl (5.26.1-6ubuntu0.3) ...
Selecting previously unselected package libtext-charwidth-perl.
Preparing to unpack .../5-libtext-charwidth-perl_0.04-7.1_amd64.deb ...
Unpacking libtext-charwidth-perl (0.04-7.1) ...
Selecting previously unselected package cowsay.
Preparing to unpack .../6-cowsay_3.03+dfsg2-4_all.deb ...
Unpacking cowsay (3.03+dfsg2-4) ...
Setting up perl-modules-5.26 (5.26.1-6ubuntu0.3) ...
Setting up libgdbm5:amd64 (1.14.1-6) ...
Setting up libtext-charwidth-perl (0.04-7.1) ...
Setting up libgdbm-compat4:amd64 (1.14.1-6) ...
Setting up libperl5.26:amd64 (5.26.1-6ubuntu0.3) ...
Setting up perl (5.26.1-6ubuntu0.3) ...
Setting up cowsay (3.03+dfsg2-4) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
 ___________________________
< Hello from rootless sarus >
 ---------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||