oci-worker: experimental support for rootless mode #419
Conversation
| # Rootless mode (Experimental) | ||
|
|
||
| Requirements: | ||
| - runc (May 30, 2018) or later |
There was a problem hiding this comment.
The upstream master is broken as of writing due to a merge conflict but will be fixed immediately in opencontainers/runc#1808
cmd/buildkitd/main.go
Outdated
| @@ -107,6 +107,13 @@ func main() { | |||
| Name: "tlscacert", | |||
| Usage: "ca certificate to verify clients", | |||
| }, | |||
| cli.BoolFlag{ | |||
| Name: "rootless", | |||
There was a problem hiding this comment.
should this be oci-worker-rootless ?
There was a problem hiding this comment.
I guess this should be global so that we can switch the default root dir and the default socket path?
There was a problem hiding this comment.
maybe both then and change the description of this field to "set all the default options to be compatible with rootless containers". Global --rootless would then behave as a shortcut that is the only flag that needs to be set for quickly running in rootless mode. But if you set all the other flags (addr, root, worker-rootless) manually then you don't even need it.
| # Rootless mode (Experimental) | ||
|
|
||
| Requirements: | ||
| - runc (May 30, 2018) or later |
There was a problem hiding this comment.
Should we update test.Dockerfile to this? Also, maybe create buildkit-runc with make binaries as executor already supports this optional override.
There was a problem hiding this comment.
Also, can we get the integration tests running using the rootless worker? Can be follow-up if there are complications.
There was a problem hiding this comment.
Probably after we have the tool for setting up newuidmap newgidmap?
The test has been substantially covered in genuinetools/img.
docs/rootless.md
Outdated
| - Some distros such as Arch Linux require `echo 1 > /proc/sys/kernel/unprivileged_ns_clone` | ||
| - To run in a Docker container with non-root `USER`, `docker run --privileged` is still required. See also Jessie's blog: https://blog.jessfraz.com/post/building-container-images-securely-on-kubernetes/ | ||
|
|
||
| Setting up rootless mode also requires some bothersome steps as follows, but we will soon have automation tool. |
There was a problem hiding this comment.
q: can this all run in buildkitd startup in the future?
There was a problem hiding this comment.
Probably it will be a separate binary so that it can be shared across different projects like
$ rootlesskit --user=suid --net=usermode buildkitdBut I'll try to implement rootlesskit so that it can be also used as a Go (>= 1.10) library that can be imported from buildkitd
There was a problem hiding this comment.
So does this setup require setuid binary or not?
There was a problem hiding this comment.
still newuidmap and newgidmap with setuid bit are required.
We could also use ptrace instead as in https://github.com/rootless-containers/runrootless , but slow and unstable. (although probably acceptable for just installing binary packages: rootless-containers/runrootless#14 )
There was a problem hiding this comment.
executor/runcexecutor/executor.go
Outdated
| // Set the oom_score_adj of our children containers to that of the current process. | ||
| b, err := ioutil.ReadFile("/proc/self/oom_score_adj") | ||
| if err != nil { | ||
| return fmt.Errorf("reading /proc/self/oom_score_adj failed: %v", err) |
There was a problem hiding this comment.
nit: errors.Errorf, line 243 as well
docs/rootless.md
Outdated
|
|
||
| Requirements: | ||
| - runc (May 30, 2018) or later | ||
| - Some distros such as Arch Linux require `echo 1 > /proc/sys/kernel/unprivileged_ns_clone` |
There was a problem hiding this comment.
isn't this unprivileged_userns_clone ?
| penguin:231072:65536 | ||
| $ grep $(whoami) /etc/subgid | ||
| penguin:231072:65536 | ||
| $ newuidmap $(cat /tmp/pid) 0 1001 1 1 231072 65536 |
There was a problem hiding this comment.
recommend installing uidmap pkg ?
AkihiroSuda
force-pushed
the
rootless
branch
7 times, most recently
from
847f240
to
4bb4e25
Compare
|
Opened containerd PR containerd/containerd#2006 Some duplicated codes can be eliminated after the containerd PR gets merged. |
| }) | ||
| defaultRoot = appdefaults.UserRoot() | ||
| defaultAddress = appdefaults.UserAddress() | ||
| appdefaults.EnsureUserAddressDir() |
There was a problem hiding this comment.
Is there an easy way to switch the snapshotter as well when overlay is not supported? Or maybe just default to naive then.
| @@ -35,6 +30,27 @@ func init() { | |||
| Name: "containerd-worker-labels", | |||
| Usage: "user-specific annotation labels (com.example.foo=bar)", | |||
| }, | |||
| } | |||
| n := "containerd-worker-rootless" | |||
There was a problem hiding this comment.
Does it make sense to add this until containerd is not actually supported?
Signed-off-by: Akihiro Suda <[email protected]>
|
updated |
|
Mergeable? |
|
Awesome! |
| @@ -225,6 +226,9 @@ export JAEGER_TRACE=0.0.0.0:6831 | |||
|
|
|||
| During development, BuildKit is tested with the version of runc that is being used by the containerd repository. Please refer to [runc.md](https://github.com/containerd/containerd/blob/v1.1.0/RUNC.md) for more information. | |||
|
|
|||
| ### Running BuildKit without root privileges | |||
|
|
|||
| Please refer to `[docs/rootless.md]`(docs/rootless.md). | |||
There was a problem hiding this comment.
@AkihiroSuda (edit) this doesn't show as link because of quotes
|
Very cool Akihiro - I think there will be a lot of interest for this. |
Signed-off-by: Akihiro Suda [email protected]
This PR allows running BuildKit without root privileges using the latest unpatched runc.
unsharedbefore runningbuildkitd. To allow subuid/subgid/setgroups (typically required byapt),newuidmapandnewgidmapcan be used. (imgautomatically does this)For further information, please refer to docs/rootless.md.
Closes #252
cc @tonistiigi @jessfraz @alexellis @cyphar