Use Kata containers for better isolation

[ishitatsuyuki]

Kata containers are a way to run Docker container in a VM-like fashion, which allows us to grant almost any permission with much less risk of getting exploited of a kernel bug.

This has a few downsides:

• KVM is required, which means it won't run on EC2 or similar VM providers
• cgroups are not supported yet, and maybe the memory management is harder due to the virtualizing nature

[sr229]

I think we circumvent that by using Kubernetes for Janitor. This is how exactly how Gitpod is doing it.

@jankeromnes Opnions on this.

[ishitatsuyuki]

@sr229 All Kubernetes has is a slightly more strict seccomp profile, so it doesn't add security nor loosen the debugging restriction. What are you proposing?

[sr229]

@ishitatsuyuki in Kubernetes, Kata Containers aren't subject to its Docker limitations, so using a Kubernetes runtime is more plausible than just using Docker all-in-all due to the networking limitations and a whole lot more documented.

[ishitatsuyuki]

I don't think Kubernetes is worth its complexity in this case. Not having a networked storage is the main reason, and secondly we probably don't want a big refactoring on Janitor codebase itself because it's too much work.