Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(src): improve containerd support. #10

Merged
merged 7 commits into from
Jan 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ load_plugins: [container]
By default, all engines are enabled on **default sockets**:
* Docker: `/var/run/docker.sock`
* Podman: `/run/podman/podman.sock` for root, + `/run/user/$uid/podman/podman.sock` for each user in the system
* Containerd: [`/run/containerd/containerd.sock`, `/run/k3s/containerd/containerd.sock`]
* Containerd: [`/run/containerd/containerd.sock`, `/run/k3s/containerd/containerd.sock`, `/run/host-containerd/containerd.sock`]
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New bottlerocket related socket.

* Cri: `/run/crio/crio.sock`

### Rules
Expand Down
14 changes: 10 additions & 4 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# TODO

Remaining unsupported fields:
* CRI:
- [ ] CniJson
* Containerd:
- [ ] Ip
- [ ] fix image related fields being empty when a docker container is spawned

- [ ] fix: docker is not able to retrieve IP because onContainerCreate is called too early
- [ ] Implement support for [`cri_extra_queries`](https://github.com/falcosecurity/libs/blob/bd0bb9baf273acc346dec881ec1d264911d74893/userspace/libsinsp/cri.hpp#L837)? It is enabled by default and moreover it does not seem needed with current code

- [x] reimplement `sinsp_container_manager::identify_category()` : https://github.com/falcosecurity/libs/blob/master/userspace/libsinsp/container.cpp#L488
- [x] finish implementing identify_category logic

Expand All @@ -15,10 +25,6 @@
* if clone/exexve syscalls are lost the plugin won't receive them and thus container_id won't be written -> this already happens in sinsp
* the latter can be fixed by letting `extract` write the foreign key in the threadtable, so that we store the thread container_id during extraction

- [ ] properly send json with all info from go-worker
- [ ] fix remaining TODOs
- [ ] fix: docker is not able to retrieve IP because onContainerCreate is called too early
- [ ] send healthprobe related infos

- [x] Implement containerd matcher
- [x] Implement some unit tests taken from sinsp
2 changes: 1 addition & 1 deletion go-worker/main_exe.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ func main() {
initCfg := `
{
"label_max_len": 100,
"with_size": false,
"with_size": true,
"engines": {
"containerd":{
"enabled":true,
Expand Down
83 changes: 54 additions & 29 deletions go-worker/pkg/container/containerd.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"github.com/containerd/containerd/v2/pkg/oci"
"github.com/containerd/typeurl/v2"
"github.com/opencontainers/runtime-spec/specs-go"
"strconv"
"strings"
"sync"
)
Expand Down Expand Up @@ -41,7 +42,7 @@ func (c *containerdEngine) ctrToInfo(namespacedContext context.Context, containe
spec, err := container.Spec(namespacedContext)
if err != nil {
spec = &oci.Spec{
Process: &specs.Process{NoNewPrivileges: true},
Process: &specs.Process{},
Mounts: nil,
}
}
Expand Down Expand Up @@ -80,59 +81,63 @@ func (c *containerdEngine) ctrToInfo(namespacedContext context.Context, containe
}
}

// Mounts related - TODO double check
// Mounts related
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

mounts := make([]event.Mount, 0)
for _, m := range spec.Mounts {
readOnly := false
for _, path := range spec.Linux.ReadonlyPaths {
if path == m.Destination {
mode := ""

for _, opt := range m.Options {
if opt == "ro" {
readOnly = true
break
} else if strings.HasPrefix(opt, "mode=") {
mode = strings.TrimPrefix(opt, "mode=")
}
}
mounts = append(mounts, event.Mount{
Source: m.Source,
Destination: m.Destination,
Mode: mode,
RW: !readOnly,
Propagation: spec.Linux.RootfsPropagation,
})
}

// Namespace related - FIXME
// Namespace related - see oci.WithHostNamespace() impl: it just removes the namespace from the list
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

var (
hostIPC bool
hostPID bool
hostNetwork bool
hostIPC = true
hostPID = true
hostNetwork = true
)
if spec.Linux != nil {
for _, ns := range spec.Linux.Namespaces {
if ns.Type == specs.PIDNamespace {
hostPID = ns.Path == "host"
hostPID = false
}
if ns.Type == specs.NetworkNamespace {
hostNetwork = ns.Path == "host"
hostNetwork = false
}
if ns.Type == specs.IPCNamespace {
hostIPC = ns.Path == "host"
hostIPC = false
}
}
}

// Image related - TODO
var size int64 = -1
// Image related
// FIXME: with docker, everything is empty because container.Image below does not return any image.
var (
imageName string
imageDigest string
imageRepo string
imageTag string
imageSize int64 = -1
)
image, _ := container.Image(context.TODO())
// TODO this is an extra API call; shall we move it behing config.GetWithSize()?
// Or rename `with_size` option with something more generic like `full_info`?
image, _ := container.Image(namespacedContext)
if image != nil {
imageName = image.Name()
imgConfig, _ := image.Config(context.TODO())
imageDigest = imgConfig.Digest.String()
imageDigest = image.Target().Digest.String()
if config.GetWithSize() {
size, _ = image.Size(context.TODO())
imageSize = image.Target().Size
Comment on lines +138 to +140
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

}
}
imageRepoTag := strings.Split(info.Image, ":")
Expand Down Expand Up @@ -166,18 +171,38 @@ func (c *containerdEngine) ctrToInfo(namespacedContext context.Context, containe
}
}

// Check for privileged:
// see https://github.com/containerd/containerd/blob/main/pkg/oci/spec_opts.go#L1295
privileged := true
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed; a bit ugly but couldn't find a better solution.

if spec.Linux != nil && spec.Process != nil &&
spec.Linux.MaskedPaths == nil && spec.Linux.ReadonlyPaths == nil &&
spec.Process.SelinuxLabel == "" &&
(spec.Process.ApparmorProfile == "" || spec.Process.ApparmorProfile == "unconfined") &&
spec.Linux.Seccomp == nil {
for _, m := range spec.Mounts {
if m.Type == "sysfs" || m.Type == "cgroup" {
for _, o := range m.Options {
if o == "ro" {
privileged = false
break
}
}
}
}
} else {
privileged = false
}

return event.Info{
Container: event.Container{
Type: typeContainerd.ToCTValue(),
ID: container.ID()[:shortIDLength],
Name: container.ID()[:shortIDLength],
ID: shortContainerID(container.ID()),
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since containerd supports container_id of any length, properly cut short ID to 12 only if it is longer than 12, to avoid crashes

Name: shortContainerID(container.ID()),
Image: info.Image,
ImageDigest: imageDigest, // FIXME, empty
ImageID: imageName, // FIXME, empty
ImageDigest: imageDigest,
ImageRepo: imageRepo,
ImageTag: imageTag,
User: spec.Process.User.Username,
CniJson: "", // TODO
User: strconv.FormatUint(uint64(spec.Process.User.UID), 10),
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed; we use uid.

CPUPeriod: int64(cpuPeriod),
CPUQuota: cpuQuota,
CPUShares: int64(cpuShares),
Expand All @@ -194,10 +219,10 @@ func (c *containerdEngine) ctrToInfo(namespacedContext context.Context, containe
MemoryLimit: memoryLimit,
SwapLimit: swapLimit,
PodSandboxID: info.SandboxID,
Privileged: false, // TODO implement
Privileged: privileged,
PodSandboxLabels: podSandboxLabels,
Mounts: mounts,
Size: size,
Size: imageSize,
},
}
}
Expand Down Expand Up @@ -266,7 +291,7 @@ func (c *containerdEngine) Listen(ctx context.Context, wg *sync.WaitGroup) (<-ch
info = event.Info{
Container: event.Container{
Type: typeContainerd.ToCTValue(),
ID: id[:shortIDLength],
ID: shortContainerID(id),
FullID: id,
Image: image,
},
Expand Down
55 changes: 34 additions & 21 deletions go-worker/pkg/container/containerd_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,25 +41,33 @@ func TestContainerd(t *testing.T) {
id := uuid.New()
var cpuQuota int64 = 2000
ctr, err := client.NewContainer(namespacedCtx, id.String(), containerd.WithImageName("docker.io/library/alpine:3.20.3"),
containerd.WithSpec(&oci.Spec{
Process: &specs.Process{
User: specs.User{
UID: 0,
GID: 0,
Umask: nil,
AdditionalGids: nil,
Username: "testuser",
containerd.WithSpec(
&oci.Spec{
Process: &specs.Process{
User: specs.User{
UID: 0,
GID: 0,
},
},
},
Linux: &specs.Linux{
Resources: &specs.LinuxResources{
CPU: &specs.LinuxCPU{
Quota: &cpuQuota,
Cpus: "0-1",
Linux: &specs.Linux{
Resources: &specs.LinuxResources{
CPU: &specs.LinuxCPU{
Quota: &cpuQuota,
Cpus: "0-1",
},
},
},
},
}))
oci.WithHostNamespace(specs.NetworkNamespace),
oci.WithLinuxNamespace(specs.LinuxNamespace{
Type: specs.PIDNamespace,
Path: "/proc/foo",
}),
oci.WithLinuxNamespace(specs.LinuxNamespace{
Type: specs.IPCNamespace,
Path: "/proc/foo",
}),
oci.WithPrivileged))
assert.NoError(t, err)

events, err := engine.List(context.Background())
Expand All @@ -69,22 +77,27 @@ func TestContainerd(t *testing.T) {
Info: event.Info{
Container: event.Container{
Type: typeContainerd.ToCTValue(),
ID: ctr.ID()[:shortIDLength],
ID: shortContainerID(ctr.ID()),
Name: shortContainerID(ctr.ID()),
Image: "docker.io/library/alpine:3.20.3",
ImageRepo: "docker.io/library/alpine",
ImageTag: "3.20.3",
ImageDigest: "sha256:1e42bbe2508154c9126d48c2b8a75420c3544343bf86fd041fb7527e017a4b4a",
CPUPeriod: defaultCpuPeriod,
CPUQuota: cpuQuota,
CPUShares: defaultCpuShares,
CPUSetCPUCount: 2, // 0-1
Env: nil, // TODO
CPUSetCPUCount: 2, // 0-1
Env: nil,
FullID: ctr.ID(),
HostIPC: false,
HostPID: false,
HostNetwork: true,
Labels: map[string]string{},
PodSandboxID: "",
Privileged: false, // TODO
Privileged: true,
PodSandboxLabels: nil,
Mounts: []event.Mount{},
User: "testuser",
User: "0",
Size: -1,
}},
IsCreate: true,
Expand Down Expand Up @@ -120,7 +133,7 @@ func TestContainerd(t *testing.T) {
Info: event.Info{
Container: event.Container{
Type: typeContainerd.ToCTValue(),
ID: ctr.ID()[:shortIDLength],
ID: shortContainerID(ctr.ID()),
FullID: ctr.ID(),
}},
IsCreate: false,
Expand Down
Loading