diff --git a/.github/workflows/cicd.yml b/.github/workflows/cicd.yml index 962d2d14c..3e9f5fd89 100644 --- a/.github/workflows/cicd.yml +++ b/.github/workflows/cicd.yml @@ -196,7 +196,6 @@ jobs: runtime: - "docker" - "podman" - # - "containerd" needs: - unit-test - staticcheck @@ -309,7 +308,6 @@ jobs: matrix: runtime: - "docker" - # - "containerd" needs: - unit-test - staticcheck diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 430f73319..49262f986 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -36,7 +36,7 @@ smoke-tests: stage: smoke-tests parallel: matrix: - - RUNTIME: [docker, containerd] + - RUNTIME: [docker] tags: - containerlab script: @@ -51,7 +51,7 @@ srl-tests: stage: integration-tests parallel: matrix: - - RUNTIME: [docker, containerd] + - RUNTIME: [docker] tags: - containerlab script: @@ -66,7 +66,7 @@ ceos-tests: stage: integration-tests parallel: matrix: - - RUNTIME: [docker, containerd] + - RUNTIME: [docker] tags: - containerlab script: diff --git a/docs/cmd/deploy.md b/docs/cmd/deploy.md index b3c63fdc9..14075c526 100644 --- a/docs/cmd/deploy.md +++ b/docs/cmd/deploy.md @@ -45,13 +45,12 @@ With `--max-workers` flag, it is possible to limit the number of concurrent work #### runtime -Containerlab nodes can be started by different runtimes, with `docker` being the default one. Besides that, containerlab has experimental support for `podman`, `containerd`, and `ignite` runtimes. +Containerlab nodes can be started by different runtimes, with `docker` being the default one. Besides that, containerlab has experimental support for `podman`, and `ignite` runtimes. A global runtime can be selected with a global `--runtime | -r` flag that will select a runtime to use. The possible value are: * `docker` - default -* `podman` - beta support -* `containerd` - experimental support +* `podman` - experimental support * `ignite` #### timeout diff --git a/docs/manual/nodes.md b/docs/manual/nodes.md index 13cf0c7e0..56de934dd 100644 --- a/docs/manual/nodes.md +++ b/docs/manual/nodes.md @@ -518,7 +518,7 @@ If you want to completely disable the networking stack on a container, you can u ### runtime -By default containerlab nodes will be started by `docker` container runtime. Besides that, containerlab has experimental support for `podman`, `containerd`, and `ignite` runtimes. +By default containerlab nodes will be started by `docker` container runtime. Besides that, containerlab has experimental support for `podman`, and `ignite` runtimes. It is possible to specify a global runtime with a global `--runtime` flag, or set the runtime on a per-node basis: @@ -526,7 +526,6 @@ Options for the runtime parameter are: - `docker` - `podman` -- `containerd` - `ignite` The default runtime can also be influenced via the `CLAB_RUNTIME` environment variable, which takes the same values as mentioned above. @@ -535,7 +534,7 @@ The default runtime can also be influenced via the `CLAB_RUNTIME` environment va # example node definition with per-node runtime definition my-node: image: alpine:3 - runtime: containerd + runtime: podman ``` ### exec diff --git a/docs/rn/0.14.0.md b/docs/rn/0.14.0.md index 304b87ece..acdba02d2 100644 --- a/docs/rn/0.14.0.md +++ b/docs/rn/0.14.0.md @@ -1,24 +1,29 @@ # Release 0.14.0 + :material-calendar: 2021-05-19 ## Container runtime support -Michael Kashin (@networkop) delivered a massive infrastructure improvement by adding the foundation that allows containerlab to run on multiple container runtimes such as `containerd`, `podman` and the likes. + +Michael Kashin (@networkop) delivered a massive infrastructure improvement by adding the foundation that allows containerlab to run on multiple container runtimes such as `podman`. For the end users of containerlab that will give more flexibility on platforms selection where containerlab can run. ## Arista `et` interfaces + Steve Ulrich (@sulrich) added support for synchronization the ENV vars passed to cEOS node and the respective container command. This makes it possible to set the cEOS specific env vars and be sure that they will be mirrored in the CMD instruction of the container. This allowed for users to, for example, overwrite the `INTFTYPE` env var to allow for using `et` interfaces with cEOS. This is documented in the [ceos kind docs](../manual/kinds/ceos.md). - ## `nodeDir` path variable + Markus Vahlenkamp (@steiler) added support for `$nodeDir` variable that you can now use in the bind paths. This is useful to simplify the configuration artifacts mapping when they are kept in the node specific directories. Read more on this in the [nodes/binds](../manual/nodes.md#binds) documentation section. ## Improved SR OS (`vr-sros`) boot procedure + With hellt/vrnetlab v0.3.1 we added a hardened process of SR OS boot sequence. Before that fix the vr-sros nodes might get problems in attaching container interfaces in time. Starting with v0.3.1 that issue is no more and vr-sros nodes will wait till the dataplane interfaces will show up in the container namespace. ## Miscellaneous + * [fixed](https://github.com/srl-labs/containerlab/commit/dbbd248591036c1e8263132e35743af2dacc6a4c) bridge attachment issue * [fixed](https://github.com/srl-labs/containerlab/commit/c1d64ff538aadcabbe1bd5f2920ed40a198177ec) docker repo naming resolution which prevented pulling certainly formatted repositories * [fixed](https://github.com/srl-labs/containerlab/commit/edc72080eab97aa809485ba4823e570cc5898e17) Arista cEOS configuration regeneration and management interface addressing. @@ -26,4 +31,5 @@ With hellt/vrnetlab v0.3.1 we added a hardened process of SR OS boot sequence. B * @networkop added support for max-workers argument for `delete` command. ## New contributors -Thanks to [@sulrich](https://github.com/sulrich), [@blinklet](https://github.com/blinklet) and [@networkop](https://github.com/networkop) for providing some of these enhancements/fixes and joining our [contributors ranks](https://github.com/srl-labs/containerlab/graphs/contributors)! \ No newline at end of file + +Thanks to [@sulrich](https://github.com/sulrich), [@blinklet](https://github.com/blinklet) and [@networkop](https://github.com/networkop) for providing some of these enhancements/fixes and joining our [contributors ranks](https://github.com/srl-labs/containerlab/graphs/contributors)! diff --git a/go.mod b/go.mod index 8e90f2e7b..59479ab89 100644 --- a/go.mod +++ b/go.mod @@ -5,8 +5,6 @@ go 1.20 require ( github.com/a8m/envsubst v1.4.2 github.com/awalterschulze/gographviz v2.0.3+incompatible - github.com/containerd/containerd v1.7.3 - github.com/containernetworking/cni v1.1.2 github.com/containernetworking/plugins v1.3.0 github.com/containers/common v0.55.3 github.com/containers/podman/v4 v4.6.1 @@ -58,7 +56,9 @@ require ( github.com/cilium/ebpf v0.10.0 // indirect github.com/cloudflare/circl v1.3.3 // indirect github.com/container-orchestrated-devices/container-device-interface v0.5.4 // indirect + github.com/containerd/containerd v1.7.3 // indirect github.com/containerd/typeurl/v2 v2.1.1 // indirect + github.com/containernetworking/cni v1.1.2 // indirect github.com/cyberphone/json-canonicalization v0.0.0-20230514072755-504adb8a8af1 // indirect github.com/go-openapi/analysis v0.21.4 // indirect github.com/go-openapi/errors v0.20.3 // indirect @@ -270,7 +270,7 @@ require ( golang.org/x/net v0.14.0 golang.org/x/oauth2 v0.9.0 // indirect golang.org/x/sync v0.3.0 // indirect - golang.org/x/text v0.12.0 + golang.org/x/text v0.12.0 // indirect golang.org/x/time v0.3.0 // indirect golang.org/x/tools v0.9.3 // indirect golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect diff --git a/runtime/all/all.go b/runtime/all/all.go index 906e1e5e0..69397b623 100644 --- a/runtime/all/all.go +++ b/runtime/all/all.go @@ -4,7 +4,6 @@ package all import ( - _ "github.com/srl-labs/containerlab/runtime/containerd" _ "github.com/srl-labs/containerlab/runtime/docker" _ "github.com/srl-labs/containerlab/runtime/ignite" ) diff --git a/runtime/all/all_with_podman.go b/runtime/all/all_with_podman.go index 7f3841868..ca43e0b20 100644 --- a/runtime/all/all_with_podman.go +++ b/runtime/all/all_with_podman.go @@ -4,7 +4,6 @@ package all import ( - _ "github.com/srl-labs/containerlab/runtime/containerd" _ "github.com/srl-labs/containerlab/runtime/docker" _ "github.com/srl-labs/containerlab/runtime/ignite" _ "github.com/srl-labs/containerlab/runtime/podman" diff --git a/runtime/containerd/containerd.go b/runtime/containerd/containerd.go deleted file mode 100644 index 5d80e75ec..000000000 --- a/runtime/containerd/containerd.go +++ /dev/null @@ -1,890 +0,0 @@ -package containerd - -import ( - "bytes" - "context" - "fmt" - "os" - "path/filepath" - "strconv" - "strings" - "syscall" - "time" - - "github.com/containerd/containerd" - "github.com/containerd/containerd/cio" - "github.com/containerd/containerd/containers" - "github.com/containerd/containerd/errdefs" - "github.com/containerd/containerd/namespaces" - "github.com/containerd/containerd/oci" - "github.com/containernetworking/cni/libcni" - current "github.com/containernetworking/cni/pkg/types/040" - "github.com/docker/go-units" - "github.com/dustin/go-humanize" - "github.com/google/shlex" - "github.com/opencontainers/runtime-spec/specs-go" - "github.com/pkg/errors" - log "github.com/sirupsen/logrus" - "github.com/srl-labs/containerlab/clab/exec" - "github.com/srl-labs/containerlab/links" - "github.com/srl-labs/containerlab/runtime" - "github.com/srl-labs/containerlab/types" - "github.com/srl-labs/containerlab/utils" - "golang.org/x/text/cases" - "golang.org/x/text/language" -) - -const ( - containerdNamespace = "clab" - cniCache = "/opt/cni/cache" - runtimeName = "containerd" - defaultTimeout = 30 * time.Second -) - -func init() { - runtime.Register(runtimeName, func() runtime.ContainerRuntime { - return &ContainerdRuntime{ - mgmt: &types.MgmtNet{}, - } - }) -} - -func (c *ContainerdRuntime) Init(opts ...runtime.RuntimeOption) error { - var err error - log.Debug("Runtime: containerd") - c.client, err = containerd.New("/run/containerd/containerd.sock") - if err != nil { - return err - } - cniPath := utils.GetCNIBinaryPath() - binaries := []string{"tuning", "bridge", "host-local"} - for _, binary := range binaries { - binary = filepath.Join(cniPath, binary) - if _, err := os.Stat(binary); err != nil { - return errors.WithMessagef(err, "CNI binaries not found. [ %s ] are required.", strings.Join(binaries, ",")) - } - } - for _, o := range opts { - o(c) - } - c.config.VerifyLinkParams = links.NewVerifyLinkParams() - return nil -} - -func (c *ContainerdRuntime) Mgmt() *types.MgmtNet { return c.mgmt } - -type ContainerdRuntime struct { - config runtime.RuntimeConfig - client *containerd.Client - mgmt *types.MgmtNet -} - -func (c *ContainerdRuntime) WithConfig(cfg *runtime.RuntimeConfig) { - c.config.Timeout = cfg.Timeout - c.config.Debug = cfg.Debug - c.config.GracefulShutdown = cfg.GracefulShutdown - if c.config.Timeout <= 0 { - c.config.Timeout = defaultTimeout - } -} - -func (c *ContainerdRuntime) WithMgmtNet(n *types.MgmtNet) { - if n.Bridge == "" { - netname := "clab" - if n.Network != "" { - netname = n.Network - } - n.Bridge = "br-" + netname - } - if n.MTU == "" { - n.MTU = "9500" - } - c.mgmt = n -} - -func (c *ContainerdRuntime) WithKeepMgmtNet() { - c.config.KeepMgmtNet = true -} -func (*ContainerdRuntime) GetName() string { return runtimeName } -func (c *ContainerdRuntime) Config() runtime.RuntimeConfig { return c.config } - -func (*ContainerdRuntime) CreateNet(_ context.Context) error { - log.Debug("CreateNet() - Not needed with containerd") - return nil -} - -func (c *ContainerdRuntime) DeleteNet(context.Context) error { - var err error - bridgename := c.mgmt.Bridge - brInUse := true - for i := 0; i < 10; i++ { - brInUse, err = utils.CheckBrInUse(bridgename) - if err != nil { - return err - } - time.Sleep(time.Millisecond * 100) - if !brInUse { - // Stop early if bridge no longer in use - // Need to wait some time, since the earlier veth deletion - // triggert from the cotnainer deletion is async and needs - // to finish. W'll have a race condition otherwise. - break - } - } - if c.config.KeepMgmtNet || brInUse { - log.Infof("Skipping deletion of bridge '%s'", bridgename) - return nil - } - return utils.DeleteLinkByName(bridgename) -} - -func (c *ContainerdRuntime) PullImage(ctx context.Context, imageName string, pullPolicy types.PullPolicyValue) error { - log.Debugf("Looking up %s container image", imageName) - ctx = namespaces.WithNamespace(ctx, containerdNamespace) - if !strings.Contains(imageName, ":") { - imageName = imageName + ":latest" - } - _, err := c.client.GetImage(ctx, imageName) - if pullPolicy == types.PullPolicyNever { - if err != nil { - // image not found but pull policy = never - return fmt.Errorf("image %s not found locally, but image-pull-policy is %s", imageName, pullPolicy) - } else { - // image present, all good - log.Debugf("Image %s present, skip pulling", imageName) - return nil - } - } - if pullPolicy == types.PullPolicyIfNotPresent && err == nil { - // pull policy == IfNotPresent and image is present - log.Debugf("Image %s present, skip pulling", imageName) - return nil - } - - n := utils.GetCanonicalImageName(imageName) - _, err = c.client.Pull(ctx, n, containerd.WithPullUnpack) - if err != nil { - return err - } - return nil -} - -func (c *ContainerdRuntime) CreateContainer(_ context.Context, _ *types.NodeConfig) (string, error) { - // this is a no-op - return "", nil -} - -func (c *ContainerdRuntime) StartContainer(ctx context.Context, _ string, node runtime.Node) (interface{}, error) { - ctx = namespaces.WithNamespace(ctx, containerdNamespace) - - nodecfg := node.Config() - - var img containerd.Image - img, err := c.client.GetImage(ctx, nodecfg.Image) - if err != nil { - // try fetching the image with canonical name - // as it might be that we pulled this image with canonical name - img, err = c.client.GetImage(ctx, utils.GetCanonicalImageName(nodecfg.Image)) - if err != nil { - return nil, err - } - } - - cmd, err := shlex.Split(nodecfg.Cmd) - if err != nil { - return nil, err - } - - mounts := make([]specs.Mount, len(nodecfg.Binds)) - - for idx, mount := range nodecfg.Binds { - s := strings.Split(mount, ":") - - m := specs.Mount{ - Source: s[0], - Destination: s[1], - Options: []string{"rbind", "rprivate"}, - } - if len(mount) == 3 { - m.Options = append(m.Options, strings.Split(s[2], ",")...) - } - mounts[idx] = m - } - - opts := []oci.SpecOpts{ - oci.WithImageConfig(img), - oci.WithEnv(utils.ConvertEnvs(nodecfg.Env)), - oci.WithHostname(nodecfg.ShortName), - WithSysctls(nodecfg.Sysctls), - oci.WithoutRunMount, - oci.WithPrivileged, - oci.WithHostLocaltime, - oci.WithNamespacedCgroup(), - oci.WithAllDevicesAllowed, - oci.WithDefaultUnixDevices, - oci.WithNewPrivileges, - } - if len(cmd) > 0 { - opts = append(opts, oci.WithProcessArgs(cmd...)) - } - if nodecfg.User != "" { - opts = append(opts, oci.WithUser(nodecfg.User)) - } - if nodecfg.Memory != "" { - mem, err := humanize.ParseBytes(nodecfg.Memory) - if err != nil { - return nil, err - } - opts = append(opts, oci.WithMemoryLimit(mem)) - } - if nodecfg.CPU != 0 { - opts = append(opts, oci.WithCPUCFS(int64(nodecfg.CPU*100000), 100000)) - } - if nodecfg.CPUSet != "" { - opts = append(opts, oci.WithCPUs(nodecfg.CPUSet)) - } - if len(mounts) > 0 { - opts = append(opts, oci.WithMounts(mounts)) - } - - var cnic *libcni.CNIConfig - var cncl *libcni.NetworkConfigList - var cnirc *libcni.RuntimeConf - - switch nodecfg.NetworkMode { - case "host": - opts = append(opts, - oci.WithHostNamespace(specs.NetworkNamespace), - oci.WithHostHostsFile, - oci.WithHostResolvconf) - case "none": - // Done! - default: - cnic, cncl, cnirc, err = cniInit(nodecfg.LongName, "eth0", c.mgmt) - if err != nil { - return nil, err - } - - // set mac if defined in node - if nodecfg.MacAddress != "" { - cnirc.CapabilityArgs["mac"] = nodecfg.MacAddress - } - - portmappings := []portMapping{} - - for contdatasl, hostdata := range nodecfg.PortBindings { - // fmt.Printf("%+v", hostdata) - // fmt.Printf("%+v", contdatasl) - for _, x := range hostdata { - hostport, err := strconv.Atoi(x.HostPort) - if err != nil { - return nil, err - } - portmappings = append(portmappings, portMapping{ - HostPort: hostport, - ContainerPort: contdatasl.Int(), Protocol: contdatasl.Proto(), - }) - } - } - if len(portmappings) > 0 { - cnirc.CapabilityArgs["portMappings"] = portmappings - } - } - - var cOpts []containerd.NewContainerOpts - cOpts = append(cOpts, - containerd.WithImage(img), - containerd.WithNewSnapshot(nodecfg.LongName+"-snapshot", img), - containerd.WithAdditionalContainerLabels(nodecfg.Labels), - containerd.WithNewSpec(opts...), - ) - - newContainer, err := c.client.NewContainer( - ctx, - nodecfg.LongName, - cOpts..., - ) - if err != nil { - return nil, err - } - - log.Debugf("Container '%s' created", nodecfg.LongName) - log.Debugf("Start container: %s", nodecfg.LongName) - - container, err := c.client.LoadContainer(ctx, nodecfg.LongName) - if err != nil { - return nil, err - } - task, err := container.NewTask(ctx, cio.LogFile("/tmp/clab/"+nodecfg.LongName+".log")) - if err != nil { - return nil, err - } - err = task.Start(ctx) - if err != nil { - return nil, err - } - log.Debugf("Container started: %s", nodecfg.LongName) - - nodecfg.NSPath, err = c.GetNSPath(ctx, nodecfg.LongName) - if err != nil { - return nil, err - } - - err = utils.LinkContainerNS(nodecfg.NSPath, nodecfg.LongName) - if err != nil { - return nil, err - } - - // if this is not a host network namespace container - // we have prepared a lot of stuff further up, which - // is now to be applied - if cnic != nil { - cnirc.NetNS = nodecfg.NSPath - res, err := cnic.AddNetworkList(ctx, cncl, cnirc) - if err != nil { - return nil, err - } - result, _ := current.NewResultFromResult(res) - - // set DNS configuration defined in topology - if nodecfg.DNS != nil { - result.DNS.Nameservers = nodecfg.DNS.Servers - result.DNS.Options = nodecfg.DNS.Options - result.DNS.Search = nodecfg.DNS.Search - } - - ipv4, ipv6, ipv4Gw := "", "", "" - ipv4nm, ipv6nm := 0, 0 - for _, ip := range result.IPs { - switch ip.Version { - case "4": - ipv4 = ip.Address.IP.String() - ipv4nm, _ = ip.Address.Mask.Size() - ipv4Gw = ip.Gateway.String() - case "6": - ipv6 = ip.Address.IP.String() - ipv6nm, _ = ip.Address.Mask.Size() - } - } - - additionalLabels := map[string]string{ - "clab.ipv4.addr": ipv4, - "clab.ipv4.netmask": strconv.Itoa(ipv4nm), - "clab.ipv6.addr": ipv6, - "clab.ipv6.netmask": strconv.Itoa(ipv6nm), - "clab.ipv4.gateway": ipv4Gw, - } - _, err = newContainer.SetLabels(ctx, additionalLabels) - if err != nil { - return nil, err - } - } - return nil, nil -} - -func (c *ContainerdRuntime) PauseContainer(ctx context.Context, cID string) error { - ctask, err := c.getContainerTask(ctx, cID) - if err != nil { - return err - } - - err = ctask.Pause(ctx) - return err -} - -func (c *ContainerdRuntime) UnpauseContainer(ctx context.Context, cID string) error { - ctask, err := c.getContainerTask(ctx, cID) - if err != nil { - return err - } - - err = ctask.Resume(ctx) - return err -} - -func cniInit(cId, ifName string, mgmtNet *types.MgmtNet) (*libcni.CNIConfig, *libcni.NetworkConfigList, *libcni.RuntimeConf, error) { - // allow overwriting cni plugin binary path via ENV var - - cnic := libcni.NewCNIConfigWithCacheDir([]string{utils.GetCNIBinaryPath()}, cniCache, nil) - - cniConfig := fmt.Sprintf(` - { - "cniVersion": "0.4.0", - "name": "clabmgmt", - "plugins": [ - { - "type": "bridge", - "bridge": "%s", - "isDefaultGateway": true, - "forceAddress": false, - "ipMasq": true, - "hairpinMode": true, - "ipam": { - "type": "host-local", - "ranges": [ - [ - { - "subnet": "%s" - } - ], - [ - { - "subnet": "%s" - } - ] - ] - } - }, - { - "type": "tuning", - "mtu": %s, - "capabilities": { - "mac": true - } - }, - { - "type": "portmap", - "capabilities": { - "portMappings": true - } - } - ] - } - `, mgmtNet.Bridge, mgmtNet.IPv4Subnet, mgmtNet.IPv6Subnet, mgmtNet.MTU) - - cncl, err := libcni.ConfListFromBytes([]byte(cniConfig)) - if err != nil { - return nil, nil, nil, err - } - - cnirc := &libcni.RuntimeConf{ - ContainerID: cId, - IfName: ifName, - // // NetNS must be set later, can just be determined after container start - // NetNS: node.NSPath, - CapabilityArgs: make(map[string]interface{}), - } - return cnic, cncl, cnirc, nil -} - -type portMapping struct { - HostPort int `json:"hostPort"` - HostIP string `json:"hostIP,omitempty"` - ContainerPort int `json:"containerPort"` - Protocol string `json:"protocol"` -} - -func WithSysctls(sysctls map[string]string) oci.SpecOpts { - return func(ctx context.Context, client oci.Client, c *containers.Container, s *specs.Spec) error { - if s.Linux == nil { - s.Linux = &specs.Linux{} - } - if s.Linux.Sysctl == nil { - s.Linux.Sysctl = make(map[string]string) - } - for k, v := range sysctls { - s.Linux.Sysctl[k] = v - } - return nil - } -} - -func (c *ContainerdRuntime) StopContainer(ctx context.Context, containername string) error { - ctask, err := c.getContainerTask(ctx, containername) - if err != nil { - return err - } - taskstatus, err := ctask.Status(ctx) - if err != nil { - return err - } - - paused := false - needsStop := true - switch taskstatus.Status { - case containerd.Created, containerd.Stopped: - needsStop = false - case containerd.Paused, containerd.Pausing: - paused = true - default: - } - - if needsStop { - // NOTE: ctx is main context so that it's ok to use for task.Wait(). - exitCh, err := ctask.Wait(ctx) - if err != nil { - return err - } - - // signal will be sent once resume is finished - if paused { - if err := ctask.Resume(ctx); err != nil { - log.Warnf("Cannot unpause container %s: %s", containername, err) - } - } - - err = ctask.Kill(ctx, syscall.SIGKILL) - if err != nil { - return err - } - - err = waitContainerStop(ctx, exitCh) - if err != nil { - return err - } - } - - existStatus, err := ctask.Delete(ctx) - if err != nil { - return err - } - log.Debugf("Container %s stopped with exit code %d", containername, existStatus.ExitCode()) - return nil -} - -func waitContainerStop(ctx context.Context, exitCh <-chan containerd.ExitStatus) error { - select { - case <-ctx.Done(): - return ctx.Err() - case status := <-exitCh: - return status.Error() - } -} - -func (c *ContainerdRuntime) getContainerTask(ctx context.Context, containername string) (containerd.Task, error) { - ctx = namespaces.WithNamespace(ctx, containerdNamespace) - cont, err := c.client.LoadContainer(ctx, containername) - if err != nil { - return nil, err - } - return cont.Task(ctx, nil) -} - -func (c *ContainerdRuntime) ListContainers(ctx context.Context, filter []*types.GenericFilter) ([]runtime.GenericContainer, error) { - log.Debug("listing containers") - ctx = namespaces.WithNamespace(ctx, containerdNamespace) - - filterstring := c.buildFilterString(filter) - containerlist, err := c.client.Containers(ctx, filterstring) - if err != nil { - return nil, err - } - - return c.produceGenericContainerList(ctx, containerlist) -} - -// GetContainer TODO this will probably not work. need to work out the exact filter format. -func (c *ContainerdRuntime) GetContainer(ctx context.Context, containerID string) (*runtime.GenericContainer, error) { - var ctr *runtime.GenericContainer - gFilter := types.GenericFilter{ - FilterType: "name", - Field: "", - Operator: "=", - Match: containerID, - } - ctrs, err := c.ListContainers(ctx, []*types.GenericFilter{&gFilter}) - if err != nil { - return ctr, err - } - if len(ctrs) != 1 { - return ctr, fmt.Errorf("found unexpected number of containers: %d", len(ctrs)) - } - return &ctrs[0], nil -} - -func (*ContainerdRuntime) buildFilterString(filter []*types.GenericFilter) string { - filterstring := "" - delim := "," - for counter, filterEntry := range filter { - isExistsOperator := false - - operator := filterEntry.Operator - switch filterEntry.Operator { - case "=": - operator = "==" - case "exists": - operator = "" - isExistsOperator = true - } - - if counter+1 == len(filter) { - delim = "" - } - - if filterEntry.FilterType == "label" { - filterstring = filterstring + "labels.\"" + filterEntry.Field + "\"" - if !isExistsOperator { - filterstring = filterstring + operator + "\"" + filterEntry.Match + "\"" + delim - } - } else if filterEntry.FilterType == "name" { - match := strings.TrimRight(strings.TrimLeft(filterEntry.Match, "^"), "$") - filterstring = "id==" + match - } - } - log.Debug("Filterstring: " + filterstring) - return filterstring -} - -// Transform docker-specific to generic container format. -func (c *ContainerdRuntime) produceGenericContainerList(ctx context.Context, - input []containerd.Container, -) ([]runtime.GenericContainer, error) { - var result []runtime.GenericContainer - - for _, i := range input { - - ctr := runtime.GenericContainer{} - - info, err := i.Info(ctx) - if err != nil { - return nil, err - } - - ctr.Names = []string{i.ID()} - ctr.ID = i.ID() - ctr.ShortID = ctr.ID - ctr.Image = info.Image - ctr.Labels = info.Labels - ctr.SetRuntime(c) - - ctr.NetworkSettings, err = extractIPInfoFromLabels(ctr.Labels) - if err != nil { - return nil, err - } - - taskfound := true - task, err := i.Task(ctx, nil) - if err != nil { - // NOTE: NotFound doesn't mean that container hasn't started. - // In docker/CRI-containerd plugin, the task will be deleted - // when it exits. So, the status will be "created" for this - // case. - if errdefs.IsNotFound(err) { - taskfound = false - } - } - if taskfound { - status, err := task.Status(ctx) - if err != nil { - return nil, fmt.Errorf("failed to retrieve task status") - } - ctr.State = string(status.Status) - - switch status.Status { - case containerd.Stopped: - ctr.Status = fmt.Sprintf("Exited (%v) %s", status.ExitStatus, timeSinceInHuman(status.ExitTime)) - case containerd.Running: - ctr.Status = "Up" - default: - ctr.Status = cases.Title(language.English).String(ctr.State) - } - - ctr.Pid = int(task.Pid()) - } else { - ctr.State = cases.Title(language.English).String(string(containerd.Unknown)) - ctr.Status = "Unknown" - ctr.Pid = -1 - } - result = append(result, ctr) - } - return result, nil -} - -func extractIPInfoFromLabels(labels map[string]string) (runtime.GenericMgmtIPs, error) { - var ipv4mask int - var ipv6mask int - var err error - if val, exists := labels["clab.ipv4.netmask"]; exists { - ipv4mask, err = strconv.Atoi(val) - if err != nil { - return runtime.GenericMgmtIPs{}, err - } - } - if val, exists := labels["clab.ipv6.netmask"]; exists { - ipv6mask, err = strconv.Atoi(val) - if err != nil { - return runtime.GenericMgmtIPs{}, err - } - } - return runtime.GenericMgmtIPs{ - IPv4addr: labels["clab.ipv4.addr"], IPv4pLen: ipv4mask, - IPv6addr: labels["clab.ipv6.addr"], IPv6pLen: ipv6mask, IPv4Gw: labels["clab.ipv4.gateway"], - IPv6Gw: labels["clab.ipv6.gateway"], - }, nil -} - -func timeSinceInHuman(since time.Time) string { - return units.HumanDuration(time.Since(since)) + " ago" -} - -func (c *ContainerdRuntime) GetNSPath(ctx context.Context, containername string) (string, error) { - ctx = namespaces.WithNamespace(ctx, containerdNamespace) - task, err := c.getContainerTask(ctx, containername) - if err != nil { - return "", err - } - return "/proc/" + strconv.Itoa(int(task.Pid())) + "/ns/net", nil -} - -func (c *ContainerdRuntime) Exec(ctx context.Context, containername string, exec *exec.ExecCmd) (*exec.ExecResult, error) { - return c.internalExec(ctx, containername, exec, false) -} - -func (c *ContainerdRuntime) ExecNotWait(ctx context.Context, containername string, exec *exec.ExecCmd) error { - _, err := c.internalExec(ctx, containername, exec, true) - return err -} - -func (c *ContainerdRuntime) internalExec(ctx context.Context, containername string, - execCmd *exec.ExecCmd, detach bool, -) (*exec.ExecResult, error) { // skipcq: RVV-A0005 - - clabExecId := "clabexec" - ctx = namespaces.WithNamespace(ctx, containerdNamespace) - container, err := c.client.LoadContainer(ctx, containername) - if err != nil { - return nil, err - } - - var stdinbuf, stdoutbuf, stderrbuf bytes.Buffer - - cio_opt := cio.WithStreams(&stdinbuf, &stdoutbuf, &stderrbuf) - ioCreator := cio.NewCreator(cio_opt) - - spec, err := container.Spec(ctx) - if err != nil { - return nil, err - } - pspec := spec.Process - pspec.Terminal = false - pspec.Args = execCmd.GetCmd() - task, err := container.Task(ctx, nil) - if err != nil { - return nil, err - } - - needToDelete := true - p, err := task.LoadProcess(ctx, clabExecId, nil) - if err != nil { - needToDelete = false - } - - if needToDelete { - log.Debugf("Deleting old process with exec-id %s", clabExecId) - _, err := p.Delete(ctx, containerd.WithProcessKill) - if err != nil { - return nil, err - } - } - - process, err := task.Exec(ctx, clabExecId, pspec, ioCreator) - // task, err := container.NewTask(ctx, cio.NewCreator(cio_opt)) - if err != nil { - return nil, err - } - - var statusC <-chan containerd.ExitStatus - if !detach { - - defer func() { - exitStatus, err := process.Delete(ctx) - if err != nil { - log.Errorf("failed to delete process: %v", err) - return - } - if exitStatus.Error() != nil { - log.Errorf("failed to delete process: %v", exitStatus.Error()) - } - }() - - statusC, err = process.Wait(ctx) - if err != nil { - return nil, err - } - } - - if err := process.Start(ctx); err != nil { - return nil, err - } - - execResult := exec.NewExecResult(execCmd) - - if !detach { - status := <-statusC - code, _, err := status.Result() - if err != nil { - return nil, err - } - - log.Infof("Exit code: %d", code) - intCode, _ := strconv.Atoi(fmt.Sprint(code)) - - execResult.SetReturnCode(intCode) - execResult.SetStdErr(stderrbuf.Bytes()) - execResult.SetStdOut(stdoutbuf.Bytes()) - } - - return execResult, nil -} - -func (c *ContainerdRuntime) DeleteContainer(ctx context.Context, containerID string) error { - log.Debugf("deleting container %s", containerID) - ctx = namespaces.WithNamespace(ctx, containerdNamespace) - - err := c.StopContainer(ctx, containerID) - if err != nil { - return err - } - - cnic, cncl, cnirc, err := cniInit(containerID, "eth0", c.mgmt) - if err != nil { - return err - } - - err = cnic.DelNetworkList(ctx, cncl, cnirc) - if err != nil { - return err - } - - cont, err := c.client.LoadContainer(ctx, containerID) - if err != nil { - return err - } - var delOpts []containerd.DeleteOpts - delOpts = append(delOpts, containerd.WithSnapshotCleanup) - - if err := cont.Delete(ctx, delOpts...); err != nil { - return err - } - - log.Debugf("successfully deleted container %s", containerID) - - return nil -} - -// GetHostsPath returns fs path to a file which is mounted as /etc/hosts into a given container -// TODO: do we need it here? currently no-op. -func (c *ContainerdRuntime) GetHostsPath(context.Context, string) (string, error) { - return "", nil -} - -// GetContainerStatus retrieves the ContainerStatus of the named container. -func (c *ContainerdRuntime) GetContainerStatus(ctx context.Context, cID string) runtime.ContainerStatus { - task, err := c.getContainerTask(ctx, cID) - if err != nil { - return runtime.NotFound - } - - status, err := task.Status(ctx) - if err != nil { - return runtime.NotFound - } - - switch status.Status { - case containerd.Running: - return runtime.Running - case containerd.Created, containerd.Paused, containerd.Pausing, containerd.Stopped, containerd.Unknown: - return runtime.Stopped - } - return runtime.NotFound -} diff --git a/schemas/clab.schema.json b/schemas/clab.schema.json index 9afe3b6bc..7fd43ce55 100644 --- a/schemas/clab.schema.json +++ b/schemas/clab.schema.json @@ -73,8 +73,8 @@ "vr-cisco_n9kv", "vr-ftosv", "vr-dell_ftosv", - "vr-aoscx", - "vr-aruba_aoscx", + "vr-aoscx", + "vr-aruba_aoscx", "linux", "bridge", "ovs-bridge", @@ -237,7 +237,6 @@ "markdownDescription": "[Runtime](https://containerlab.dev/manual/nodes/#runtime) for the node", "enum": [ "docker", - "containerd", "ignite" ] }, @@ -638,7 +637,7 @@ "vr-vsrx": { "$ref": "#/definitions/node-config" }, - "vr-aruba_aoscx": { + "vr-aruba_aoscx": { "$ref": "#/definitions/node-config" }, "vr-aoscx": { @@ -735,4 +734,4 @@ "name", "topology" ] -} +} \ No newline at end of file diff --git a/tests/01-smoke/01-basic-flow.robot b/tests/01-smoke/01-basic-flow.robot index f091eb835..fbfccddfb 100644 --- a/tests/01-smoke/01-basic-flow.robot +++ b/tests/01-smoke/01-basic-flow.robot @@ -37,15 +37,13 @@ Deploy ${lab-name} lab Set Suite Variable ${deploy-output} ${output} Ensure exec node option works - [Documentation] This tests ensures that the node's exec property that sets commands to be executed upon node deployment works. NOTE that containerd runtime is excluded because it often doesn't have one of the exec commands. To be investigated further. - Skip If '${runtime}' == 'containerd' + [Documentation] This tests ensures that the node's exec property that sets commands to be executed upon node deployment works. # ensure exec commands work Should Contain ${deploy-output} this_is_an_exec_test Should Contain ${deploy-output} ID=alpine Exec command with no filtering [Documentation] This tests ensures that when `exec` command is called without user provided filters, the command is executed on all nodes of the lab. - Skip If '${runtime}' == 'containerd' ${rc} ${output} = Run And Return Rc And Output ... sudo -E ${CLAB_BIN} --runtime ${runtime} exec -t ${CURDIR}/${lab-file} --cmd 'uname -n' Log ${output} @@ -63,7 +61,6 @@ Exec command with no filtering Exec command with filtering [Documentation] This tests ensures that when `exec` command is called with user provided filters, the command is executed ONLY on selected nodes of the lab. - Skip If '${runtime}' == 'containerd' ${rc} ${output} = Run And Return Rc And Output ... sudo -E ${CLAB_BIN} --runtime ${runtime} exec -t ${CURDIR}/${lab-file} --label clab-node-name\=l1 --cmd 'uname -n' Log ${output} @@ -77,7 +74,6 @@ Exec command with filtering Exec command with json output and filtering [Documentation] This tests ensures that when `exec` command is called with user provided filters and json output, the command is executed ONLY on selected nodes of the lab and the actual JSON is populated to stdout. - Skip If '${runtime}' == 'containerd' ${output} = Process.Run Process ... sudo -E ${CLAB_BIN} --runtime ${runtime} exec -t ${CURDIR}/${lab-file} --label clab-node-name\=l1 --format json --cmd 'cat /test.json' | jq '.[][0].stdout.containerlab' ... shell=True @@ -94,9 +90,6 @@ Inspect ${lab-name} lab Should Be Equal As Integers ${rc} 0 Define runtime exec command - IF "${runtime}" == "containerd" - Set Suite Variable ${runtime-cli-exec-cmd} sudo ctr -n clab t exec --exec-id clab - END IF "${runtime}" == "podman" Set Suite Variable ${runtime-cli-exec-cmd} sudo podman exec END @@ -243,7 +236,6 @@ Verify Hosts entries exist Verify Mem and CPU limits are set [Documentation] Checking if cpu and memory limits set for a node has been reflected in the host config - Skip If '${runtime}' == 'containerd' ${rc} ${output} = Run And Return Rc And Output ... sudo ${runtime} inspect clab-${lab-name}-l1 -f '{{.HostConfig.Memory}} {{.HostConfig.CpuQuota}}' Log ${output} diff --git a/tests/06-ext-container/01-ext-container.robot b/tests/06-ext-container/01-ext-container.robot index 070d8ddae..4b615af0e 100644 --- a/tests/06-ext-container/01-ext-container.robot +++ b/tests/06-ext-container/01-ext-container.robot @@ -21,7 +21,6 @@ Start ext-containers ... sudo ${runtime} run --name ext2 --label clab-node-name=ext2 --rm -d --cap-add NET_ADMIN alpine sleep infinity Deploy ${lab-name} lab - Skip If '${runtime}' == 'containerd' Log ${CURDIR} ${rc} ${output} = Run And Return Rc And Output ... sudo -E ${CLAB_BIN} --runtime ${runtime} deploy -t ${CURDIR}/${lab-file-name} diff --git a/tests/rf-run.sh b/tests/rf-run.sh index 44d157eee..135a72a2b 100755 --- a/tests/rf-run.sh +++ b/tests/rf-run.sh @@ -4,7 +4,7 @@ # SPDX-License-Identifier: BSD-3-Clause # arguments -# $1 - container runtime: [docker, containerd] +# $1 - container runtime: [docker, podman] # $2 - test suite to execute # set containerlab binary path to a value of CLAB_BIN env variable diff --git a/utils/networkcli.go b/utils/networkcli.go index 6d9438fd7..3d3a15d41 100644 --- a/utils/networkcli.go +++ b/utils/networkcli.go @@ -26,10 +26,6 @@ var ( "exec": "docker", "open": "exec -it", }, - "containerd": { - "exec": "ctr", - "open": "-n clab task exec -t --exec-id clab", - }, } )