Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch from shellscripts-based CI to Go-based CI #310

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ktock
Copy link
Member

@ktock ktock commented Apr 28, 2021

Recently, it is becoming harder and harder to maintain our shellscript-based CI (stored under /script). Let's switch to golang-based tests. This will help us to write test codes in a type-safe & modular manner, leveraging rich language features of Go.

The core package that enables this switch is util/dockershell. This provides exec.Cmd-like API + some useful abstraction over docker, docker-compose and kind. This package allows us to handle multiple shell environments (e.g. node + registry) in object-oriented manners, leveraging Go's rich features including standard lib, goroutine, duck-typing, etc.

Based on that pkg, this commit has converted shellscript-based CI into the Go-based testing scripts.

  • /integration pkg contains all integratoin tests previously resided under /script.
  • /benchmark pkg contains hello-bench-based benchmark previously resided under /script/benchmark.

Additional value of this is that this allows contributors to easily run our CI on any hosts, even on their laptop as long as docker, docker-compose and kind are installed.

About util/dockershell

This pkg provides intuitive shellscript-like API with containers, aiming at tests use-case.

The core of this pkg is util/dockershell/exec. This enables exec.New(containerName).Command(args...).Run() which runs command args... inside a container named containerName. Here, exec.New(containerName) gives us an execution environment *exec.Exec where we can run any commands inside containerName.

Based on this pkg, there are the following wrappers/helpers that allows us to control multiple execution environments.

  • util/dockershell/compose creates a set of *exec.Execs from docker-compose YAML.
  • util/dockershell/kind creates a set of *exec.Execs from Kind config YAML. This also provides a kubectl wrapper. This wrapper enables kind.New(kindYAML).KubeCtl(args ...).Run() which executes arbitrary kubectl commands against the created cluster.
  • util/dockershell provides wrapper of util/dockershell/exec. This gives us shellscript-like experience for executing commands in containers. For example, dockershell.New(nil, container).X(a).X(b).X(c) executes commands a, b and c sequencially in the container.

Example:

import (
	"bytes"
        "fmt"
        "os"

	shell "github.com/containerd/stargz-snapshotter/util/dockershell"
	"github.com/containerd/stargz-snapshotter/util/dockershell/compose"
	"github.com/containerd/stargz-snapshotter/util/dockershell/kind"
)

// Prepares "hello" container and runs "echo hello" inside it.
func echoFromContainer() {
	// Create set of containers
	containers, _ := compose.New(`
version: "3.7"
services:
  hello:
    image: ghcr.io/stargz-containers/ubuntu:20.04-org
    init: true
    entrypoint: [ "sleep", "infinity" ]
`, compose.WithStdio(os.Stdin, os.Stdout))
	defer containers.Cleanup()

	// Get the shell of service "hello"
	container, _ := containers.Get("hello")
	sh := shell.New(container, nil)

	// Echo hello inside the container
	buf := new(bytes.Buffer)
	sh.
		X("apt-get", "update", "-y").
		X("apt-get", "install", "-y", "figlet").
		Pipe(buf, shell.C("echo", "hello"), shell.C("figlet"))
	fmt.Println(string(buf.Bytes()))
}

// Creates Kind cluster and dumps kubelet log
func dumpKindNodeLog() {
	// Create Kind cluster
	k, _ := kind.New(`
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  image: docker.io/kindest/node:v1.20.0
`, kind.WithStdio(os.Stdin, os.Stdout))
	defer k.Cleanup()

	// Get shell of the node
	node, _ := k.Get(k.List()[0])
	sh := shell.New(node, nil)

	// Dump log
	fmt.Println(string(sh.O("journalctl", "-u", "kubelet")))
}

Currently, util/dockershell contains only methods needed for our integration test and benchmark. Some features may be still missing for general use and we still have room for improvement.

@ktock ktock marked this pull request as draft April 28, 2021 02:40
@ktock ktock force-pushed the test-from-go branch 6 times, most recently from a0845c4 to 685a9bb Compare April 29, 2021 15:03
@ktock ktock force-pushed the test-from-go branch 5 times, most recently from ac25d37 to 77da4c9 Compare May 2, 2021 05:20
Recently, it is becoming harder and harder to maintain shellscript-based CI.
Let's switch to golang-based tests.
This will help us to write test codes in a type-safe & modular manner,
leveraging rich language features of Go.

Signed-off-by: Kohei Tokunaga <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant