Skip to content

Latest commit

 

History

History
118 lines (87 loc) · 4.3 KB

README.md

File metadata and controls

118 lines (87 loc) · 4.3 KB
Raw

Benchmarks for Spin on Kubernetes

This repository contains source code, scripts, IaC and Kubernetes manifests to repeatably test Spin on Kubernetes.

Requirements

  • Helm - used to install nginx-ingress and the containerd shim for Fermyon Spin
  • Kustomize - used to templatize manifests for Spin deployments
  • Spin v1.5 - current version of the containerd shim (v0.9.2) supports Spin v1.5
  • Azure AKS - IaC currently targets AKS but installing the charts on other containerd based Kubernetes distributions should work and contributions for additional IaC is welcomed

Setup the AKS cluster

To setup a cluster you can use the terraform module provided in infra/azure.

terraform init
terraform apply

Installing required Helm charts

Several helm charts are required for the purposes of the benchmarks. We currently support using the ingress-nginx chart to expose the services. The spin-containerd-shim-installer chart is used to add support for Spin to Kubernetes nodes.

# install using the Makefile
make helm-install

# install the ingress controller manually
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx \
    -n kube-system  \
    --version 4.8.3 \
    --values manifests/nginx-ingress-values.yaml

# install the containerd shim for spin
helm install spin-containerd-shim-installer oci://ghcr.io/fermyon/charts/spin-containerd-shim-installer \
  -n kube-system \
  --version 0.10.0 \
  --values manifests/spin-values.yaml

Building docker images

The docker images for Spin functions are the target of the benchmark tests. For now these are simple "Hello, World" functions and we're looking to add more complex examples in the future. The provided Makefile has targets to help build and push the docker images. Note that you should substitute the REPO and TAG variables to target a container registry your cluster will have access to.

# use the Makefile
export REPO=docker.io/<username>
export TAG=latest
make docker-build
make docker-push

Alternatively you can manually build the docker images yourself. Note that building the .wasm assembly is currently done on the host before the docker build.

# manually build the Spin JavaScript function
cd src/spin-func-js
npm install
spin build
docker buildx build --platform=wasi/wasm --provenance=false docker.io/<username>/spin-func-js:latest

# manually build the Spin Python function
cd src/spin-func-py
spin build
docker buildx build --platform=wasi/wasm --provenance=false docker.io/<username>/spin-func-py:latest

Deploying the functions

The functions are currently templatized using kustomize for simplicity.

# use the Makefile target
make apply-manifests

You can also run the commands manually in your terminal.

# manually run kustomize
kubectl apply -k ./manifests

# alternatively view the manifests before applying them
kubectl kustomize ./manifests

Running the tests from local

To start with, you can run the k6 benchmark tests from your local computer. Currently it uses port-forwarding to interact with the pods you deployed in the previous step.

make k6-run

If you want to run them manually you can use the following command.

./benches/run.sh spin js
./benches/run.sh spin py
./benches/run.sh azfn js
./benches/run.sh azfn py

Summary Results

This is a summary of request execution time generated from the k6 benchmark performed on 12/14/2023. Full representation of the data can be found in json format under the benches directory.

Runtime Language Avg. Response Time Min. Response Time Max. Response Time 90th Percentile 95th Percentile
spin js 86.1ms 59.25ms 391.64ms 108.9ms 186.51ms
spin py 91ms 58.74ms 400.75ms 154.36ms 254.73ms
azfn js 88.63ms 58.58ms 376.16ms 118.33ms 217.46ms
azfn py 87.83ms 59.19ms 445.16ms 128.87ms 190.21ms