Toolbox

This repository contains a collection of scripts to automate various tasks related to installation and end-to-end testing.

Table of Contents

• Toolbox
  • Table of Contents
  • Prerequisites
  • Utility Scripts
    • Create a Kind Cluster
      • Usage
      • Arguments
    • Install MetalLB
      • Usage
      • Arguments
  • Kgateway
    • Install Kgateway
      • Usage
      • User-Facing Variables
    • Inference Extension Testing
      • Usage
      • Arguments
      • User-Facing Variables
    • HTTPRoute Testing
      • Usage
      • Arguments
      • User-Facing Variables
    • TCPRoute Testing
      • Usage
      • Arguments
      • User-Facing Variables
    • Uninstall Kgateway
      • Usage
  • Istio
    • Install Istio
      • Usage
      • Arguments
      • User-Facing Variables
    • Ambient Testing
      • Usage
      • Arguments
      • User-Facing Variables
    • Uninstall Istio
      • Usage
      • Arguments
  • Contributing

Prerequisites

Ensure the following tools are installed:

• kubectl: The Kubernetes command-line tool.
• istioctl: The Istio command-line tool.
• helm: A package manager for Kubernetes.

Utility Scripts

Create a Kind Cluster

The kind-cluster.sh script creates or deletes a Kubernetes kind cluster based on the provided argument. The cluster consists of 1 control-plane and 2 worker nodes.

Usage

./scripts/kind-cluster.sh [create|delete]

Arguments

• create: Create the kind cluster.
• delete: Delete the cluster.

Install MetalLB

The metallb.sh script installs or uninstalls MetalLB in the Kubernetes cluster configured in the current kubectl context.

NOte: Mac users must run docker-mac-net-connect to get LoadBalancer services to work (required by testing scripts).

Usage

./scripts/metallb.sh [apply|delete]

Arguments

• apply: Install MetalLB in the Kubernetes cluster.
• delete: Uninstall MetalLB in the Kubernetes cluster.

Kgateway

Kgateway is a feature-rich, fast, and flexible Kubernetes-native ingress controller and next-generation API gateway that is built on top of Envoy proxy and Gateway API.

Install Kgateway

The install-kgateway.sh script automates the installation of Kgateway on a Kubernetes cluster.

Usage

./scripts/install-kgateway.sh

User-Facing Variables

• KGTW_VERSION: The version of Kgateway to install. Defaults to "v2.0.0-main".
• KGTW_REGISTRY: The name of the image registry to pull the Kgateway image from. Defaults to "ghcr.io/kgateway-dev".
• HELM_CHART: The location of the Kgateway Helm chart. Specify the full path to the tarball for local charts. Defaults to "oci://ghcr.io/kgateway-dev/charts/kgateway".
• HELM_CRD_CHART: The location of the Kgateway CRDs Helm chart. Specify the full path to the tarball for local charts. Defaults to "oci://ghcr.io/kgateway-dev/charts/kgateway".
• INSTALL_CRDS: Install the Gateway API CRDs. Defaults to true.
• GATEWAY_API_VERSION: The version of Gateway API CRDs to install. Defaults to "v1.2.1".
• GATEWAY_API_CHANNEL: The channel of Gateway API CRDs to install. Defaults to "experimental" (required for TCPRoute testing).
• INF_EXT_VERSION: The version of Gateway API Inference Extension to use. Defaults to "v0.1.0".
• SVC_TYPE: The type of Service to use for Gateway resources. Defaults to "LoadBalancer".

Inference Extension Testing

The test-kgateway-inference-ext.sh script automates the setup and testing of Gateway API Inference Extension support for Kgateway (./scripts/install-kgateway.sh required).

Usage

./scripts/test-kgateway-inference-ext.sh [apply|delete]

Arguments

• apply: Deploy all resources and test connectivity.
• delete: Clean up all resources.

User-Facing Variables

• HF_TOKEN: Your Hugging Face API token with access to the Llama-2-7b-hf model. Defaults to "" is required when PROC_TYPE=gpu (default).
• NS: The namspace to use for testing. The script will create the namespace if it does not exist. Defaults to "" (meaning the default namespace).
• NUM_REPLICAS: The number of replicas for the model server deployment. Defaults to 3.
• CURL_POD: Whether or not to use a pod to run the client curl commands. Defaults to true.
• BACKOFF_TIME: Specifies the time in seconds to wait between retries during connectivity testing. Defaults to 5.
• MAX_RETRIES: The maximum number of retry attempts for connectivity testing. Defaults to 12.
• PROC_TYPE: The processor type to use for vLLM, either "cpu" or "gpu" (default).
• METAL_LB: Whether or not to use MetalLB for LoadBalancer type services. Defaults to "false"

HTTPRoute Testing

The test-kgateway-httproute.sh script automates the setup and testing of HTTProute support for Kgateway (./scripts/install-kgateway.sh required).

Usage

./scripts/test-kgateway-httproute.sh [apply|delete]

Arguments

• apply: Deploy all resources and test connectivity.
• delete: Clean up all resources.

User-Facing Variables

• NS: Specifies the namespace in which resources will be created or deleted. If a different namespace is used, the script will create it if it doesn't already exist. Defaults to "default"
• CURL_POD: Whether or not to use a pod to run the client curl commands. Defaults to true.
• NUM_REPLICAS: The number of replicas for the echo server deployment. Defaults to 1.
• BACKOFF_TIME: Specifies the time in seconds to wait between retries during connectivity testing. Defaults to 5.
• MAX_RETRIES: The maximum number of retry attempts for connectivity testing. Defaults to 12.
• METAL_LB: Whether or not to use MetalLB for LoadBalancer type services. Defaults to "false"

Example

NS=test BACKOFF_TIME=10 ./scripts/test-kgateway-httproute.sh apply

This command creates namespace 'test' and applies the Kubernetes resources in this namespace with a 10-second backoff time between connectivity testing retries.

TCPRoute Testing

The test-kgateway-tcproute.sh script automates the setup and testing of TCProute support for Kgateway (./scripts/install-kgateway.sh required).

Usage

./scripts/test-kgateway-tcproute.sh [apply|delete]

Arguments

• apply: Deploy all resources and test connectivity.
• delete: Clean up all resources.

User-Facing Variables

• NS: Specifies the namespace in which resources will be created or deleted. If a different namespace is used, the script will create it if it doesn't already exist. Defaults to "default"
• BACKOFF_TIME: Specifies the time in seconds to wait between retries during connectivity testing. Defaults to 5.
• MAX_RETRIES: The maximum number of retry attempts for connectivity testing. Defaults to 12.

Example

NS=test BACKOFF_TIME=10 ./scripts/test-kgateway-tcproute.sh apply

Uninstall Kgateway

The uninstall-kgateway.sh script automates the removal of Kgateway on the Kubernetes cluster in the current kubectl context.

Usage

./scripts/uninstall-kgateway.sh

Istio

Install Istio

The install-istio.sh script automates the installation of Istio on a Kubernetes cluster. It handles the installation of the Istio control plane, ambient mesh configuration, and enables various Istio features such as gateways, telemetry, and ingress.

Usage

./scripts/install-istio.sh [ambient|sidecar]

Arguments

• ambient: Install Istio in ambient mode.
• sidecar: Install Istio in sidecar mode.

User-Facing Variables

• ISTIO_VERSION (default: 1.23.0): The version of Istio to install.
• ISTIO_REPO (default: docker.io/istio): The Docker repository to pull Istio control plane images from.
• ROLLOUT_TIMEOUT (default: 5m): A time unit, e.g. 1s, 2m, 3h, to wait for Istio control-plane component deployment rollout to complete.

Example:

ISTIO_VERSION=1.22.1 ROLLOUT_TIMEOUT=15m ./scripts/install-istio.sh ambient

The example installs Istio in ambient mode and waits up to 15-minutes for each control plane component deployment to report a ready status.

Ambient Testing

The test-ambient.sh script automates the setup and testing of Istio's ambient mode in a Kubernetes cluster. It handles the deployment of Istio waypoints, services, and other resources, and performs connectivity checks between pods.

Usage

./scripts/test-ambient.sh [apply|delete]

Arguments

• apply: Deploy all resources and test connectivity.
• delete: Clean up all resources.

User-Facing Variables

• NS: Specifies the namespace in which resources will be created or deleted. If a different namespace is used, the script will create it if it doesn't already exist. Defaults to "default"
• BACKOFF_TIME: Specifies the time in seconds to wait between retries during connectivity and waypoint stats checks. Defaults to 5.
• MAX_RETRIES: The maximum number of retry attempts for connectivity and waypoint stats checks. Defaults to 12.
• WAYPOINT_STATS_KEY: The specific Istio waypoint stats key to monitor during waypoint connectivity testing. Defaults to "default: http.inbound_0.0.0.0_80;.rbac.allowed".

Example

NS=test BACKOFF_TIME=10 ./scripts/test-ambient.sh apply

This command creates namespace 'test' and applies the Kubernetes resources in this namespace with a 10-second backoff time between retries.

Uninstall Istio

The uninstall-istio.sh script automates the removal of Istio on a Kubernetes cluster in the current kubectl context. It uninstalls the Istio control plane, ambient mesh configuration, etc.

Usage

./scripts/uninstall-istio.sh [ambient|sidecar]

Arguments

• ambient: Uninstall Istio in ambient mode.
• sidecar: Uninstall Istio in sidecar mode.

Contributing

Feel free to open issues or submit pull requests if you'd like to contribute to improving the scripts.