Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update documentation regarding containerlab migration.
Browse files Browse the repository at this point in the history
Gerrit91 committed Nov 15, 2021

Unverified

This user has not yet uploaded their public signing key.
1 parent f596728 commit f297ec5
Showing 3 changed files with 46 additions and 67 deletions.
111 changes: 45 additions & 66 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,70 +1,62 @@
# mini-lab

The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development.
The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a metal-stack partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development.

![overview components](docs/overview.png)

This project can also be used as a template for writing your own metal-stack deployments.
This project can also be used as a template for writing your own metal-stack deployments.

<!-- TOC depthFrom:2 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 -->
<!-- TOC depthfrom:2 depthto:6 withlinks:false updateonsave:false orderedlist:false -->

- [mini-lab](#mini-lab)
- [Requirements](#requirements)
- [Known Limitations](#known-limitations)
- [Try it out](#try-it-out)
- [Reinstall machine](#reinstall-machine)
- [Remove machine](#remove-machine)
- [Development of metal-api, metal-hammer and metal-core](#development-of-metal-api-metal-hammer-and-metal-core)
- Requirements
- Known Limitations
- Try it out
- Reinstall machine
- Remove machine
- Flavors
- Development of metal-api, metal-hammer and metal-core

<!-- /TOC -->

## Requirements

- Linux machine with hardware virtualization support
- [Vagrant](https://www.vagrantup.com/) == 2.2.14 with vagrant-libvirt plugin >= 0.3.0 (for running the switch and machine VMs)
- kvm as hypervisor for the VMs
- Ubuntu 20.04:

sudo apt update
sudo apt install -y qemu qemu-kvm libvirt-daemon bridge-utils virtinst libvirt-dev git curl build-essential wget

- kvm as hypervisor for the VMs (you can check through the `kvm-ok` command)
- [docker](https://www.docker.com/) >= 18.09 (for using kind and our deployment base image)
- [docker-compose](https://docs.docker.com/compose/) >= 1.25.4 (for ease of use and for parallelizing control plane and partition deployment)
- [kind](https://github.com/kubernetes-sigs/kind/releases) == v0.9.0 (for hosting the metal control plane on a kubernetes cluster v1.19.1)
- [ovmf](https://wiki.ubuntu.com/UEFI/OVMF) to have a uefi firmware for virtual machines
- the lab creates a virtual network 192.168.121.0/24 on your host machine, this hopefully does not overlap with other networks you have
- [containerlab](https://containerlab.srlinux.dev/install/)
- the lab creates a docker network on your host machine (`172.17.0.1`), this hopefully does not overlap with other networks you have
- (recommended) haveged to have enough random entropy (only needed if the PXE process does not work)

Here is some code that should help you setting up most of the requirements:

```bash
# Install kvm
sudo apt install -y git curl qemu qemu-kvm havaged

# Install Docker
curl -fsSL https://get.docker.com | sh
# if you want to be on the safe side, follow the original installation
# instructions at https://docs.docker.com/engine/install/ubuntu/

# Install vagrant and other stuff
wget https://releases.hashicorp.com/vagrant/2.2.14/vagrant_2.2.14_x86_64.deb
sudo apt-get install ./vagrant_2.2.14_x86_64.deb qemu-kvm virt-manager ovmf net-tools libvirt-dev haveged

# Ensure that your user is member of the group "libvirt" and "docker"
# Ensure that your user is member of the group "docker"
# you need to login again in order to make this change take effect
sudo usermod -G libvirt,docker -a ${USER}
sudo usermod -G docker -a ${USER}

# Install libvirt plugin for vagrant
vagrant plugin install vagrant-libvirt
# Install containerlab
bash -c "$(curl -sL https://get-clab.srlinux.dev)"

# Install kind (kubernetes in docker), for more details see https://kind.sigs.k8s.io/docs/user/quick-start/#installation
curl -Lo ./kind https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind
sudo curl -Lo /usr/local/bin/kind "https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64"
sudo chmod +x /usr/local/bin/kind

# Install docker-compose, for more details see https://docs.docker.com/compose/install/
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo curl -Lo /usr/local/bin/docker-compose "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)"
sudo chmod +x /usr/local/bin/docker-compose
```

The following ports are getting used statically:
The following ports are used statically on your host machine:

| Port | Bind Address | Description |
|:----:|:------------ |:---------------------------------- |
@@ -82,15 +74,18 @@ The following ports are getting used statically:

## Try it out

Start the mini-lab with a kind cluster, a metal-api instance as well as some vagrant VMs with two leaf switches and two machine skeletons.

```bash
git clone https://github.com/metal-stack/mini-lab.git
cd mini-lab
```

Start the mini-lab with a kind cluster, a metal-api instance as well as two containers wrapping leaf switches and another container that hosts two user-allocatable machines:

```bash
make
```

Two machines in status `PXE booting` are visible with `metalctl machine ls`
After the deployment and waiting for a short amoung of time, two machines in status `PXE booting` become visible through `metalctl machine ls`:

```bash
docker-compose run metalctl machine ls
@@ -100,7 +95,7 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258   PXE Booting 3s
2294c949-88f6-5390-8154-fa53d93a3313 PXE Booting 5s
```

Wait until the machines reach the waiting state
Wait until the machines reach the waiting state:

```bash
docker-compose run metalctl machine ls
@@ -110,42 +105,42 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258   Waiting 8s
2294c949-88f6-5390-8154-fa53d93a3313   Waiting 8s v1-small-x86 vagrant
```

Create a machine/firewall with
Create a firewall and a machine with:

```bash
make firewall
make machine
```

__Alternatively__ you may want to issue the `metalctl` commands by your own:
__Alternatively__, you may want to issue the `metalctl` commands on your own:

```bash
docker-compose run metalctl network allocate \
--partition vagrant \
--partition mini-lab \
--project 00000000-0000-0000-0000-000000000000 \
--name vagrant
--name user-private-network

# Lookup the network ID and create a machine
# lookup the network ID and create a machine
docker-compose run metalctl machine create \
--description test \
--name machine \
--hostname machine \
--project 00000000-0000-0000-0000-000000000000 \
--partition vagrant \
--partition mini-lab \
--image ubuntu-20.04 \
--size v1-small-x86 \
--networks <network-ID>

# Create a firewall that is also connected to the virtual internet-vagrant-lab network
# create a firewall that is also connected to the virtual internet-mini-lab network
docker-compose run metalctl machine create \
--description fw \
--name fw \
--hostname fw \
--project 00000000-0000-0000-0000-000000000000 \
--partition vagrant \
--partition mini-lab \
--image firewall-ubuntu-2.0 \
--size v1-small-x86 \
--networks internet-vagrant-lab,$(privatenet)
--networks internet-mini-lab,$(privatenet)
```

See the installation process in action
@@ -170,12 +165,10 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258   Phoned Home 2s 21s machin
Login with user name metal and the console password from

```bash
docker-compose run metalctl machine describe e0ab02d2-27cd-5a5e-8efc-080ba80cf258 | grep password

consolepassword: ...
docker-compose run metalctl machine consolepassword e0ab02d2-27cd-5a5e-8efc-080ba80cf258
```

If you want to access the firewall with SSH or have internet connectivity from the firewall and machine, you'll need to have a static route configured that points to the vagrant boxes of the leaf switches:
If you want to access the firewall with SSH or have internet connectivity from the firewall and machine, you'll need to have a static route configured that points to the leaf switches:

```bash
# Add the route to the network internet-vagrant-lab 100.255.254.0/24 via leaf01 and leaf02, whose IPs are dynamically allocated. Make sure there's no old route before execution.
@@ -185,7 +178,7 @@ make route
ssh [email protected]
```

To remove the kind cluster and the vagrant boxes, run
To remove the kind cluster and VMs, run:

```bash
make cleanup
@@ -201,9 +194,9 @@ docker-compose run metalctl machine reinstall \
e0ab02d2-27cd-5a5e-8efc-080ba80cf258
```

### Remove machine
### Free machine

Remove a machine with
Free a machine with

```bash
docker-compose run metalctl machine rm e0ab02d2-27cd-5a5e-8efc-080ba80cf258
@@ -222,17 +215,3 @@ In order to start specific flavor, you can define the flavor as follows:
export MINI_LAB_FLAVOR=cluster-api
make
```

## Development of metal-api, metal-hammer and metal-core

To simplify developing changes for the `metal-api`, `metal-hammer` and `metal-core`, it is possible to use development artifacts from within the mini-lab.
See the [dev instructions](DEV_INSTRUCTIONS.md) for more details.

FROM ubuntu:20.04

RUN apt update -y && \
DEBIAN_FRONTEND=noninteractive apt install -y virt-manager ovmf net-tools haveged \
qemu qemu-kvm bridge-utils virtinst libvirt-dev libvirt-daemon-system build-essential libvirt-clients

CMD ["/usr/sbin/init"]

2 changes: 1 addition & 1 deletion docs/overview.drawio

Large diffs are not rendered by default.

Binary file modified docs/overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit f297ec5

Please sign in to comment.