Skip to content

Commit

Permalink
memtierd: Added Memtierd demo files
Browse files Browse the repository at this point in the history
The demo showcases the benefits of using Memtierd as a memory manager for your workloads.

Signed-off-by: Luukas Mäkilä <[email protected]>
  • Loading branch information
Luukas Mäkilä committed Jun 29, 2023
1 parent 893a277 commit 06c749b
Show file tree
Hide file tree
Showing 15 changed files with 1,523 additions and 0 deletions.
34 changes: 34 additions & 0 deletions demo/memtierd/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Memtierd demo

To replicate the demo see [replicating the demo](https://github.com/containers/nri-plugins/demo/Replicating-the-demo.md) for instructions.

## What was showcased

The demo showcases the differenece between how low priority and high priority workloads are treated when using Memtierd as the memory manager. Low priority workloads and high priority workloads are defined by giving your deployments the following annotations:

```yaml
class.memtierd.nri: "high-prioconfiguration"
# or
class.memtierd.nri: "low-prio-configuration"
```
This annotation defines whether the workload will be swapped agressively (low-prio) or more moderately (high-prio). More agressive swapping will lead to an increase in the number of page faults the process will have.
## About the metrics
RAM Saved (G)
- How much RAM is being saved by swapping out the idle workloads.
RAM Saved (%)
- How big the total memory saved is in comparison to the overall memory of the system.
Compressed (%)
- How well the data is being compressed.
RAM vs Swap
- How the memory is being distributed between RAM and the swap.
Page faults
- How many new page faults happen in between the requests from Grafana. This is a way to express the possible performance hit workloads experience if being tracked by Memtierd.
![alt text](https://github.com/containers/nri-plugins/demo/memtierd-demo.png)
113 changes: 113 additions & 0 deletions demo/memtierd/Replicating-the-demo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Replicating the Memtierd demo

## Prerequisites
- Python 3
- Memtierd and Meme installed see [here](https://github.com/askervin/cri-resource-manager/tree/5FD_memtierd_devel/cmd/memtierd)
- NRI enabled on your container runtime (Containerd/CRI-O)
- Grafana dashboard ready to go
- Grafana [infinity](https://grafana.com/grafana/plugins/yesoreyeram-infinity-datasource/) data source plugin downloaded

## Installing the Memtierd grafana dashboard

- Go to the "Data Sources" tab and apply the Infinity data source (needed to handle the showcase the json data)
- Go to the "Dashboards" section on Grafana
- Click "New" and download the "memtierd-demo-grafana-dashboard.json" file and import it or use the dashboard id [18744](https://grafana.com/grafana/dashboards/18744-memtierd-demo/)
- Select Infinity data source as the data source

## Creating a swap in RAM

Needed in Ubuntu because zram.ko kernel module might not be installed in the system by default
```console
apt install linux-modules-extra-$(uname -r)
```

Create a 4GB compressed swap in RAM
```console
modprobe zram
echo 4G > /sys/block/zram0/disksize
mkswap /dev/zram0
swapon /dev/zram0
```

Check that the swap got created
```console
free -h
```

## Running the API

Edit the "path" variables found on the top of the main.py file to point to the correct data files in data/ aswell as zram and meminfo paths. When ran with the default workloads /tmp/memtierd directory will be created to read the output from, so unless the workload configurations are changed, those paths won't need editing.

Install FastAPI:
```console
pip install fastapi
```

Start the API with:
```console
uvicorn main:app --reload
```

Make sure the files in data/ are in the correct format:

Page fault files:
```json
{
"page_faults_lowprio_1": [
]
}
```
```json
{
"page_faults_highprio_1": [
]
}
```

Time series files:
```json
{
"time_series_lowprio_1": [
]
}
```
```json
{
"time_series_highprio_1": [
]
}
```

## Configuring the Memtierd NRI plugin:

Follow the steps found at cmd/memtierd and then:
```console
kubectl apply -f cmd/memtierd/templates/pod-memtierd.yaml
```

## Replicating the workloads

Install meme with the instructions found [here](https://github.com/intel/memtierd/blob/main/cmd/memtierd/README.md#install-memtierd-on-the-vm).

Then create a meme image, you can use this Dockerfile:
```dockerfile
FROM debian:stable-slim

COPY . .

RUN mv meme /bin

CMD ["meme", "-bs", "1G", "-bwc", "1", "-bws", "100M", "-bwi", "1s", "-bwod", "17M", "-brc", "0", "-p", "10s", "-ttl", "24h"]
```

Then push that to your image registry and point the high-prio and low-prio deployment files to pull from there.

Deploy the high priority workloads:
```console
kubectl apply -f templates/high-prio.yaml
```

Deploy the low priority workloads:
```console
kubectl apply -f templates/low-prio.yaml
```
4 changes: 4 additions & 0 deletions demo/memtierd/data/highprio_1_time_series.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"time_series_highprio_1": [
]
}
4 changes: 4 additions & 0 deletions demo/memtierd/data/lowprio_1_time_series.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"time_series_lowprio_1": [
]
}
4 changes: 4 additions & 0 deletions demo/memtierd/data/page_faults_highprio_1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"page_faults_highprio_1": [
]
}
4 changes: 4 additions & 0 deletions demo/memtierd/data/page_faults_lowprio_1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"page_faults_lowprio_1": [
]
}
11 changes: 11 additions & 0 deletions demo/memtierd/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Welcome to Memtierd demo documentation

```{toctree}
---
maxdepth: 2
caption: Contents
---
README.md
Replicating-the-demo.md
Project GitHub repository <https://github.com/containers/nri-plugins>
```
Loading

0 comments on commit 06c749b

Please sign in to comment.