Adding Benchmark section to our documentation page

.github/workflows/download_svgs.yml

@@ -0,0 +1,26 @@
+name: Download SVGs
+
+on:
+  workflow_dispatch: # You can trigger the workflow manually, or set it on a schedule.
+
+jobs:
+  download-svgs:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v2
+
+      - name: Set up environment variables
+        run: |
+          echo "MUNIN_USERNAME=${{ secrets.MUNIN_USERNAME }}" >> $GITHUB_ENV
+          echo "MUNIN_PASSWORD=${{ secrets.MUNIN_PASSWORD }}" >> $GITHUB_ENV
+
+      - name: Make download_svgs.sh executable
+        run: chmod +x /scripts/download_svgs.sh
+
+      - name: Run download_svgs.sh
+        env:
+          MUNIN_USERNAME: ${{ secrets.MUNIN_USERNAME }}
+          MUNIN_PASSWORD: ${{ secrets.MUNIN_PASSWORD }}
+        run: |
+          /scripts/download_svgs.sh

docs/src/benchmarks/index.md

@@ -0,0 +1,90 @@
+---
+title: Benchmarks
+sidebar_position: 10
+---
+
+# Benchmarks
+
+This section contains performance benchmarks for ThinEdge.io, including detailed CPU and memory usage for key processes, measured on a device running in the OSADL QA Farm.
+
+## Hardware Information
+
+The following hardware and software specifications describe the environment in which these benchmarks are measured. The device runs ThinEdge.io and its associated plugins for data collection.
+
+| Component      | Specification                                                 |
+|----------------|---------------------------------------------------------------|
+| **Processor**  | Quad-core ARM Cortex-A72 (ARM v8)                             |
+| **Memory**     | 1.8 GB RAM                                                    |
+| **Operating System** | Debian GNU/Linux 12 (bookworm)                          |
+| **MQTT Broker**| Mosquitto (v2.0.18)                                           |
+| **Firmware**   | ThinEdge.io (v1.1.2)                                          |
+| **Plugins Used** | `tedgecpuprocent`, `memory`, `tedge-agent`, `tedge-mapper`  |
+
+> The device is a Raspberry Pi 4 Model B Rev 1.5, a member of the OSADL QA Farm, which continuously collects data on performance and resource consumption.
+
+## CPU Run-time Consumption of Key ThinEdge.io Tasks
+
+In this section, we monitor the CPU usage of the most critical ThinEdge.io tasks: `mosquitto`, `tedge-agent`, and `tedge-mapper`. These tasks are essential for device-to-cloud communication, and their performance is crucial for the overall efficiency of the system.
+
+### Graph of CPU Usage
+
+Below is a visual representation of the CPU consumption by the main ThinEdge.io processes over the past month.
+
+![CPU Usage](./tedgecpuprocent-month.svg)
+
+For each process, the graph shows:
+- **Command Name (comm)**: The name of the task running on the device.
+- **CPU Usage (cuc)**: The percentage of CPU time utilized by each process.
+
+
+The graph allows you to visually track CPU performance trends and identify potential bottlenecks in the system.
+
+### Detailed Metrics
+
+The following table provides a detailed description of the CPU consumption of measured tasks:
+
+| Task                 | Description                                                    |
+|----------------------|----------------------------------------------------------------|
+| `mosquitto`          | Handles MQTT communications between devices and cloud          |
+| `tedge-agent`        | Coordinates device management, software updates, and telemetry |
+| `tedge-mapper_c8y`   | Converts telemetry data into the format required by cloud      |
+| `tedge-mapper-colle` | Converts telemetry data into the format required               |
+
+> Note: The values in this table are updated dynamically once per month.
+
+
+## Memory Usage
+
+In this section, we monitor the memory consumption of key ThinEdge.io components to ensure efficient performance and to view how the system memory is being utilized.
+
+### Graph of Memory Usage
+
+Below is a visual representation of the memory consumption of ThinEdge.io tasks over the past month.
+
+![Memory Usage](./tedgemem-month.svg)
+
+### Monitored Metrics:
+- **Mosquitto Memory Usage**: Tracks the memory consumed by the mosquitto process, which manages MQTT messages and connections.
+- **Tedge-Agent Memory Usage**: Tracks the memory consumed by the tedge-agent process, which handles software updates, configurations, and telemetry operations.
+- **Tedge-Mapper Memory Usage**: Tracks the memory consumed by the tedge-mapper process, which converts telemetry data between MQTT and other protocols (e.g., Cumulocity, AWS). For multi-instance tedge-mapper, each instance is tracked individually.
+
+For each metric, the graph shows the current and historical values, allowing you to monitor how memory is utilized by the system.
+
+### Detailed Metrics
+
+The following table provides a description of memory usage metrics:
+
+| Metric                   | Description                                                                 |
+|--------------------------|-----------------------------------------------------------------------------|
+| **Mosquitto Memory**     | Memory used by the mosquitto process, which manages MQTT messages and connections. |
+| **Tedge-Agent Memory**   | Memory used by the tedge-agent process, responsible for software updates, configurations, and telemetry operations |
+| **Tedge-Mapper Memory**  | Memory used by the tedge-mapper process, which converts telemetry data between MQTT and other protocols (e.g., Cumulocity, AWS). Each instance of tedge-mapper is monitored individually if multiple instances exist |
+
+
+> The values in this table are updated dynamically once per month.
+
+By monitoring these metrics, we can ensure that the system is using memory efficiently, and we can detect potential memory leaks or bottlenecks in the system's operation.
+
+---
+
+This section now contains all relevant memory metrics for ThinEdge tasks, helping you track and analyze how resources are allocated and used by mosquitto, tedge-agent, and tedge-mapper over time. These benchmarks are crucial for maintaining optimal performance of ThinEdge.io components and ensuring efficient memory usage across the system.

plugins/munin/tedgecpuprocent

@@ -0,0 +1,73 @@
+#!/bin/sh
+# -*- sh -*-
+
+: << =cut
+
+=head1 NAME
+
+tedgecpuprocent - Monitor CPU % consumption of ThinEdge tasks mosquitto, tedge-agent, and tedge-mapper
+
+=head1 AUTHOR
+
+Gligor Isaev
+
+=head1 LICENSE
+
+GPL-3.0-or-higher
+
+=head1 MAGIC MARKERS
+
+ #%# family=manual
+
+=cut
+
+. "$MUNIN_LIBDIR/plugins/plugin.sh"
+
+# Get the PIDs of mosquitto, tedge-agent, and tedge-mapper
+mosquitto_pid=$(pidof mosquitto)
+tedgeagent_pid=$(pidof tedge-agent)
+tedgemapper_pids=$(pidof tedge-mapper)
+
+tasks=""
+cpu=""
+
+# Fetch CPU usage percentage for each process using ps
+for i in $mosquitto_pid $tedgeagent_pid; do
+    commcpu=$(ps -p $i -o comm=,%cpu= | tr -s ' ')
+    comm=$(echo $commcpu | cut -d' ' -f1)     # Get the process name
+    cpu_usage=$(echo $commcpu | cut -d' ' -f2)  # Get the CPU percentage
+    tasks="$tasks $comm"
+    cpu="$cpu $cpu_usage"
+done
+
+# Fetch CPU usage percentage for tedge-mapper processes
+for i in $tedgemapper_pids; do
+    commcpu=$(ps -p $i -o args=,%cpu= | tr -s ' ')
+    comm=$(echo $commcpu | cut -d/ -f4 | cut -d' ' -f1,2 | tr ' ' _)  # Extract process name
+    cpu_usage=$(echo $commcpu | cut -d' ' -f3)                        # Get CPU percentage
+    tasks="$tasks $comm"
+    cpu="$cpu $cpu_usage"
+done
+
+# If config is requested, output graph configuration
+if [ "$1" = "config" ]; then
+    echo 'graph_title Real-time usage CPU percentage of ThinEdge.io tasks'
+    echo 'graph_args --base 1000 --vertical-label % -l 0'
+    echo 'graph_category thin-edge'
+    echo 'graph_vlabel CPU Usage (%)'
+    echo 'graph_scale no'  # Ensures no unit scaling
+    echo 'graph_info This graph shows CPU usage as a percentage for key ThinEdge.io processes.'
+    for i in $tasks; do
+        echo "$i.label $i"    # Set the label name for each process
+        echo "$i.draw LINE2"   # Optional: ensure the drawing style is consistent
+    done
+    exit 0
+fi
+
+# Output the actual data for Munin to plot
+tasknum=$(echo "$tasks" | tr ' ' "\n" | wc -l)
+for i in $(seq 1 "$tasknum"); do
+    task=$(echo "$tasks" | cut -d' ' -f$i)
+    cpu_usage=$(echo "$cpu" | cut -d' ' -f$i)
+    echo "$task.value $cpu_usage"
+done

plugins/munin/tedgemonitorcpu

@@ -0,0 +1,111 @@
+#PLEASE NOTE THIS PLUGIN IS STILL UNDER CONSTRUCTION
+
+#!/bin/sh
+# -*- sh -*-
+
+: << =cut
+
+=head1 NAME
+
+tedgecpuprocent - Monitor CPU % consumption of ThinEdge tasks mosquitto, tedge-agent, and tedge-mapper and send alarm if deviated
+
+=head1 AUTHOR
+
+Gligor Isaev
+
+=head1 LICENSE
+
+GPL-3.0-or-higher
+
+=head1 MAGIC MARKERS
+
+ #%# family=manual
+
+=cut
+. "$MUNIN_LIBDIR/plugins/plugin.sh"
+
+# Get the PIDs of mosquitto, tedge-agent, and tedge-mapper
+mosquitto_pid=$(pidof mosquitto)
+tedgeagent_pid=$(pidof tedge-agent)
+tedgemapper_pids=$(pidof tedge-mapper)
+
+tasks=""
+cpu=""
+mem_usage=""
+mem_prev=""
+
+# Fetch CPU and memory usage percentage for each process using ps
+for i in $mosquitto_pid $tedgeagent_pid; do
+    commcpu=$(ps -p $i -o comm=,%cpu=,%mem= | tr -s ' ')
+    comm=$(echo $commcpu | cut -d' ' -f1)     # Get the process name
+    cpu_usage=$(echo $commcpu | cut -d' ' -f2)  # Get the CPU percentage
+    mem_now=$(echo $commcpu | cut -d' ' -f3)    # Get the Memory percentage
+    tasks="$tasks $comm"
+    cpu="$cpu $cpu_usage"
+    mem_usage="$mem_usage $mem_now"
+
+    # Check for memory increase threshold
+    mem_prev=$(cat /tmp/${comm}_mem_prev 2>/dev/null || echo "$mem_now")
+    mem_increase=$(echo "scale=2; $mem_now - $mem_prev" | bc)
+    threshold=20.0
+
+    if [ "$(echo "$mem_increase > $threshold" | bc)" -eq 1 ]; then
+        # Raise MQTT alarm if memory increase is greater than 20%
+        tedge mqtt pub "te/device/main/a/memory_increase" '{
+          "text": "Memory consumption increased by more than 20% for '$comm'",
+          "severity": "major"
+        }' --retained
+    fi
+
+    echo "$mem_now" > /tmp/${comm}_mem_prev  # Store the current memory for the next check
+done
+
+# Fetch CPU and memory usage percentage for tedge-mapper processes
+for i in $tedgemapper_pids; do
+    commcpu=$(ps -p $i -o args=,%cpu=,%mem= | tr -s ' ')
+    comm=$(echo $commcpu | cut -d/ -f4 | cut -d' ' -f1,2 | tr ' ' _)
+    cpu_usage=$(echo $commcpu | cut -d' ' -f3)
+    mem_now=$(echo $commcpu | cut -d' ' -f4)
+    tasks="$tasks $comm"
+    cpu="$cpu $cpu_usage"
+    mem_usage="$mem_usage $mem_now"
+
+    mem_prev=$(cat /tmp/${comm}_mem_prev 2>/dev/null || echo "$mem_now")
+    mem_increase=$(echo "scale=2; $mem_now - $mem_prev" | bc)
+
+    if [ "$(echo "$mem_increase > $threshold" | bc)" -eq 1 ]; then
+        tedge mqtt pub "te/device/main/a/memory_increase" '{
+          "text": "Memory consumption increased by more than 20% for '$comm'",
+          "severity": "major"
+        }' --retained
+    fi
+
+    echo "$mem_now" > /tmp/${comm}_mem_prev
+done
+
+# Output graph configuration if requested
+if [ "$1" = "config" ]; then
+    echo 'graph_title Real-time CPU and Memory usage of ThinEdge.io tasks'
+    echo 'graph_args --base 1000 --vertical-label % -l 0'
+    echo 'graph_category thin-edge'
+    echo 'graph_vlabel CPU & Memory Usage (%)'
+    echo 'graph_scale no'
+    echo 'graph_info This graph shows CPU and Memory usage as a percentage for key ThinEdge.io processes.'
+    for i in $tasks; do
+        echo "$i_cpu.label $i CPU"
+        echo "$i_cpu.draw LINE2"
+        echo "$i_mem.label $i Memory"
+        echo "$i_mem.draw LINE2"
+    done
+    exit 0
+fi
+
+# Output the actual data for Munin to plot
+tasknum=$(echo "$tasks" | tr ' ' "\n" | wc -l)
+for i in $(seq 1 "$tasknum"); do
+    task=$(echo "$tasks" | cut -d' ' -f$i)
+    cpu_usage=$(echo "$cpu" | cut -d' ' -f$i)
+    mem_now=$(echo "$mem_usage" | cut -d' ' -f$i)
+    echo "$task_cpu.value $cpu_usage"
+    echo "$task_mem.value $mem_now"
+done

scripts/download_svgs.sh

@@ -0,0 +1,41 @@
+#!/bin/bash
+
+# Define your Munin credentials
+MUNIN_USERNAME="your_username"
+MUNIN_PASSWORD="your_password"
+
+# List of SVG files to download
+SVG_FILES=("tedgecpuprocent-month.svg" "tedgemem-month.svg")
+
+# Base URL of the Munin server
+BASE_URL="https://munin.osadl.org/munin/osadl.org/rackfslot1.osadl.org/"
+
+# Set the output folder relative to the project folder
+PROJECT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && cd ../.. && pwd)"
+OUTPUT_FOLDER="$PROJECT_DIR/thin-edge.io/docs/src/benchmarks"
+
+# Print the project directory and output folder for debugging
+echo "Project Directory: $PROJECT_DIR"
+echo "Output Folder: $OUTPUT_FOLDER"
+
+# Create the output folder if it doesn't exist
+mkdir -p "$OUTPUT_FOLDER"
+
+# Download each SVG file
+for SVG_FILE in "${SVG_FILES[@]}"; do
+    SVG_URL="${BASE_URL}${SVG_FILE}"
+    OUTPUT_PATH="${OUTPUT_FOLDER}/${SVG_FILE}"
+
+    # Print the output path for debugging
+    echo "Downloading $SVG_FILE to $OUTPUT_PATH"
+
+    # Download the file using curl with basic authentication and skipping certificate verification
+    curl --insecure --user "$MUNIN_USERNAME:$MUNIN_PASSWORD" "$SVG_URL" --output "$OUTPUT_PATH"
+
+    # Check if the download was successful
+    if [[ $? -eq 0 ]]; then
+        echo "Downloaded $SVG_FILE to $OUTPUT_PATH"
+    else
+        echo "Failed to download $SVG_FILE"
+    fi
+done