From e68d004bf4bf448f3e535d82bda14ae181244937 Mon Sep 17 00:00:00 2001 From: Yanis KHAROUNI Date: Thu, 27 Apr 2023 12:02:50 +0200 Subject: [PATCH 1/9] add stressTest scalability stats --- .docs/test_plan.md | 140 ++++++++++++++++++++++----------------------- 1 file changed, 70 insertions(+), 70 deletions(-) diff --git a/.docs/test_plan.md b/.docs/test_plan.md index 72612d2e5..8da0fd1c4 100644 --- a/.docs/test_plan.md +++ b/.docs/test_plan.md @@ -70,10 +70,10 @@ These tests are dependent of the configuration of the underlying infrastructure | | | | ------ | ----- | | **Date** | 03/29/2023 | -| **Infra version** | 2.12.0 | -| **Core version** | 0.11.1 | -| **API version** | 3.5.2 | -| **Extension c# version** | 0.9.1 | +| **Infra version** | 2.12.3 | +| **Core version** | 0.12.4 | +| **API version** | 3.6.0 | +| **Extension c# version** | 0.9.5 | | **Instance type** | c24.xlarge | | **Processor** | Intel Xeon de 2e (Cascade Lake) | | **CPU frequency** | 3.6 GHz - 3.9 GHz | @@ -83,7 +83,7 @@ These tests are dependent of the configuration of the underlying infrastructure | **EBS Bandwidth (Mbps)** | 19000 | -The versions of the different ArmoniK components and the third-party tools are defined in [versions.tfvars.json](https://github.com/aneoconsulting/ArmoniK/blob/v2.12.0/versions.tfvars.json). +The versions of the different ArmoniK components and the third-party tools are defined in [versions.tfvars.json](https://github.com/aneoconsulting/ArmoniK/blob/v2.12.3/versions.tfvars.json). # Exploited ressources @@ -113,63 +113,63 @@ Faster task processing by increasing the number of pods. #### StressTest -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | | | | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 3.1 | 2.51 | 321.6 | 6.52 | 153.23 | 80.6 | 12.41 | 0.1 | 81.94 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 8.04 | 4.86 | 621.54 | 11.55 | 432.68 | 375.24 | 13.32 | 0.1 | 375.83 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 16.14 | 4.48 | 619.41 | 19.24 | 519.56 | 802.29 | 12.47 | 0.1 | 802.29 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| 148.53 | 5.26 | 673.23 | 155.6 | 642.67 | 10022.1 | 9.98 | 0.08 | 10023.1 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 161.67 | 0.07 | 9.01 | 166.55 | 8.75 | 185.03 | 7.84 | 0.06 | 185.03 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 34.39 | 1.28 | 163.82 | 37.9 | 148.67 | 380.43 | 14.81 | 0.12 | 378.85 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 16.77 | 4.69 | 600.11 | 38.8 | 259.43 | 2979.71 | 3.38 | 0.03 | 2977.71 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| 158.74 | 4.92 | 629.59 | 158.08 | 632.59 | 8773.48 | 11.40 | 0.09 | 8773.48 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | **X** | #### Bench -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | #### HtcMock -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Aggregation level | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Aggregation level | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | | ## Submission scalability tests @@ -180,23 +180,23 @@ Faster task processing by increasing the number of pods. #### StressTest -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1KB | 5 | 1 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10KB | 5 | 5 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 100KB | 5 | 10 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1KB | 5 | 1 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 10KB | 5 | 5 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100KB | 5 | 10 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1KB | 5 | 1 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 10KB | 5 | 5 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 100KB | 5 | 10 | | | | | | | | | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1KB | 5 | 1 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10KB | 5 | 5 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 100KB | 5 | 10 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1KB | 5 | 1 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 10KB | 5 | 5 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100KB | 5 | 10 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1KB | 5 | 1 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 10KB | 5 | 5 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 100KB | 5 | 10 | | | | | | | | | | ## Other tests -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 80K | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1 | CPU: 1000m / Memory: 2048Mi | 300M | 1s | 1KB | 1KB | | | | | | | | | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 80K | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1 | CPU: 1000m / Memory: 2048Mi | 300M | 1s | 1KB | 1KB | | | | | | | | | | This document will be completed in the future with more tests to target more detailed and deeper ArmoniK's features and components. \ No newline at end of file From 325bfbf80d9a35ede717fe840da5d565387a3e2c Mon Sep 17 00:00:00 2001 From: Yanis KHAROUNI Date: Fri, 28 Apr 2023 10:20:10 +0200 Subject: [PATCH 2/9] add stats bench on 100 pods --- .docs/test_plan.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/.docs/test_plan.md b/.docs/test_plan.md index 8da0fd1c4..637b86daf 100644 --- a/.docs/test_plan.md +++ b/.docs/test_plan.md @@ -135,11 +135,11 @@ Faster task processing by increasing the number of pods. | Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Total time (s) | | :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 6.17 | 17.4 | 163.05 | 14.98 | 38.61 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 25.32 | 93.45 | 181.77 | 77.72 | 195.53 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 53.76 | 203.23 | 186.8 | 168.08 | 425 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| 650.33 | 1987.38 | 153.87 | 2482.16 | 5119.94 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **X** | **X** | **X** | **X** | **X** | | 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | | 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | | 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | | From e38ef908af8b1f308ca75adce26752e52f2858da Mon Sep 17 00:00:00 2001 From: Yanis KHAROUNI Date: Wed, 3 May 2023 17:53:59 +0200 Subject: [PATCH 3/9] add benchmarking stats of Armonik v2.12.3 --- .docs/test_plan.md | 136 ++++++++++++++++++++++----------------------- 1 file changed, 68 insertions(+), 68 deletions(-) diff --git a/.docs/test_plan.md b/.docs/test_plan.md index 72612d2e5..6bf13c59c 100644 --- a/.docs/test_plan.md +++ b/.docs/test_plan.md @@ -70,10 +70,10 @@ These tests are dependent of the configuration of the underlying infrastructure | | | | ------ | ----- | | **Date** | 03/29/2023 | -| **Infra version** | 2.12.0 | -| **Core version** | 0.11.1 | +| **Infra version** | 2.12.3 | +| **Core version** | 0.12.4 | | **API version** | 3.5.2 | -| **Extension c# version** | 0.9.1 | +| **Extension c# version** | 0.9.5 | | **Instance type** | c24.xlarge | | **Processor** | Intel Xeon de 2e (Cascade Lake) | | **CPU frequency** | 3.6 GHz - 3.9 GHz | @@ -83,13 +83,13 @@ These tests are dependent of the configuration of the underlying infrastructure | **EBS Bandwidth (Mbps)** | 19000 | -The versions of the different ArmoniK components and the third-party tools are defined in [versions.tfvars.json](https://github.com/aneoconsulting/ArmoniK/blob/v2.12.0/versions.tfvars.json). +The versions of the different ArmoniK components and the third-party tools are defined in [versions.tfvars.json](https://github.com/aneoconsulting/ArmoniK/blob/v2.12.3/versions.tfvars.json). # Exploited ressources | Kubernetes | Object stroage type | Storage queue type | Storage table type | OS | | :--- | :--- | :--- | :--- | :--- | -| AWS EKS "1.25" | AWS S3 | Amazon MQ, broker : ActiveMQ "5.16.4" | MongoDB "6.0.1" | Linux | +| AWS EKS "1.25" | Redis | Amazon MQ, broker : ActiveMQ "5.16.4" | MongoDB "6.0.1" | Linux | # The estimated results @@ -113,63 +113,63 @@ Faster task processing by increasing the number of pods. #### StressTest -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | | | | | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 1 | 7.79 | 997.74 | 3.63 | 274.80 | 22.67 | 44.1 | 0.34 | 23.41 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 4.53 | 8.61 | 1101.45 | 4.89 | 1022.29 | 97.3 | 51.38 | 0.40 | 98.11 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 9.91 | 7.88 | 1008.70 | 9.55 | 1046.57 | 195.72 | 51.09 | 0.40 | 196.46 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B | 96.52 | 8.09 | 1035.96 | 96.18 | 1039.62 | 1740.38 | 57.46 | 0.45 | 1741.16 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 1.81 | 4.31 | 552.26 | 8.11 | 123.30 | 23.92 | 41.80 | 0.33 | 25.06 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 3.36 | 11.59 | 1483.88 | 6.48 | 770.89 | 101 | 49.50 | 0.39 | 101.71 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 27.92 | 2.80 | 358.14 | 27.01 | 370.11 | 363 | 27.55 | 0.22 | 364.4 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| 240.66 | 3.25 | 415.52 | 240.05 | 416.58 | 2661.68 | 37.57 | 0.29 | 2662.67 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | #### Bench -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | | | | | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Total time (s)| +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 5.32 | 10.81 | 195.54 | 2.79 | 19.21 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 21.50 | 57.81 | 233.44 | 15.71 | 95.05 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 48.14 | 174.32 | 208.32 | 51.63 | 274.14 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mii | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 7.35 | 14.90 | 131.32 | 3.87 | 26.57 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 24.21 | 71.84 | 207.46 | 19.16 | 114.27 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| 49.91 | 129.70 | 200.73 | 33.39 | 213.04 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | #### HtcMock | Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Aggregation level | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | | :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CCPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | ## Submission scalability tests @@ -180,23 +180,23 @@ Faster task processing by increasing the number of pods. #### StressTest -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1KB | 5 | 1 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10KB | 5 | 5 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 100KB | 5 | 10 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1KB | 5 | 1 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 10KB | 5 | 5 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100KB | 5 | 10 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1KB | 5 | 1 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 10KB | 5 | 5 | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 100KB | 5 | 10 | | | | | | | | | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1KB | 5 | 1 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi |CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10KB | 5 | 5 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 100KB | 5 | 10 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1KB | 5 | 1 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 10KB | 5 | 5 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100KB | 5 | 10 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1KB | 5 | 1 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 10KB | 5 | 5 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 100KB | 5 | 10 | | | | | | | | | | ## Other tests -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 80K | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1 | CPU: 1000m / Memory: 2048Mi | 300M | 1s | 1KB | 1KB | | | | | | | | | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 80K | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | 1 | CPU: 1000m / Memory: 2048Mi | 300M | 1s | 1KB | 1KB | | | | | | | | | | This document will be completed in the future with more tests to target more detailed and deeper ArmoniK's features and components. \ No newline at end of file From 44a52e5b65357f05efc8469f3bd1cfe059019080 Mon Sep 17 00:00:00 2001 From: Yanis KHAROUNI Date: Wed, 10 May 2023 10:02:34 +0200 Subject: [PATCH 4/9] fix date --- .docs/test_plan.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.docs/test_plan.md b/.docs/test_plan.md index 637b86daf..56e8bb247 100644 --- a/.docs/test_plan.md +++ b/.docs/test_plan.md @@ -69,7 +69,7 @@ These tests are dependent of the configuration of the underlying infrastructure | | | | ------ | ----- | -| **Date** | 03/29/2023 | +| **Date** | 05/01/2023 | | **Infra version** | 2.12.3 | | **Core version** | 0.12.4 | | **API version** | 3.6.0 | From e03c9c4d83b68a913916dd6a1142f5f6de18b851 Mon Sep 17 00:00:00 2001 From: Yanis KHAROUNI Date: Wed, 10 May 2023 10:07:18 +0200 Subject: [PATCH 5/9] fix date --- .docs/test_plan.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.docs/test_plan.md b/.docs/test_plan.md index 6bf13c59c..5c8f5a4fa 100644 --- a/.docs/test_plan.md +++ b/.docs/test_plan.md @@ -69,7 +69,7 @@ These tests are dependent of the configuration of the underlying infrastructure | | | | ------ | ----- | -| **Date** | 03/29/2023 | +| **Date** | 05/01/2023 | | **Infra version** | 2.12.3 | | **Core version** | 0.12.4 | | **API version** | 3.5.2 | From 2590dc8d7da64cf6ae8f085f064a44c5a7493e8b Mon Sep 17 00:00:00 2001 From: Yanis KHAROUNI Date: Mon, 15 May 2023 09:58:41 +0200 Subject: [PATCH 6/9] add benchmarking stats folder and set test plan into template --- .docs/benchmarking/stats_v2.12.3.md | 206 ++++++++++++++++++++++++++++ .docs/test_plan.md | 193 +++++++++++++------------- 2 files changed, 303 insertions(+), 96 deletions(-) create mode 100644 .docs/benchmarking/stats_v2.12.3.md diff --git a/.docs/benchmarking/stats_v2.12.3.md b/.docs/benchmarking/stats_v2.12.3.md new file mode 100644 index 000000000..5b5dbd880 --- /dev/null +++ b/.docs/benchmarking/stats_v2.12.3.md @@ -0,0 +1,206 @@ + # Test plan of Armonik + +- [Introduction](#Introduction) +- [The product and the functionalities to test](#the-product-and-the-functionalities-to-test) +- [The tools used to do the tests](#the-tools-used-to-do-the-tests) +- [Tests environment](#Tests-environment) +- [Exploited ressources](#exploited-ressources) +- [The estimated results](#the-estimated-results) +- [Tests to do](#tests-to-do) + +# Introduction + +In this document we describe a performance test plan of ArmoniK. The purpose is to measure the performance of the different functionalities of ArmoniK (task submissions, processing of tasks, ...). We will perform the following tests: +* Measure the execution time of different numbers of submitted tasks and different size of the tasks. +* The strong scaling. + +The purpose of these tests is to have performance measurements based on ArmoniK versions. As this is the first iteration of the performance tests, we will establish our baseline based on a specific version (see hereafter). + +# The product and the functionalities to test + +## Products + +### Stresstest + +This is a stress test which runs a number of tasks with a specific I/O sizes and task workloads. It uses ArmoniK C# extension which copies the DLLs of the application as a zip and installs them on the workers. + +### Bench + +This test is similar to StressTest but it doesn't use ArmoniK C# extension. The application is already installed on the workers. + +### HtcMock + +With this test we can have an idea of how ArmoniK can handle the applications with subtasking and aggregation tasks. + +## functionalities + +We want to test in this first iteration, the three main functionnalities of ArmoniK described below. + +### The submission of the tasks + +With these tests, we want to measure the performances of tasks submissions od ArmoniK. This is the first step of the task ochestration which consists in submitting the tasks and the input data to be processed by ArmoniK workers. + +### The processing of the tasks + +In addition to the tasks submission, these tests will allow us to measure the ArmoniK ability to process different numbers of tasks with different workload durations. + +### The retrieving of the results + +These tests will allow us to determine ArmoniK performances to retrieve the results data after tasks processing. This step consists in the recovery of the output data after the processing of the tasks and it is the last step of the task orchestration. + +# The tools used to do the tests + +## Bash scripts + +The scripts start a warming up run, launch a batch of runs with the same parameters, and store performance data in files. + +## Python + +* Clean the files of the performance data and store them in readable Json files. +* Compute the median and the mean of each batch of runs +## Json + +We store the results of the performance tests in Json files which we will be stored in a database. + +# Tests environment + +These tests are dependent of the configuration of the underlying infrastructure and are a prerequirement of every test execution used as comparison. This method could be reused as reproducer for anyone desiring to execute our tests. + + +| | | +| ------ | ----- | +| **Date** | 05/01/2023 | +| **Infra version** | 2.12.3 | +| **Core version** | 0.12.4 | +| **API version** | 3.5.2 | +| **Extension c# version** | 0.9.5 | +| **Metrics exporter** | 0.12.4 | +| **StressTest Client** | 2.12.3 | +| **Bench Client** | 0.12.4 | +| **HtcMock Client** | 0.12.4 | +| **Instance type** | c24.xlarge | +| **Processor** | Intel Xeon de 2e (Cascade Lake) | +| **CPU frequency** | 3.6 GHz - 3.9 GHz | +| **nb vCPUs** | 96 | +| **RAM (GB)** | 192 | +| **Network bandwidth (Gbps)** | 25 | +| **EBS Bandwidth (Mbps)** | 19000 | + + +The versions of the different ArmoniK components and the third-party tools are defined in [versions.tfvars.json](https://github.com/aneoconsulting/ArmoniK/blob/v2.12.3/versions.tfvars.json). + +# Exploited ressources + +| Kubernetes | Object stroage type | Storage queue type | Storage table type | OS | +| :--- | :--- | :--- | :--- | :--- | +| AWS EKS "1.25" | Redis | Amazon MQ, broker : ActiveMQ "5.16.4" | MongoDB "6.0.1" | Linux | + +# The estimated results + +Faster task processing by increasing the number of pods. + +# Tests to do + +## Throughput scalability tests + +- 1000 tasks on : 100 pods, 1000 pods and 10000 pods +- 5000 tasks on : 100 pods, 1000 pods and 10000 pods +- 10K tasks on : 100 pods, 1000 pods and 10000 pods +- 100K tasks on : 100 pods, 1000 pods and 10000 pods +- 1M tasks on : 100 pods, 1000 pods and 10000 pods + +### with + +- Tasks duration : 1 ms +- Input payload size : 8B +- Output payload size : 8B + +#### StressTest + +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 1 | 7.79 | 997.74 | 3.63 | 274.80 | 22.67 | 44.1 | 0.34 | 23.41 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 4.53 | 8.61 | 1101.45 | 4.89 | 1022.29 | 97.3 | 51.38 | 0.40 | 98.11 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 9.91 | 7.88 | 1008.70 | 9.55 | 1046.57 | 195.72 | 51.09 | 0.40 | 196.46 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B | 96.52 | 8.09 | 1035.96 | 96.18 | 1039.62 | 1740.38 | 57.46 | 0.45 | 1741.16 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 1.81 | 4.31 | 552.26 | 8.11 | 123.30 | 23.92 | 41.80 | 0.33 | 25.06 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 3.36 | 11.59 | 1483.88 | 6.48 | 770.89 | 101 | 49.50 | 0.39 | 101.71 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 27.92 | 2.80 | 358.14 | 27.01 | 370.11 | 363 | 27.55 | 0.22 | 364.4 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| 240.66 | 3.25 | 415.52 | 240.05 | 416.58 | 2661.68 | 37.57 | 0.29 | 2662.67 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | + +#### Bench + +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Total time (s)| +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 5.32 | 10.81 | 195.54 | 2.79 | 19.21 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 21.50 | 57.81 | 233.44 | 15.71 | 95.05 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 48.14 | 174.32 | 208.32 | 51.63 | 274.14 | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mii | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 7.35 | 14.90 | 131.32 | 3.87 | 26.57 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 24.21 | 71.84 | 207.46 | 19.16 | 114.27 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| 49.91 | 129.70 | 200.73 | 33.39 | 213.04 | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | + +#### HtcMock + +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Aggregation level | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CCPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | +| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | +| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | + + +## Submission scalability tests + +- Number of pods : 1000 +- Tasks duration : 500 ms +- Output payload size : 8B + +#### StressTest + +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1KB | 5 | 1 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi |CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10KB | 5 | 5 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 100KB | 5 | 10 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1KB | 5 | 1 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 10KB | 5 | 5 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100KB | 5 | 10 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1KB | 5 | 1 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 10KB | 5 | 5 | | | | | | | | | | +| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 100KB | 5 | 10 | | | | | | | | | | + + +## Other tests + +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 80K | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | 1 | CPU: 1000m / Memory: 2048Mi | 300M | 1s | 1KB | 1KB | | | | | | | | | | + +This document will be completed in the future with more tests to target more detailed and deeper ArmoniK's features and components. \ No newline at end of file diff --git a/.docs/test_plan.md b/.docs/test_plan.md index 5c8f5a4fa..15ea43a19 100644 --- a/.docs/test_plan.md +++ b/.docs/test_plan.md @@ -1,9 +1,9 @@ - # Test plan of Armonik +# Test plan of Armonik -- [Introduction](#Introduction) +- [Introduction](#introduction) - [The product and the functionalities to test](#the-product-and-the-functionalities-to-test) - [The tools used to do the tests](#the-tools-used-to-do-the-tests) -- [Tests environment](#Tests-environment) +- [Tests environment](#tests-environment) - [Exploited ressources](#exploited-ressources) - [The estimated results](#the-estimated-results) - [Tests to do](#tests-to-do) @@ -11,8 +11,8 @@ # Introduction In this document we describe a performance test plan of ArmoniK. The purpose is to measure the performance of the different functionalities of ArmoniK (task submissions, processing of tasks, ...). We will perform the following tests: -* Measure the execution time of different numbers of submitted tasks and different size of the tasks. -* The strong scaling. +- Measure the execution time of different numbers of submitted tasks and different size of the tasks. +- The strong scaling. The purpose of these tests is to have performance measurements based on ArmoniK versions. As this is the first iteration of the performance tests, we will establish our baseline based on a specific version (see hereafter). @@ -44,7 +44,7 @@ With these tests, we want to measure the performances of tasks submissions od Ar In addition to the tasks submission, these tests will allow us to measure the ArmoniK ability to process different numbers of tasks with different workload durations. -### The retrieving of the results +### The retrieving of the results These tests will allow us to determine ArmoniK performances to retrieve the results data after tasks processing. This step consists in the recovery of the output data after the processing of the tasks and it is the last step of the task orchestration. @@ -56,40 +56,43 @@ The scripts start a warming up run, launch a batch of runs with the same paramet ## Python -* Clean the files of the performance data and store them in readable Json files. -* Compute the median and the mean of each batch of runs -## Json +- Clean the files of the performance data and store them in readable JSON files. +- Compute the median and the mean of each batch of runs -We store the results of the performance tests in Json files which we will be stored in a database. +## JSON + +We store the results of the performance tests in JSON files which we will be stored in a database. # Tests environment These tests are dependent of the configuration of the underlying infrastructure and are a prerequirement of every test execution used as comparison. This method could be reused as reproducer for anyone desiring to execute our tests. - | | | | ------ | ----- | -| **Date** | 05/01/2023 | -| **Infra version** | 2.12.3 | -| **Core version** | 0.12.4 | -| **API version** | 3.5.2 | -| **Extension c# version** | 0.9.5 | -| **Instance type** | c24.xlarge | -| **Processor** | Intel Xeon de 2e (Cascade Lake) | -| **CPU frequency** | 3.6 GHz - 3.9 GHz | -| **nb vCPUs** | 96 | -| **RAM (GB)** | 192 | -| **Network bandwidth (Gbps)** | 25 | -| **EBS Bandwidth (Mbps)** | 19000 | - - -The versions of the different ArmoniK components and the third-party tools are defined in [versions.tfvars.json](https://github.com/aneoconsulting/ArmoniK/blob/v2.12.3/versions.tfvars.json). +| **Date** | | +| **Infra version** | | +| **Core version** | | +| **API version** | | +| **Extension c# version** | | +| **Metrics exporter** | | +| **StressTest Client** | | +| **Bench Client** | | +| **HtcMock Client** | | +| **Instance type** | | +| **Processor** | | +| **CPU frequency** | | +| **nb vCPUs** | | +| **RAM (GB)** | | +| **Network bandwidth (Gbps)** | | +| **EBS Bandwidth (Mbps)** | | + +The versions of the different ArmoniK components and the third-party tools are defined in versions.tfvars.json # Exploited ressources | Kubernetes | Object stroage type | Storage queue type | Storage table type | OS | | :--- | :--- | :--- | :--- | :--- | -| AWS EKS "1.25" | Redis | Amazon MQ, broker : ActiveMQ "5.16.4" | MongoDB "6.0.1" | Linux | +| | | | | | # The estimated results @@ -113,64 +116,63 @@ Faster task processing by increasing the number of pods. #### StressTest -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 1 | 7.79 | 997.74 | 3.63 | 274.80 | 22.67 | 44.1 | 0.34 | 23.41 | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 4.53 | 8.61 | 1101.45 | 4.89 | 1022.29 | 97.3 | 51.38 | 0.40 | 98.11 | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 9.91 | 7.88 | 1008.70 | 9.55 | 1046.57 | 195.72 | 51.09 | 0.40 | 196.46 | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B | 96.52 | 8.09 | 1035.96 | 96.18 | 1039.62 | 1740.38 | 57.46 | 0.45 | 1741.16 | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 1.81 | 4.31 | 552.26 | 8.11 | 123.30 | 23.92 | 41.80 | 0.33 | 25.06 | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 3.36 | 11.59 | 1483.88 | 6.48 | 770.89 | 101 | 49.50 | 0.39 | 101.71 | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 27.92 | 2.80 | 358.14 | 27.01 | 370.11 | 363 | 27.55 | 0.22 | 364.4 | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| 240.66 | 3.25 | 415.52 | 240.05 | 416.58 | 2661.68 | 37.57 | 0.29 | 2662.67 | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1K | 1 ms | 8B | 8B | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 5K | 1 ms | 8B | 8B | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 1 ms | 8B | 8B | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 1 ms | 8B | 8B| | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1 ms | 8B | 8B | | | | | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1K | 1 ms | 8B | 8B | | | | | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 5K | 1 ms | 8B | 8B | | | | | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 1 ms | 8B | 8B| | | | | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 1 ms | 8B | 8B| | | | | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1 ms | 8B | 8B | | | | | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1K | 1 ms | 8B | 8B | | | | | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 5K | 1 ms | 8B | 8B | | | | | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 1 ms | 8B | 8B| | | | | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 1 ms | 8B | 8B| | | | | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1 ms | 8B | 8B | | | | | | | | | #### Bench -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Total time (s)| -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 5.32 | 10.81 | 195.54 | 2.79 | 19.21 | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 21.50 | 57.81 | 233.44 | 15.71 | 95.05 | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B | 48.14 | 174.32 | 208.32 | 51.63 | 274.14 | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mii | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | 7.35 | 14.90 | 131.32 | 3.87 | 26.57 | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | 24.21 | 71.84 | 207.46 | 19.16 | 114.27 | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| 49.91 | 129.70 | 200.73 | 33.39 | 213.04 | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1 ms | 8B | 8B| **-** | **-** | **-** | **-** | **-** | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1 ms | 8B | 8B | **-** | **-** | **-** | **-** | **-** | +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1K | 1 ms | 8B | 8B | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 5K | 1 ms | 8B | 8B | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 1 ms | 8B | 8B | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 1 ms | 8B | 8B| | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1K | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 5K | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 1 ms | 8B | 8B| | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 1 ms | 8B | 8B| | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1K | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 5K | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 1 ms | 8B | 8B| | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 1 ms | 8B | 8B| | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1 ms | 8B | 8B | | | | | #### HtcMock | Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Aggregation level | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CCPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | -| 1000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1K | 1 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 5K | 5 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10 | 1 ms | 8B | 8B | | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100 | 1 ms | 8B | 8B| | | | | -| 10000 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1000 | 1 ms | 8B | 8B | | | | | - +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1K | 1 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 5K | 5 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 10 | 1 ms | 8B | 8B | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 100 | 1 ms | 8B | 8B| | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1000 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1K | 1 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 5K | 5 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 10 | 1 ms | 8B | 8B | | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 100 | 1 ms | 8B | 8B| | | | | +| 1000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1000 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1K | 1 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 5K | 5 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 10 | 1 ms | 8B | 8B | | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 100 | 1 ms | 8B | 8B| | | | | +| 10000 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1000 | 1 ms | 8B | 8B | | | | | ## Submission scalability tests @@ -178,25 +180,24 @@ Faster task processing by increasing the number of pods. - Tasks duration : 500 ms - Output payload size : 8B -#### StressTest - -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 1KB | 5 | 1 | | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi |CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 10KB | 5 | 5 | | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 10K | 100KB | 5 | 10 | | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 1KB | 5 | 1 | | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 10KB | 5 | 5 | | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 100K | 100KB | 5 | 10 | | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi| CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 1KB | 5 | 1 | | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi |CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m/ Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 10KB | 5 | 5 | | | | | | | | | | -| 100 | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | CPU: 500m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | 1M | 100KB | 5 | 10 | | | | | | | | | | +## StressTest +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 1KB | 5 | 1 | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 10KB | 5 | 5 | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 10K | 100KB | 5 | 10 | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 1KB | 5 | 1 | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 10KB | 5 | 5 | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 100K | 100KB | 5 | 10 | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 1KB | 5 | 1 | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 10KB | 5 | 5 | | | | | | | | | +| 100 | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1M | 100KB | 5 | 10 | | | | | | | | | ## Other tests - -| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | Total time (s) | -| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 80K | CPU: 200m / Memory: 512Mi | CPU: 1000m / Memory: 2048Mi | CPU: 1000m / Memory: 256Mi | CPU: 2000m / Memory: 2048Mi | 1 | CPU: 1000m / Memory: 2048Mi | 300M | 1s | 1KB | 1KB | | | | | | | | | | -This document will be completed in the future with more tests to target more detailed and deeper ArmoniK's features and components. \ No newline at end of file +| Number of pods | Resource requests per control-plane| Resource limits per control-plane| Resource requests per scheduling-agent| Resource limits per scheduling-agent| Resource requests per worker | Resource limits for per worker | Number of tasks | Task workload duration | Input payload size | Output payload size | Duration of submissions (s) | Upload speed (s) | Throughtput for submissions (tasks/s) | Duration of processing (s) | Throughput for processing (tasks/s) | Duration of retrieving results (s) | Throughtput for retrieving results (tasks/s) | Download speed (s) | +| :--- | :--- | :--- | :--- | :--- | :----: | ---: | :----: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 80K | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | CPU: / Memory: | 1 | CPU: / Memory: | 300M | 1s | 1KB | 1KB | | | | | | | | | + +This document will be completed in the future with more tests to target more detailed and deeper ArmoniK's features and components. From e7dbfa4edf14916e13858a53a75860daaa9048cf Mon Sep 17 00:00:00 2001 From: Yanis KHAROUNI Date: Wed, 17 May 2023 09:37:56 +0200 Subject: [PATCH 7/9] add csv stats file of version 2.12.3 --- .docs/benchmarking/stats_v2.12.3.csv | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 .docs/benchmarking/stats_v2.12.3.csv diff --git a/.docs/benchmarking/stats_v2.12.3.csv b/.docs/benchmarking/stats_v2.12.3.csv new file mode 100644 index 000000000..7431e7611 --- /dev/null +++ b/.docs/benchmarking/stats_v2.12.3.csv @@ -0,0 +1,3 @@ +id;date;test_type;armonik_version;control_plane_version;polling_agent_version;worker_version;metrics_exporter_version;api_version;client_version;control_plane_cpu_request;control_plane_cpu_limit;control_plane_memory_request;control_plane_memory_limit;polling_agent_cpu_request;polling_agent_cpu_limit;polling_agent_memory_request;polling_agent_memory_limit;worker_cpu_request;worker_cpu_limit;worker_memory_request;worker_memory_limit;kubernetes_type;kuberntetes_version;object_storage_type;object_storage_version;storage_queue_type;storage_queue_version;storage_table_type;storage_table_version;shared_storage_type;shared_storage_version;os;cloud_region;instance_type;processor;cpu_frequency;nb_vcpus;ram_gb;network_bandwidth_gbps;ebs_bandwidth_mbps;number_tasks;tasks_workload_duration_s;input_payload_size_b;output_payload_size_b;duration_submission_s;upload_speed;throughpu_submission_ts;processing_duration_s;processing_throughput_ts;retrieving_duration_s;retrieving_throughput;download_speed;total_time_s +0;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;0.001;8;8;1;7.79;997.74;3.63;274.80;22.67;44.1;0.34;23.41 +0;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;;;;;;;;;;;;; From e2319616e739b5ef0b079d5849cee65cc98d8452 Mon Sep 17 00:00:00 2001 From: Yanis KHAROUNI Date: Wed, 17 May 2023 09:52:41 +0200 Subject: [PATCH 8/9] move csv file --- .docs/benchmarking/stats_v2.12.3.csv | 3 --- 1 file changed, 3 deletions(-) delete mode 100644 .docs/benchmarking/stats_v2.12.3.csv diff --git a/.docs/benchmarking/stats_v2.12.3.csv b/.docs/benchmarking/stats_v2.12.3.csv deleted file mode 100644 index 7431e7611..000000000 --- a/.docs/benchmarking/stats_v2.12.3.csv +++ /dev/null @@ -1,3 +0,0 @@ -id;date;test_type;armonik_version;control_plane_version;polling_agent_version;worker_version;metrics_exporter_version;api_version;client_version;control_plane_cpu_request;control_plane_cpu_limit;control_plane_memory_request;control_plane_memory_limit;polling_agent_cpu_request;polling_agent_cpu_limit;polling_agent_memory_request;polling_agent_memory_limit;worker_cpu_request;worker_cpu_limit;worker_memory_request;worker_memory_limit;kubernetes_type;kuberntetes_version;object_storage_type;object_storage_version;storage_queue_type;storage_queue_version;storage_table_type;storage_table_version;shared_storage_type;shared_storage_version;os;cloud_region;instance_type;processor;cpu_frequency;nb_vcpus;ram_gb;network_bandwidth_gbps;ebs_bandwidth_mbps;number_tasks;tasks_workload_duration_s;input_payload_size_b;output_payload_size_b;duration_submission_s;upload_speed;throughpu_submission_ts;processing_duration_s;processing_throughput_ts;retrieving_duration_s;retrieving_throughput;download_speed;total_time_s -0;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;0.001;8;8;1;7.79;997.74;3.63;274.80;22.67;44.1;0.34;23.41 -0;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;;;;;;;;;;;;; From 9ea20c4c0088b1e2b29557393540693833df5f2d Mon Sep 17 00:00:00 2001 From: Yanis KHAROUNI Date: Wed, 17 May 2023 09:56:02 +0200 Subject: [PATCH 9/9] add benchmarking csv file --- benchmarking/benchmarking_stats.csv | 31 +++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 benchmarking/benchmarking_stats.csv diff --git a/benchmarking/benchmarking_stats.csv b/benchmarking/benchmarking_stats.csv new file mode 100644 index 000000000..5687d87f9 --- /dev/null +++ b/benchmarking/benchmarking_stats.csv @@ -0,0 +1,31 @@ +id;date;test_type;armonik_version;control_plane_version;polling_agent_version;worker_version;metrics_exporter_version;api_version;client_version;control_plane_cpu_request;control_plane_cpu_limit;control_plane_memory_request;control_plane_memory_limit;polling_agent_cpu_request;polling_agent_cpu_limit;polling_agent_memory_request;polling_agent_memory_limit;worker_cpu_request;worker_cpu_limit;worker_memory_request;worker_memory_limit;kubernetes_type;kuberntetes_version;object_storage_type;object_storage_version;storage_queue_type;storage_queue_version;storage_table_type;storage_table_version;shared_storage_type;shared_storage_version;os;cloud_region;instance_type;processor;cpu_frequency;nb_vcpus;ram_gb;network_bandwidth_gbps;ebs_bandwidth_mbps;nb_pods;number_tasks;tasks_workload_duration_s;input_payload_size_b;output_payload_size_b;duration_submission_s;upload_speed;throughpuT_submission_ts;processing_duration_s;processing_throughput_ts;retrieving_duration_s;retrieving_throughput;download_speed;total_time_s +0;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;1000;0.001;8;8;1;7.79;997.74;3.63;274.80;22.67;44.1;0.34;23.41 +1;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;5000;0.001;8;8;4.53;8.61;1101.45;4.89;1022.29;97.3;51.38;0.40;98.11 +2;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;10000;0.001;8;8;9.91;7.88;1008.7;9.55;1046.57;195.72;51.09;0.40;196.46 +3;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;100000;0.001;8;8;96.52;8.09;1035.96;96.18;1039.62;1740.38;57.46;0.45;1741.16 +4;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;1000000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +5;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;1000;0.001;8;8;1.81;4.31;552.26;8.11;123.30;23.92;41.80;0.33;25.06 +6;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;5000;0.001;8;8;3.36;11.59;1483.88;6.48;770.89;101;49.50;0.39;101.71 +7;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;10000;0.001;8;8;27.92;2.80;358.14;27.01;370.11;363;27.55;0.22;364.4 +8;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;100000;0.001;8;8;240.66;3.25;415.52;240.05;416.58;2661.68;37.57;0.29;2662.67 +9;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;1000000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +10;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;1000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +11;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;5000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +12;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;10000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +13;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;100000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;stresstest;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;1000000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;1000;0.001;8;8;5.32;NULL;NULL;10.81;195.54;2.79;NULL;NULL;19.21 +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;5000;0.001;8;8;21.50;NULL;NULL;57.81;233.44;15.71;NULL;NULL;95.05 +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;10000;0.001;8;8;48.14;NULL;NULL;174.32;208.32;51.63;NULL;NULL;274.14 +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;100000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;100;1000000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;1000;0.001;8;8;7.35;NULL;NULL;14.90;131.32;3.87;NULL;NULL;26.57 +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;5000;0.001;8;8;24.21;NULL;NULL;71.84;207.46;19.16;NULL;NULL;114.27 +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;10000;0.001;8;8;49.91;NULL;NULL;129.70;200.73;33.39;NULL;NULL;213.04 +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;100000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;1000;1000000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;1000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;5000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;10000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;100000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL +14;05/01/2023;bench;2.12.3;0.12.4;0.12.4;0.9.5;0.12.4;3.6.0;2.12.3;200m;1000m;500Mi;2048Mi;1000m;2000m;256Mi;2048Mi;500m;1000m;512Mi;2048Mi;eks;1.25;elasticache;6.x;amazon_mq_activemq;5.16.4;mongodb;6.0.1;s3;NULL;linux;aws_eu_west_3;c24.xlarge;Intel Xeon de 2e (Cascade Lake);3.6 GHz - 3.9 GHz;96;192;25;19000;10000;1000000;0.001;8;8;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL