Skip to content

Conversation

riquich
Copy link
Contributor

@riquich riquich commented Oct 10, 2025

Issue #, if available:

No automated performance tests are present

Description of changes:

Migrating integrated testing tools, modifying them to run through ssm, and plugging them to actions to trigger on PRs.

The tests will do following:

  1. Create m6a.xlarge,m6a.4xlarge,m6a.8xlarge,m6a.16xlarge,m6a.24xlarge,m6a.48xlarge instances
  2. Install agent on those, along with some testing tools.
  3. With X minutes intervals, run load on them and log agent outputs and host metrics.
  4. Repeat the load with increased amounts consecutively.
  5. Gather all reports from test runner hosts, create graphs on collected metrics.
  6. Attach collected graphs to PR.
  7. Compare the current resource usage metrics with previous metrics and decide on pass / fail.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@riquich riquich changed the title [Alpha] add automated performance testing Add automated performance testing Oct 10, 2025
Copy link

codecov bot commented Oct 10, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.87%. Comparing base (9af1e5b) to head (1c6354d).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #59   +/-   ##
=======================================
  Coverage   75.87%   75.87%           
=======================================
  Files          32       32           
  Lines        1981     1981           
=======================================
  Hits         1503     1503           
  Misses        478      478           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This comment was marked as duplicate.

This comment was marked as outdated.

This comment was marked as outdated.

github-actions bot added a commit that referenced this pull request Oct 13, 2025
github-actions bot added a commit that referenced this pull request Oct 13, 2025
github-actions bot added a commit that referenced this pull request Oct 14, 2025

This comment has been minimized.

github-actions bot added a commit that referenced this pull request Oct 14, 2025

This comment was marked as duplicate.

github-actions bot added a commit that referenced this pull request Oct 14, 2025

This comment was marked as duplicate.

github-actions bot added a commit that referenced this pull request Oct 14, 2025

This comment was marked as duplicate.

github-actions bot added a commit that referenced this pull request Oct 14, 2025

This comment was marked as duplicate.

github-actions bot added a commit that referenced this pull request Oct 14, 2025
github-actions bot added a commit that referenced this pull request Oct 14, 2025
Copy link

github-actions bot commented Oct 14, 2025

📊 Performance Test Results

## 📊 Performance Comparison Table

Metric Instance Type TPS Baseline Current Change Limit
BPF CPU m6a.16xlarge 10000 N/A 0.127 N/A ✅ 2.5
BPF CPU m6a.16xlarge 100000 N/A 0.266 N/A ✅ 2.5
BPF CPU m6a.16xlarge 20000 N/A 0.21 N/A ✅ 2.5
BPF CPU m6a.16xlarge 30000 N/A 0.179 N/A ✅ 2.5
BPF CPU m6a.16xlarge 40000 N/A 0.226 N/A ✅ 2.5
BPF CPU m6a.16xlarge 60000 N/A 0.215 N/A ✅ 2.5
BPF CPU m6a.16xlarge 80000 N/A 0.242 N/A ✅ 2.5
BPF CPU m6a.24xlarge 10000 N/A 0.096 N/A ✅ 3
BPF CPU m6a.24xlarge 100000 N/A 0.219 N/A ✅ 3
BPF CPU m6a.24xlarge 20000 N/A 0.143 N/A ✅ 3
BPF CPU m6a.24xlarge 30000 N/A 0.213 N/A ✅ 3
BPF CPU m6a.24xlarge 40000 N/A 0.226 N/A ✅ 3
BPF CPU m6a.24xlarge 60000 N/A 0.199 N/A ✅ 3
BPF CPU m6a.24xlarge 80000 N/A 0.253 N/A ✅ 3
BPF CPU m6a.48xlarge 10000 N/A 0.055 N/A ✅ 1.5
BPF CPU m6a.48xlarge 100000 N/A 0.138 N/A ✅ 1.5
BPF CPU m6a.48xlarge 20000 N/A 0.094 N/A ✅ 1.5
BPF CPU m6a.48xlarge 30000 N/A 0.119 N/A ✅ 1.5
BPF CPU m6a.48xlarge 40000 N/A 0.131 N/A ✅ 1.5
BPF CPU m6a.48xlarge 60000 N/A 0.139 N/A ✅ 1.5
BPF CPU m6a.48xlarge 80000 N/A 0.19 N/A ✅ 1.5
BPF CPU m6a.4xlarge 10000 N/A 0.2 N/A ✅ 4
BPF CPU m6a.4xlarge 100000 N/A 0.172 N/A ✅ 4
BPF CPU m6a.4xlarge 20000 N/A 0.174 N/A ✅ 4
BPF CPU m6a.4xlarge 30000 N/A 0.18 N/A ✅ 4
BPF CPU m6a.4xlarge 40000 N/A 0.179 N/A ✅ 4
BPF CPU m6a.4xlarge 60000 N/A 0.174 N/A ✅ 4
BPF CPU m6a.4xlarge 80000 N/A 0.178 N/A ✅ 4
BPF CPU m6a.8xlarge 10000 N/A 0.199 N/A ✅ 3
BPF CPU m6a.8xlarge 100000 N/A 0.323 N/A ✅ 3
BPF CPU m6a.8xlarge 20000 N/A 0.203 N/A ✅ 3
BPF CPU m6a.8xlarge 30000 N/A 0.221 N/A ✅ 3
BPF CPU m6a.8xlarge 40000 N/A 0.213 N/A ✅ 3
BPF CPU m6a.8xlarge 60000 N/A 0.253 N/A ✅ 3
BPF CPU m6a.8xlarge 80000 N/A 0.276 N/A ✅ 3
BPF CPU m6a.xlarge 10000 N/A 0.184 N/A ✅ 5
BPF CPU m6a.xlarge 100000 N/A 0.203 N/A ✅ 5
BPF CPU m6a.xlarge 20000 N/A 0.179 N/A ✅ 5
BPF CPU m6a.xlarge 30000 N/A 0.177 N/A ✅ 5
BPF CPU m6a.xlarge 40000 N/A 0.171 N/A ✅ 5
BPF CPU m6a.xlarge 60000 N/A 0.181 N/A ✅ 5
BPF CPU m6a.xlarge 80000 N/A 0.187 N/A ✅ 5
Memory m6a.16xlarge 10000 N/A 26423.915KB N/A ✅ 60000KB
Memory m6a.16xlarge 100000 N/A 43730.553KB N/A ✅ 60000KB
Memory m6a.16xlarge 20000 N/A 39050.383KB N/A ✅ 60000KB
Memory m6a.16xlarge 30000 N/A 40133.702KB N/A ✅ 60000KB
Memory m6a.16xlarge 40000 N/A 40140.426KB N/A ✅ 60000KB
Memory m6a.16xlarge 60000 N/A 40123.745KB N/A ✅ 60000KB
Memory m6a.16xlarge 80000 N/A 40143.574KB N/A ✅ 60000KB
Memory m6a.24xlarge 10000 N/A 27177.702KB N/A ✅ 75000KB
Memory m6a.24xlarge 100000 N/A 54469.532KB N/A ✅ 75000KB
Memory m6a.24xlarge 20000 N/A 39166.894KB N/A ✅ 75000KB
Memory m6a.24xlarge 30000 N/A 37784.255KB N/A ✅ 75000KB
Memory m6a.24xlarge 40000 N/A 40576.17KB N/A ✅ 75000KB
Memory m6a.24xlarge 60000 N/A 54573.617KB N/A ✅ 75000KB
Memory m6a.24xlarge 80000 N/A 54733.957KB N/A ✅ 75000KB
Memory m6a.48xlarge 10000 N/A 27942.128KB N/A ✅ 75000KB
Memory m6a.48xlarge 100000 N/A 55671.234KB N/A ✅ 75000KB
Memory m6a.48xlarge 20000 N/A 40090.298KB N/A ✅ 75000KB
Memory m6a.48xlarge 30000 N/A 53295.66KB N/A ✅ 75000KB
Memory m6a.48xlarge 40000 N/A 58424.766KB N/A ✅ 75000KB
Memory m6a.48xlarge 60000 N/A 55526.383KB N/A ✅ 75000KB
Memory m6a.48xlarge 80000 N/A 55633.787KB N/A ✅ 75000KB
Memory m6a.4xlarge 10000 N/A 21721.833KB N/A ✅ 35000KB
Memory m6a.4xlarge 100000 N/A 24685.25KB N/A ✅ 35000KB
Memory m6a.4xlarge 20000 N/A 23593.917KB N/A ✅ 35000KB
Memory m6a.4xlarge 30000 N/A 23815.833KB N/A ✅ 35000KB
Memory m6a.4xlarge 40000 N/A 23871.25KB N/A ✅ 35000KB
Memory m6a.4xlarge 60000 N/A 23944.417KB N/A ✅ 35000KB
Memory m6a.4xlarge 80000 N/A 23841.167KB N/A ✅ 35000KB
Memory m6a.8xlarge 10000 N/A 26079.234KB N/A ✅ 40000KB
Memory m6a.8xlarge 100000 N/A 27423.064KB N/A ✅ 40000KB
Memory m6a.8xlarge 20000 N/A 27036.936KB N/A ✅ 40000KB
Memory m6a.8xlarge 30000 N/A 27276.426KB N/A ✅ 40000KB
Memory m6a.8xlarge 40000 N/A 26914.638KB N/A ✅ 40000KB
Memory m6a.8xlarge 60000 N/A 27067.66KB N/A ✅ 40000KB
Memory m6a.8xlarge 80000 N/A 27227.745KB N/A ✅ 40000KB
Memory m6a.xlarge 10000 N/A 17849.75KB N/A ✅ 30000KB
Memory m6a.xlarge 100000 N/A 17932.851KB N/A ✅ 30000KB
Memory m6a.xlarge 20000 N/A 17536.833KB N/A ✅ 30000KB
Memory m6a.xlarge 30000 N/A 17819.25KB N/A ✅ 30000KB
Memory m6a.xlarge 40000 N/A 17722.333KB N/A ✅ 30000KB
Memory m6a.xlarge 60000 N/A 17851.5KB N/A ✅ 30000KB
Memory m6a.xlarge 80000 N/A 17789.957KB N/A ✅ 30000KB
User CPU m6a.16xlarge 10000 N/A 0.135 N/A ✅ 2.5
User CPU m6a.16xlarge 100000 N/A 0.367 N/A ✅ 2.5
User CPU m6a.16xlarge 20000 N/A 0.289 N/A ✅ 2.5
User CPU m6a.16xlarge 30000 N/A 0.24 N/A ✅ 2.5
User CPU m6a.16xlarge 40000 N/A 0.299 N/A ✅ 2.5
User CPU m6a.16xlarge 60000 N/A 0.318 N/A ✅ 2.5
User CPU m6a.16xlarge 80000 N/A 0.343 N/A ✅ 2.5
User CPU m6a.24xlarge 10000 N/A 0.093 N/A ✅ 2.5
User CPU m6a.24xlarge 100000 N/A 0.354 N/A ✅ 2.5
User CPU m6a.24xlarge 20000 N/A 0.191 N/A ✅ 2.5
User CPU m6a.24xlarge 30000 N/A 0.317 N/A ✅ 2.5
User CPU m6a.24xlarge 40000 N/A 0.333 N/A ✅ 2.5
User CPU m6a.24xlarge 60000 N/A 0.312 N/A ✅ 2.5
User CPU m6a.24xlarge 80000 N/A 0.383 N/A ✅ 2.5
User CPU m6a.48xlarge 10000 N/A 0.053 N/A ✅ 2.5
User CPU m6a.48xlarge 100000 N/A 0.192 N/A ✅ 2.5
User CPU m6a.48xlarge 20000 N/A 0.104 N/A ✅ 2.5
User CPU m6a.48xlarge 30000 N/A 0.167 N/A ✅ 2.5
User CPU m6a.48xlarge 40000 N/A 0.175 N/A ✅ 2.5
User CPU m6a.48xlarge 60000 N/A 0.168 N/A ✅ 2.5
User CPU m6a.48xlarge 80000 N/A 0.211 N/A ✅ 2.5
User CPU m6a.4xlarge 10000 N/A 0.285 N/A ✅ 2.5
User CPU m6a.4xlarge 100000 N/A 0.285 N/A ✅ 2.5
User CPU m6a.4xlarge 20000 N/A 0.272 N/A ✅ 2.5
User CPU m6a.4xlarge 30000 N/A 0.283 N/A ✅ 2.5
User CPU m6a.4xlarge 40000 N/A 0.291 N/A ✅ 2.5
User CPU m6a.4xlarge 60000 N/A 0.277 N/A ✅ 2.5
User CPU m6a.4xlarge 80000 N/A 0.284 N/A ✅ 2.5
User CPU m6a.8xlarge 10000 N/A 0.273 N/A ✅ 2.5
User CPU m6a.8xlarge 100000 N/A 0.321 N/A ✅ 2.5
User CPU m6a.8xlarge 20000 N/A 0.283 N/A ✅ 2.5
User CPU m6a.8xlarge 30000 N/A 0.299 N/A ✅ 2.5
User CPU m6a.8xlarge 40000 N/A 0.287 N/A ✅ 2.5
User CPU m6a.8xlarge 60000 N/A 0.307 N/A ✅ 2.5
User CPU m6a.8xlarge 80000 N/A 0.298 N/A ✅ 2.5
User CPU m6a.xlarge 10000 N/A 0.275 N/A ✅ 2.5
User CPU m6a.xlarge 100000 N/A 0.282 N/A ✅ 2.5
User CPU m6a.xlarge 20000 N/A 0.28 N/A ✅ 2.5
User CPU m6a.xlarge 30000 N/A 0.274 N/A ✅ 2.5
User CPU m6a.xlarge 40000 N/A 0.273 N/A ✅ 2.5
User CPU m6a.xlarge 60000 N/A 0.272 N/A ✅ 2.5
User CPU m6a.xlarge 80000 N/A 0.277 N/A ✅ 2.5
User CPU Single Core m6a.16xlarge 10000 N/A 18.176000000000002 N/A ✅ 60
User CPU Single Core m6a.16xlarge 100000 N/A 41.856 N/A ✅ 60
User CPU Single Core m6a.16xlarge 20000 N/A 34.048 N/A ✅ 60
User CPU Single Core m6a.16xlarge 30000 N/A 28.352 N/A ✅ 60
User CPU Single Core m6a.16xlarge 40000 N/A 35.456 N/A ✅ 60
User CPU Single Core m6a.16xlarge 60000 N/A 36.992 N/A ✅ 60
User CPU Single Core m6a.16xlarge 80000 N/A 42.24 N/A ✅ 60
User CPU Single Core m6a.24xlarge 10000 N/A 20.160000000000004 N/A ✅ 80
User CPU Single Core m6a.24xlarge 100000 N/A 59.616 N/A ✅ 80
User CPU Single Core m6a.24xlarge 20000 N/A 34.464 N/A ✅ 80
User CPU Single Core m6a.24xlarge 30000 N/A 54.336000000000006 N/A ✅ 80
User CPU Single Core m6a.24xlarge 40000 N/A 57.215999999999994 N/A ✅ 80
User CPU Single Core m6a.24xlarge 60000 N/A 50.976 N/A ✅ 80
User CPU Single Core m6a.24xlarge 80000 N/A 64.224 N/A ✅ 80
User CPU Single Core m6a.48xlarge 10000 N/A 24.576 N/A ✅ 100
User CPU Single Core m6a.48xlarge 100000 N/A 72 N/A ✅ 100
User CPU Single Core m6a.48xlarge 20000 N/A 43.968 N/A ✅ 100
User CPU Single Core m6a.48xlarge 30000 N/A 60.288 N/A ✅ 100
User CPU Single Core m6a.48xlarge 40000 N/A 67.008 N/A ✅ 100
User CPU Single Core m6a.48xlarge 60000 N/A 66.624 N/A ✅ 100
User CPU Single Core m6a.48xlarge 80000 N/A 86.784 N/A ✅ 100
User CPU Single Core m6a.4xlarge 10000 N/A 10.368 N/A ✅ 30
User CPU Single Core m6a.4xlarge 100000 N/A 13.488 N/A ✅ 30
User CPU Single Core m6a.4xlarge 20000 N/A 10.16 N/A ✅ 30
User CPU Single Core m6a.4xlarge 30000 N/A 10.608 N/A ✅ 30
User CPU Single Core m6a.4xlarge 40000 N/A 10.687999999999999 N/A ✅ 30
User CPU Single Core m6a.4xlarge 60000 N/A 11.28 N/A ✅ 30
User CPU Single Core m6a.4xlarge 80000 N/A 13.36 N/A ✅ 30
User CPU Single Core m6a.8xlarge 10000 N/A 16.032 N/A ✅ 40
User CPU Single Core m6a.8xlarge 100000 N/A 21.76 N/A ✅ 40
User CPU Single Core m6a.8xlarge 20000 N/A 16.576 N/A ✅ 40
User CPU Single Core m6a.8xlarge 30000 N/A 17.728 N/A ✅ 40
User CPU Single Core m6a.8xlarge 40000 N/A 18.240000000000002 N/A ✅ 40
User CPU Single Core m6a.8xlarge 60000 N/A 19.200000000000003 N/A ✅ 40
User CPU Single Core m6a.8xlarge 80000 N/A 20.48 N/A ✅ 40
User CPU Single Core m6a.xlarge 10000 N/A 3.192 N/A ✅ 20
User CPU Single Core m6a.xlarge 100000 N/A 2.98 N/A ✅ 20
User CPU Single Core m6a.xlarge 20000 N/A 2.848 N/A ✅ 20
User CPU Single Core m6a.xlarge 30000 N/A 3.216 N/A ✅ 20
User CPU Single Core m6a.xlarge 40000 N/A 3.352 N/A ✅ 20
User CPU Single Core m6a.xlarge 60000 N/A 3.1159999999999997 N/A ✅ 20
User CPU Single Core m6a.xlarge 80000 N/A 3.168 N/A ✅ 20

[16:40:17] 🔴 = Significant increase (>10%) 🟢 = Significant improvement (>10% decrease) ⚠️ = Exceeds performance limit
Performance tests completed successfully! Generated 47 performance graphs.

📊 Performance Comparison Graphs


###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###
###

📎 Download Performance Reports (JSON files with detailed metrics)

Copy link
Contributor

@nahuelfilipuzzi nahuelfilipuzzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@riquich riquich merged commit afce63c into main Oct 15, 2025
8 checks passed
@riquich riquich deleted the perf_tests branch October 15, 2025 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants