Script that recalibrates performance benchmarks

[igor-aptos]

Description

Move performance benchmark values into tsv files, and add script that updates them.

How Has This Been Tested?

used the script to update values based on stable runs over the holidays.

Type of Change

• New feature
• Bug fix
• Breaking change
• Performance improvement
• Refactoring
• Dependency update
• Documentation update
• Tests

Which Components or Systems Does This Change Impact?

• Validator Node
• Full Node (API, Indexer, etc.)
• Move/Aptos Virtual Machine
• Aptos Framework
• Aptos CLI/SDK
• Developer Infrastructure
• Move Compiler
• Other (specify)

Checklist

• I have read and followed the CONTRIBUTING doc
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I identified and added all stakeholders and component owners affected by this change as reviewers
• I tested both happy and unhappy path of the functionality
• I have made corresponding changes to the documentation

[trunk-io]

⏱️ 4h 41m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
execution-performance / single-node-performance	3h 5m	🟥 🟩 🟩 🟩
forge-e2e-test / forge	14m	🟩
execution-performance / test-target-determinator	13m	🟩 🟩 🟩 🟩
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
rust-move-tests	12m	🟩
test-target-determinator	7m	🟩 ⬜ 🟩
rust-cargo-deny	5m	🟩 🟩 ⬜ 🟩
rust-doc-tests	5m	🟩
check-dynamic-deps	5m	🟩 🟩 🟩 🟩 🟩
check	4m	🟩
semgrep/ci	2m	🟩 🟩 🟩 🟩 🟩
fetch-last-released-docker-image-tag	1m	🟩
general-lints	1m	🟩 🟩 ⬜ 🟩
file_change_determinator	38s	🟩 🟩 🟩 🟩

🚨 3 jobs on the last run were significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
execution-performance / single-node-performance	45m	22m
execution-performance / test-target-determinator	3m	4m
test-target-determinator	3m	4m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[msmouse]

Thanks boss

[github-actions]

✅ Forge suite compat success on 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8

Compatibility test results for 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8 (PR)
1. Check liveness of validators at old version: 010570d3b7aa20889fb5ad0e5b23800aa33f5634
compatibility::simple-validator-upgrade::liveness-check : committed: 14175.06 txn/s, latency: 2427.89 ms, (p50: 1800 ms, p70: 1900, p90: 3900 ms, p99: 14200 ms), latency samples: 498400
2. Upgrading first Validator to new version: e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7455.56 txn/s, latency: 3877.70 ms, (p50: 4300 ms, p70: 4500, p90: 4600 ms, p99: 4700 ms), latency samples: 141160
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6865.59 txn/s, latency: 4690.10 ms, (p50: 4700 ms, p70: 4800, p90: 6700 ms, p99: 6900 ms), latency samples: 229020
3. Upgrading rest of first batch to new version: e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6640.15 txn/s, latency: 4245.59 ms, (p50: 4600 ms, p70: 4800, p90: 5600 ms, p99: 5900 ms), latency samples: 127800
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7169.50 txn/s, latency: 4520.84 ms, (p50: 4700 ms, p70: 4800, p90: 6400 ms, p99: 6900 ms), latency samples: 238800
4. upgrading second batch to new version: e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 11153.47 txn/s, latency: 2493.46 ms, (p50: 2600 ms, p70: 2900, p90: 3400 ms, p99: 3600 ms), latency samples: 192800
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10653.02 txn/s, latency: 2894.41 ms, (p50: 2700 ms, p70: 3300, p90: 4100 ms, p99: 5300 ms), latency samples: 348680
5. check swarm health
Compatibility test for 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8

Compatibility test results for 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8 (PR)
Upgrade the nodes to version: e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1324.89 txn/s, submitted: 1327.40 txn/s, failed submission: 2.51 txn/s, expired: 2.51 txn/s, latency: 2385.52 ms, (p50: 2000 ms, p70: 2400, p90: 4400 ms, p99: 6000 ms), latency samples: 116100
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1375.97 txn/s, submitted: 1378.73 txn/s, failed submission: 2.76 txn/s, expired: 2.76 txn/s, latency: 2311.03 ms, (p50: 1800 ms, p70: 2400, p90: 4200 ms, p99: 5700 ms), latency samples: 119560
5. check swarm health
Compatibility test for 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8 passed
Upgrade the remaining nodes to version: e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1373.23 txn/s, submitted: 1376.59 txn/s, failed submission: 3.37 txn/s, expired: 3.37 txn/s, latency: 2201.21 ms, (p50: 2100 ms, p70: 2300, p90: 3400 ms, p99: 5400 ms), latency samples: 122300
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite realistic_env_max_load success on e4d29e8afcfed4ab0fb493f8889a6832bf65f3b8

two traffics test: inner traffic : committed: 13772.65 txn/s, latency: 2884.34 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3600 ms), latency samples: 5237280
two traffics test : committed: 100.03 txn/s, latency: 2107.24 ms, (p50: 1900 ms, p70: 2100, p90: 2300 ms, p99: 15600 ms), latency samples: 1760
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.359, avg: 1.255", "ConsensusProposalToOrdered: max: 0.328, avg: 0.301", "ConsensusOrderedToCommit: max: 0.409, avg: 0.392", "ConsensusProposalToCommit: max: 0.706, avg: 0.693"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.09s no progress at version 2080474 (avg 0.21s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 15.43s no progress at version 2080472 (avg 14.23s) [limit 16].
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking