ci: Benchmark against google-quiche #2218
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2218 +/- ##
=======================================
Coverage 95.39% 95.39%
=======================================
Files 112 112
Lines 36449 36449
=======================================
+ Hits 34769 34770 +1
+ Misses 1680 1679 -1 ☔ View full report in Codecov by Sentry. |
Failed Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
All resultsSucceeded Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
Unsupported Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
|
larseggert
requested review from
KershawChang,
martinthomson and
mxinden
as code owners
Benchmark resultsPerformance differences relative to 978aa4e. coalesce_acked_from_zero 1+1 entries: No change in performance detected. time: [98.643 ns 98.925 ns 99.217 ns]
change: [-0.5343% -0.0809% +0.3642%] (p = 0.73 > 0.05)
coalesce_acked_from_zero 3+1 entries: No change in performance detected. time: [117.04 ns 117.39 ns 117.77 ns]
change: [-0.3343% +0.1598% +0.6820%] (p = 0.54 > 0.05)
coalesce_acked_from_zero 10+1 entries: No change in performance detected. time: [116.62 ns 117.15 ns 117.77 ns]
change: [-0.9708% -0.1619% +0.5509%] (p = 0.70 > 0.05)
coalesce_acked_from_zero 1000+1 entries: No change in performance detected. time: [97.227 ns 97.369 ns 97.535 ns]
change: [-1.1854% -0.2263% +0.7491%] (p = 0.68 > 0.05)
RxStreamOrderer::inbound_frame(): No change in performance detected. time: [112.30 ms 112.44 ms 112.66 ms]
change: [-0.2098% +0.0251% +0.2825%] (p = 0.86 > 0.05)
transfer/pacing-false/varying-seeds: No change in performance detected. time: [27.046 ms 28.128 ms 29.201 ms]
change: [-3.5214% +1.6503% +7.6242%] (p = 0.56 > 0.05)
transfer/pacing-true/varying-seeds: Change within noise threshold. time: [33.187 ms 34.610 ms 36.055 ms]
change: [-12.740% -7.8303% -1.9586%] (p = 0.01 < 0.05)
transfer/pacing-false/same-seed: No change in performance detected. time: [26.194 ms 27.140 ms 28.089 ms]
change: [-1.1497% +3.5901% +8.4123%] (p = 0.14 > 0.05)
transfer/pacing-true/same-seed: No change in performance detected. time: [42.033 ms 44.004 ms 45.983 ms]
change: [-4.9051% +1.8604% +9.0924%] (p = 0.60 > 0.05)
1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: time: [912.04 ms 918.95 ms 925.93 ms]
thrpt: [108.00 MiB/s 108.82 MiB/s 109.64 MiB/s]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: time: [317.17 ms 320.25 ms 323.37 ms]
thrpt: [30.924 Kelem/s 31.225 Kelem/s 31.528 Kelem/s]1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: time: [33.484 ms 33.688 ms 33.906 ms]
thrpt: [29.494 elem/s 29.684 elem/s 29.865 elem/s]
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe1-conn/1-100mb-resp/mtu-1504 (aka. Upload)/client: time: [1.6338 s 1.6506 s 1.6678 s]
thrpt: [59.959 MiB/s 60.585 MiB/s 61.206 MiB/s]
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mildClient/server transfer resultsTransfer of 33554432 bytes over loopback.
|
mxinden
approved these changes
Nov 13, 2024
.github/workflows/bench.yml
Outdated
Comment on lines
138
to
163
| for mtu in 1504 65536; do | ||
| for mtu in 1504; do # We used to measure 65536 here, too, but that's somewhat unrealistic. |
There was a problem hiding this comment.
How about removing the loop, given that it iterates at most once. Would reduce the level of indentation.
There was a problem hiding this comment.
Can do. Was wondering if we wanted to put it back at some point.
There was a problem hiding this comment.
Should I remove the 65536 MTU everywhere? Would cut down the time on benches.
There was a problem hiding this comment.
I assume we will focus on Internet paths for now, thus I suggest removing 64k MTU entirely.
I don't know. |
|
Bad push @larseggert? |
This reverts commit 648a380.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.