Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigating SemaphoreSlim on net6.0+ #64

Open
NickCraver opened this issue Jan 13, 2022 · 1 comment
Open

Investigating SemaphoreSlim on net6.0+ #64

NickCraver opened this issue Jan 13, 2022 · 1 comment

Comments

@NickCraver
Copy link
Collaborator

Issue for tracking this discussion (so I can also put benchmarks somewhere).

Getting benchmarks running again for a discussion around going back to SemaphoreSlim now that the async issue has been fixed. Current benchmark run shows positive results there:

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
AMD Ryzen 9 5900HX with Radeon Graphics, 1 CPU, 16 logical and 8 physical cores
.NET SDK=6.0.101
  [Host]               : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
  Job-LZCCVU           : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
  Job-MZIHQK           : .NET Framework 4.8 (4.8.4420.0), X64 RyuJIT
  .NET 6.0             : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
  .NET Framework 4.7.2 : .NET Framework 4.8 (4.8.4420.0), X64 RyuJIT
Method Job Runtime Toolchain Mean Error StdDev Min Max Ratio RatioSD Gen 0 Gen 1 Gen 2 Allocated
Monitor_Sync Job-LZCCVU .NET 6.0 net6.0 5.526 ns 0.0613 ns 0.0573 ns 5.434 ns 5.615 ns 0.93 0.02 - - - -
Monitor_Sync Job-MZIHQK .NET Framework 4.7.2 net472 5.928 ns 0.1174 ns 0.1098 ns 5.753 ns 6.115 ns 1.00 0.00 - - - -
Monitor_Sync .NET 6.0 .NET 6.0 Default 5.337 ns 0.0380 ns 0.0317 ns 5.250 ns 5.378 ns 0.90 0.02 - - - -
Monitor_Sync .NET Framework 4.7.2 .NET Framework 4.7.2 Default 6.172 ns 0.0383 ns 0.0358 ns 6.110 ns 6.215 ns 1.04 0.02 - - - -
SemaphoreSlim_Sync Job-LZCCVU .NET 6.0 net6.0 18.246 ns 0.2540 ns 0.2375 ns 17.968 ns 18.590 ns 0.28 0.00 - - - -
SemaphoreSlim_Sync Job-MZIHQK .NET Framework 4.7.2 net472 65.291 ns 0.5218 ns 0.4357 ns 64.263 ns 66.220 ns 1.00 0.00 - - - -
SemaphoreSlim_Sync .NET 6.0 .NET 6.0 Default 18.510 ns 0.3072 ns 0.2874 ns 18.105 ns 19.075 ns 0.28 0.00 - - - -
SemaphoreSlim_Sync .NET Framework 4.7.2 .NET Framework 4.7.2 Default 63.148 ns 0.3578 ns 0.3347 ns 62.689 ns 63.714 ns 0.97 0.01 - - - -
SemaphoreSlim_Async Job-LZCCVU .NET 6.0 net6.0 20.920 ns 0.2461 ns 0.2302 ns 20.527 ns 21.220 ns 0.36 0.00 - - - -
SemaphoreSlim_Async Job-MZIHQK .NET Framework 4.7.2 net472 57.513 ns 0.5600 ns 0.5238 ns 56.611 ns 58.147 ns 1.00 0.00 - - - -
SemaphoreSlim_Async .NET 6.0 .NET 6.0 Default 21.221 ns 0.1525 ns 0.1426 ns 20.910 ns 21.382 ns 0.37 0.00 - - - -
SemaphoreSlim_Async .NET Framework 4.7.2 .NET Framework 4.7.2 Default 55.002 ns 0.0908 ns 0.0805 ns 54.828 ns 55.092 ns 0.96 0.01 - - - -
SemaphoreSlim_Async_HotPath Job-LZCCVU .NET 6.0 net6.0 15.182 ns 0.1708 ns 0.1598 ns 14.925 ns 15.495 ns 0.30 0.01 - - - -
SemaphoreSlim_Async_HotPath Job-MZIHQK .NET Framework 4.7.2 net472 50.912 ns 0.8782 ns 0.8215 ns 49.626 ns 52.584 ns 1.00 0.00 - - - -
SemaphoreSlim_Async_HotPath .NET 6.0 .NET 6.0 Default 14.724 ns 0.1402 ns 0.1311 ns 14.473 ns 14.935 ns 0.29 0.01 - - - -
SemaphoreSlim_Async_HotPath .NET Framework 4.7.2 .NET Framework 4.7.2 Default 47.183 ns 0.1631 ns 0.1525 ns 46.863 ns 47.457 ns 0.93 0.01 - - - -
MutexSlim_Sync Job-LZCCVU .NET 6.0 net6.0 22.292 ns 0.1948 ns 0.1627 ns 22.014 ns 22.564 ns 0.52 0.01 - - - -
MutexSlim_Sync Job-MZIHQK .NET Framework 4.7.2 net472 43.145 ns 0.3944 ns 0.3689 ns 42.555 ns 43.827 ns 1.00 0.00 - - - -
MutexSlim_Sync .NET 6.0 .NET 6.0 Default 22.173 ns 0.1043 ns 0.0975 ns 22.052 ns 22.354 ns 0.51 0.01 - - - -
MutexSlim_Sync .NET Framework 4.7.2 .NET Framework 4.7.2 Default 41.903 ns 0.2398 ns 0.2243 ns 41.381 ns 42.193 ns 0.97 0.01 - - - -
MutexSlim_Async Job-LZCCVU .NET 6.0 net6.0 42.810 ns 0.4583 ns 0.4287 ns 41.971 ns 43.376 ns 0.56 0.01 - - - -
MutexSlim_Async Job-MZIHQK .NET Framework 4.7.2 net472 76.444 ns 0.3811 ns 0.3379 ns 75.667 ns 76.857 ns 1.00 0.00 - - - -
MutexSlim_Async .NET 6.0 .NET 6.0 Default 42.568 ns 0.2998 ns 0.2804 ns 42.157 ns 42.997 ns 0.56 0.00 - - - -
MutexSlim_Async .NET Framework 4.7.2 .NET Framework 4.7.2 Default 72.216 ns 0.2122 ns 0.1985 ns 71.871 ns 72.586 ns 0.94 0.01 - - - -
MutexSlim_Async_HotPath Job-LZCCVU .NET 6.0 net6.0 29.913 ns 0.5884 ns 0.6776 ns 29.128 ns 30.955 ns 0.54 0.01 - - - -
MutexSlim_Async_HotPath Job-MZIHQK .NET Framework 4.7.2 net472 55.409 ns 0.7513 ns 0.6660 ns 54.379 ns 56.648 ns 1.00 0.00 - - - -
MutexSlim_Async_HotPath .NET 6.0 .NET 6.0 Default 29.443 ns 0.1544 ns 0.1444 ns 29.206 ns 29.638 ns 0.53 0.01 - - - -
MutexSlim_Async_HotPath .NET Framework 4.7.2 .NET Framework 4.7.2 Default 53.680 ns 0.3121 ns 0.2919 ns 53.190 ns 54.231 ns 0.97 0.01 - - - -
MutexSlim_ConcurrentLoadAsync Job-LZCCVU .NET 6.0 net6.0 2,120.714 ns 28.5866 ns 26.7399 ns 2,061.812 ns 2,149.688 ns 0.74 0.02 0.0125 - - 125 B
MutexSlim_ConcurrentLoadAsync Job-MZIHQK .NET Framework 4.7.2 net472 2,883.994 ns 46.5535 ns 41.2685 ns 2,762.735 ns 2,927.068 ns 1.00 0.00 0.0438 0.0063 0.0031 284 B
MutexSlim_ConcurrentLoadAsync .NET 6.0 .NET 6.0 Default 2,109.083 ns 17.1215 ns 16.0154 ns 2,072.544 ns 2,126.885 ns 0.73 0.01 0.0125 - - 125 B
MutexSlim_ConcurrentLoadAsync .NET Framework 4.7.2 .NET Framework 4.7.2 Default 2,982.906 ns 11.4533 ns 9.5640 ns 2,967.364 ns 2,995.070 ns 1.04 0.02 0.0406 0.0063 0.0031 279 B
MutexSlim_ConcurrentLoadAsync_DisableContext Job-LZCCVU .NET 6.0 net6.0 2,077.706 ns 16.0061 ns 14.9721 ns 2,041.016 ns 2,094.909 ns 0.74 0.02 0.0063 - - 61 B
MutexSlim_ConcurrentLoadAsync_DisableContext Job-MZIHQK .NET Framework 4.7.2 net472 2,830.493 ns 54.1615 ns 57.9522 ns 2,658.874 ns 2,864.015 ns 1.00 0.00 0.0313 0.0031 - 223 B
MutexSlim_ConcurrentLoadAsync_DisableContext .NET 6.0 .NET 6.0 Default 2,007.664 ns 14.7015 ns 13.7518 ns 1,980.703 ns 2,029.217 ns 0.71 0.02 0.0063 - - 61 B
MutexSlim_ConcurrentLoadAsync_DisableContext .NET Framework 4.7.2 .NET Framework 4.7.2 Default 2,863.572 ns 8.1600 ns 6.8140 ns 2,853.829 ns 2,875.984 ns 1.02 0.03 0.0344 0.0031 - 223 B
SemaphoreSlim_ConcurrentLoadAsync Job-LZCCVU .NET 6.0 net6.0 2,084.316 ns 4.5909 ns 4.0698 ns 2,072.476 ns 2,090.046 ns 0.55 0.01 0.0656 0.0063 - 547 B
SemaphoreSlim_ConcurrentLoadAsync Job-MZIHQK .NET Framework 4.7.2 net472 3,812.444 ns 42.4014 ns 37.5877 ns 3,760.819 ns 3,890.354 ns 1.00 0.00 0.2250 0.0125 - 1,449 B
SemaphoreSlim_ConcurrentLoadAsync .NET 6.0 .NET 6.0 Default 2,124.172 ns 15.9106 ns 14.1043 ns 2,077.397 ns 2,135.378 ns 0.56 0.01 0.0656 0.0063 - 547 B
SemaphoreSlim_ConcurrentLoadAsync .NET Framework 4.7.2 .NET Framework 4.7.2 Default 3,596.999 ns 31.8994 ns 29.8387 ns 3,540.641 ns 3,652.883 ns 0.94 0.01 0.2250 0.0125 - 1,448 B
AsyncSemaphore_ConcurrentLoadAsync Job-LZCCVU .NET 6.0 net6.0 2,027.560 ns 40.4162 ns 51.1136 ns 1,823.880 ns 2,072.102 ns 0.70 0.03 0.0188 - - 163 B
AsyncSemaphore_ConcurrentLoadAsync Job-MZIHQK .NET Framework 4.7.2 net472 2,911.152 ns 57.9191 ns 59.4786 ns 2,767.561 ns 2,985.664 ns 1.00 0.00 0.0500 0.0125 0.0031 328 B
AsyncSemaphore_ConcurrentLoadAsync .NET 6.0 .NET 6.0 Default 2,192.759 ns 42.9130 ns 60.1580 ns 1,910.755 ns 2,232.827 ns 0.75 0.03 0.0188 - - 163 B
AsyncSemaphore_ConcurrentLoadAsync .NET Framework 4.7.2 .NET Framework 4.7.2 Default 3,017.960 ns 13.2881 ns 11.7795 ns 3,001.102 ns 3,039.382 ns 1.04 0.02 0.0500 0.0094 0.0031 328 B
AsyncSemaphore_Sync Job-LZCCVU .NET 6.0 net6.0 15.422 ns 0.2894 ns 0.2565 ns 15.093 ns 15.865 ns 0.75 0.01 - - - -
AsyncSemaphore_Sync Job-MZIHQK .NET Framework 4.7.2 net472 20.470 ns 0.2239 ns 0.2094 ns 20.101 ns 20.818 ns 1.00 0.00 - - - -
AsyncSemaphore_Sync .NET 6.0 .NET 6.0 Default 15.330 ns 0.2029 ns 0.1898 ns 15.131 ns 15.654 ns 0.75 0.01 - - - -
AsyncSemaphore_Sync .NET Framework 4.7.2 .NET Framework 4.7.2 Default 29.478 ns 0.5677 ns 0.6074 ns 27.719 ns 30.165 ns 1.44 0.04 - - - -
AsyncSemaphore_Async Job-LZCCVU .NET 6.0 net6.0 19.468 ns 0.1271 ns 0.1189 ns 19.285 ns 19.687 ns 0.73 0.00 - - - -
AsyncSemaphore_Async Job-MZIHQK .NET Framework 4.7.2 net472 26.625 ns 0.1710 ns 0.1600 ns 26.295 ns 26.857 ns 1.00 0.00 - - - -
AsyncSemaphore_Async .NET 6.0 .NET 6.0 Default 18.151 ns 0.1799 ns 0.1682 ns 17.834 ns 18.381 ns 0.68 0.01 - - - -
AsyncSemaphore_Async .NET Framework 4.7.2 .NET Framework 4.7.2 Default 24.769 ns 0.1090 ns 0.0910 ns 24.530 ns 24.856 ns 0.93 0.01 - - - -
AsyncSemaphore_Async_HotPath Job-LZCCVU .NET 6.0 net6.0 16.201 ns 0.3189 ns 0.5058 ns 15.063 ns 17.121 ns 0.79 0.02 - - - -
AsyncSemaphore_Async_HotPath Job-MZIHQK .NET Framework 4.7.2 net472 20.112 ns 0.1511 ns 0.1340 ns 19.905 ns 20.335 ns 1.00 0.00 - - - -
AsyncSemaphore_Async_HotPath .NET 6.0 .NET 6.0 Default 14.742 ns 0.1243 ns 0.1163 ns 14.502 ns 14.923 ns 0.73 0.01 - - - -
AsyncSemaphore_Async_HotPath .NET Framework 4.7.2 .NET Framework 4.7.2 Default 20.343 ns 0.1257 ns 0.1115 ns 20.154 ns 20.541 ns 1.01 0.01 - - - -

cc @mgravell ^ I can take a poke at what a swap looks like this week just wanted to post numbers up!
@NickCraver
Copy link
Collaborator Author

NickCraver commented Jan 15, 2022 •
edited
Loading

Slimming down the above table to compare what we are asking here: MutexSlim vs. SemaphoreSlim:

Method Runtime Mean Error StdDev Allocated
SemaphoreSlim_Sync .NET 6.0 18.246 ns 0.2540 ns 0.2375 ns -
MutexSlim_Sync .NET 6.0 22.292 ns 0.1948 ns 0.1627 ns -
SemaphoreSlim_Sync .NET 4.7.2 65.291 ns 0.5218 ns 0.4357 ns -
MutexSlim_Sync .NET 4.7.2 43.145 ns 0.3944 ns 0.3689 ns -
SemaphoreSlim_Async .NET 6.0 20.920 ns 0.2461 ns 0.2302 ns -
MutexSlim_Async .NET 6.0 42.810 ns 0.4583 ns 0.4287 ns -
SemaphoreSlim_Async .NET 4.7.2 57.513 ns 0.5600 ns 0.5238 ns -
MutexSlim_Async .NET 4.7.2 76.444 ns 0.3811 ns 0.3379 ns -
SemaphoreSlim_Async_HotPath .NET 6.0 15.182 ns 0.1708 ns 0.1598 ns -
MutexSlim_Async_HotPath .NET 6.0 29.913 ns 0.5884 ns 0.6776 ns -
SemaphoreSlim_Async_HotPath .NET 4.7.2 50.912 ns 0.8782 ns 0.8215 ns -
MutexSlim_Async_HotPath .NET 4.7.2 55.409 ns 0.7513 ns 0.6660 ns -
SemaphoreSlim_ConcurrentAsync .NET 6.0 2,084.316 ns 4.5909 ns 4.0698 ns 547 B
MutexSlim_ConcurrentAsync .NET 6.0 2,120.714 ns 28.5866 ns 26.7399 ns 125 B
SemaphoreSlim_ConcurrentAsync .NET 4.7.2 3,812.444 ns 42.4014 ns 37.5877 ns 1,449 B
MutexSlim_ConcurrentAsync .NET 4.7.2 2,883.994 ns 46.5535 ns 41.2685 ns 284 B

NickCraver added a commit that referenced this issue Jan 15, 2022
@NickCraver
Benchmarcs: update to net6.0 and stuff
1bbcb51 
Done for #64, runs can now be:
```ps
.\Benchmarks.cmd --filter *Lock*
```
@NickCraver NickCraver mentioned this issue Jan 15, 2022
Merged
NickCraver added a commit to StackExchange/StackExchange.Redis that referenced this issue Jan 15, 2022
@NickCraver
Writer: switch back to SemaphoreSlim
bf9fa07 
Since Semaphore slim has been fixed on all the platforms we're building for these days, this tests moving back. Getting some test run comparison data, but all synthetic bechmarks are looking good. See mgravell/Pipelines.Sockets.Unofficial#64 for details
@NickCraver NickCraver mentioned this issue Jan 15, 2022
Merged
NickCraver added a commit to StackExchange/StackExchange.Redis that referenced this issue Jan 19, 2022
@NickCraver
Writer: switch back to SemaphoreSlim (#1949)
11943b3 
Since Semaphore slim has been fixed on all the platforms we're building for these days, this tests moving back. Getting some test run comparison data, but all synthetic benchmarks are looking good. See mgravell/Pipelines.Sockets.Unofficial#64 for details

Here's the main contention test metrics from those benchmarks:

|                        Method |    Runtime |         Mean |      Error |     StdDev | Allocated |
|------------------------------ |----------- |-------------:|-----------:|-----------:|----------:|
|            SemaphoreSlim_Sync |   .NET 6.0 |    18.246 ns |  0.2540 ns |  0.2375 ns |         - |
|                MutexSlim_Sync |   .NET 6.0 |    22.292 ns |  0.1948 ns |  0.1627 ns |         - |
|            SemaphoreSlim_Sync | .NET 4.7.2 |    65.291 ns |  0.5218 ns |  0.4357 ns |         - |
|                MutexSlim_Sync | .NET 4.7.2 |    43.145 ns |  0.3944 ns |  0.3689 ns |         - |
|                               |            |              |            |            |           |
|           SemaphoreSlim_Async |   .NET 6.0 |    20.920 ns |  0.2461 ns |  0.2302 ns |         - |
|               MutexSlim_Async |   .NET 6.0 |    42.810 ns |  0.4583 ns |  0.4287 ns |         - |
|           SemaphoreSlim_Async | .NET 4.7.2 |    57.513 ns |  0.5600 ns |  0.5238 ns |         - |
|               MutexSlim_Async | .NET 4.7.2 |    76.444 ns |  0.3811 ns |  0.3379 ns |         - |
|                               |            |              |            |            |           |
|   SemaphoreSlim_Async_HotPath |   .NET 6.0 |    15.182 ns |  0.1708 ns |  0.1598 ns |         - |
|       MutexSlim_Async_HotPath |   .NET 6.0 |    29.913 ns |  0.5884 ns |  0.6776 ns |         - |
|   SemaphoreSlim_Async_HotPath | .NET 4.7.2 |    50.912 ns |  0.8782 ns |  0.8215 ns |         - |
|       MutexSlim_Async_HotPath | .NET 4.7.2 |    55.409 ns |  0.7513 ns |  0.6660 ns |         - |
|                               |            |              |            |            |           |
| SemaphoreSlim_ConcurrentAsync |   .NET 6.0 | 2,084.316 ns |  4.5909 ns |  4.0698 ns |     547 B |
|     MutexSlim_ConcurrentAsync |   .NET 6.0 | 2,120.714 ns | 28.5866 ns | 26.7399 ns |     125 B |
| SemaphoreSlim_ConcurrentAsync | .NET 4.7.2 | 3,812.444 ns | 42.4014 ns | 37.5877 ns |   1,449 B |
|     MutexSlim_ConcurrentAsync | .NET 4.7.2 | 2,883.994 ns | 46.5535 ns | 41.2685 ns |     284 B |

We don't have high contention tests, but sanity checking against our test suite (where we don't expect this to matter much):

Main branch:
![main test speed](https://user-images.githubusercontent.com/454813/149633288-0b1fb4ac-44f8-4151-92e1-610b678610d2.png)

PR branch:
![pr test speed](https://user-images.githubusercontent.com/454813/149633275-0cc20e3a-ba6b-49b6-bfed-563ed8e343d0.png)

We could scope this back to .NET 6.0+ only, but the code's a lot more `#ifdef` and complicated (because `LockTokens` aren't a thing - it's just a bool "did I get it?")...thoughts?
mgravell pushed a commit that referenced this issue Mar 3, 2023
@NickCraver
Benchmarcs: update to net6.0 and stuff (#65)
f044f67 
Done for #64, runs can now be:
```ps
.\Benchmarks.cmd --filter *Lock*
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@NickCraver