Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix strict_length in prioritized slice sampler #2194

Merged
merged 3 commits into from
Jun 4, 2024
Merged

[BugFix] Fix strict_length in prioritized slice sampler #2194

merged 3 commits into from
Jun 4, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jun 3, 2024

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jun 3, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2194

Note: Links to docs will display an error until the docs builds have been completed.

❌ 10 New Failures, 9 Unrelated Failures

As of commit 5659001 with merge base 3e6cb84 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 3, 2024
@vmoens vmoens linked an issue Jun 3, 2024 that may be closed by this pull request
[BUG] outdated PrioritizedSliceSampler implementation #2193
Closed
3 tasks
@vmoens vmoens added the bug Something isn't working label Jun 3, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 3, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1088s 59.9196ms 16.6890 Ops/s 17.6010 Ops/s $\textbf{\color{#d91a1a}-5.18\%}$
test_sync 41.2009ms 31.9943ms 31.2556 Ops/s 31.4979 Ops/s $\color{#d91a1a}-0.77\%$
test_async 54.6765ms 27.6045ms 36.2260 Ops/s 35.3370 Ops/s $\color{#35bf28}+2.52\%$
test_simple 0.4525s 0.3956s 2.5281 Ops/s 2.6419 Ops/s $\color{#d91a1a}-4.31\%$
test_transformed 0.5381s 0.5365s 1.8639 Ops/s 1.8065 Ops/s $\color{#35bf28}+3.18\%$
test_serial 1.3295s 1.2815s 0.7803 Ops/s 0.7852 Ops/s $\color{#d91a1a}-0.62\%$
test_parallel 1.1425s 1.0723s 0.9326 Ops/s 0.9314 Ops/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[True-True-True-True-True] 0.1888ms 22.1613μs 45.1236 KOps/s 46.1921 KOps/s $\color{#d91a1a}-2.31\%$
test_step_mdp_speed[True-True-True-True-False] 55.1830μs 13.0749μs 76.4824 KOps/s 75.9968 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[True-True-True-False-True] 44.4630μs 12.6257μs 79.2033 KOps/s 77.9766 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-True-True-False-False] 27.1300μs 7.6613μs 130.5258 KOps/s 130.1141 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-True-False-True-True] 56.6350μs 23.0702μs 43.3460 KOps/s 43.5025 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[True-True-False-True-False] 0.2071ms 14.5516μs 68.7211 KOps/s 69.1018 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[True-True-False-False-True] 42.0590μs 14.0145μs 71.3545 KOps/s 70.0615 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-True-False-False-False] 31.7690μs 8.9724μs 111.4528 KOps/s 110.4699 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-False-True-True-True] 51.9970μs 24.4159μs 40.9570 KOps/s 40.6226 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[True-False-True-True-False] 61.8660μs 15.7601μs 63.4512 KOps/s 62.9673 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[True-False-True-False-True] 56.5760μs 13.8212μs 72.3528 KOps/s 70.8508 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[True-False-True-False-False] 46.7970μs 8.9198μs 112.1100 KOps/s 110.6007 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[True-False-False-True-True] 54.0210μs 25.1965μs 39.6881 KOps/s 39.0289 KOps/s $\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-False-False-True-False] 66.3620μs 16.9909μs 58.8551 KOps/s 58.1133 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[True-False-False-False-True] 38.3120μs 15.1170μs 66.1507 KOps/s 64.9034 KOps/s $\color{#35bf28}+1.92\%$
test_step_mdp_speed[True-False-False-False-False] 30.2960μs 10.1072μs 98.9397 KOps/s 96.7453 KOps/s $\color{#35bf28}+2.27\%$
test_step_mdp_speed[False-True-True-True-True] 89.1650μs 24.1416μs 41.4222 KOps/s 40.7280 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-True-True-True-False] 38.7320μs 15.7190μs 63.6172 KOps/s 63.1559 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[False-True-True-False-True] 41.9180μs 16.2753μs 61.4427 KOps/s 60.8096 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-True-True-False-False] 30.7770μs 10.0710μs 99.2954 KOps/s 97.9863 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[False-True-False-True-True] 53.2800μs 25.3831μs 39.3963 KOps/s 38.7690 KOps/s $\color{#35bf28}+1.62\%$
test_step_mdp_speed[False-True-False-True-False] 83.1750μs 16.9367μs 59.0436 KOps/s 58.3294 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-True-False-False-True] 52.5090μs 17.5094μs 57.1123 KOps/s 56.8139 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-True-False-False-False] 44.9440μs 11.3579μs 88.0446 KOps/s 87.5312 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-False-True-True-True] 56.2350μs 26.6299μs 37.5518 KOps/s 37.3474 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[False-False-True-True-False] 45.5750μs 18.2938μs 54.6633 KOps/s 54.2269 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-False-True-False-True] 39.8540μs 17.4362μs 57.3518 KOps/s 57.1067 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-False-True-False-False] 36.5780μs 11.3125μs 88.3981 KOps/s 87.9064 KOps/s $\color{#35bf28}+0.56\%$
test_step_mdp_speed[False-False-False-True-True] 41.3980μs 28.5809μs 34.9884 KOps/s 34.6130 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[False-False-False-True-False] 56.9270μs 19.3325μs 51.7263 KOps/s 51.3578 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[False-False-False-False-True] 48.6610μs 18.3740μs 54.4248 KOps/s 53.6460 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-False-False-False-False] 40.2050μs 12.4531μs 80.3010 KOps/s 79.2848 KOps/s $\color{#35bf28}+1.28\%$
test_values[generalized_advantage_estimate-True-True] 10.9184ms 9.5582ms 104.6227 Ops/s 104.0153 Ops/s $\color{#35bf28}+0.58\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.2712ms 33.3193ms 30.0127 Ops/s 28.3472 Ops/s $\textbf{\color{#35bf28}+5.88\%}$
test_values[td0_return_estimate-False-False] 0.2130ms 0.1649ms 6.0660 KOps/s 6.0739 KOps/s $\color{#d91a1a}-0.13\%$
test_values[td1_return_estimate-False-False] 25.7394ms 24.2233ms 41.2826 Ops/s 40.8849 Ops/s $\color{#35bf28}+0.97\%$
test_values[vec_td1_return_estimate-False-False] 34.3838ms 33.6216ms 29.7428 Ops/s 28.3962 Ops/s $\color{#35bf28}+4.74\%$
test_values[td_lambda_return_estimate-True-False] 35.4160ms 34.8928ms 28.6592 Ops/s 28.1690 Ops/s $\color{#35bf28}+1.74\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.3653ms 33.6508ms 29.7170 Ops/s 28.3522 Ops/s $\color{#35bf28}+4.81\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5625ms 8.4055ms 118.9695 Ops/s 117.3254 Ops/s $\color{#35bf28}+1.40\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.1360ms 1.7977ms 556.2643 Ops/s 523.7379 Ops/s $\textbf{\color{#35bf28}+6.21\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4747ms 0.3615ms 2.7659 KOps/s 2.7744 KOps/s $\color{#d91a1a}-0.30\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.0336ms 43.1777ms 23.1601 Ops/s 21.5753 Ops/s $\textbf{\color{#35bf28}+7.35\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 5.0093ms 3.1592ms 316.5379 Ops/s 329.3042 Ops/s $\color{#d91a1a}-3.88\%$
test_dqn_speed 1.6010ms 1.3708ms 729.4943 Ops/s 716.9809 Ops/s $\color{#35bf28}+1.75\%$
test_ddpg_speed 3.7614ms 2.9090ms 343.7563 Ops/s 343.1693 Ops/s $\color{#35bf28}+0.17\%$
test_sac_speed 9.1264ms 8.6430ms 115.7008 Ops/s 116.2022 Ops/s $\color{#d91a1a}-0.43\%$
test_redq_speed 15.0114ms 13.4831ms 74.1668 Ops/s 75.2284 Ops/s $\color{#d91a1a}-1.41\%$
test_redq_deprec_speed 14.4841ms 13.6731ms 73.1365 Ops/s 65.8990 Ops/s $\textbf{\color{#35bf28}+10.98\%}$
test_td3_speed 16.3614ms 8.6124ms 116.1119 Ops/s 116.8474 Ops/s $\color{#d91a1a}-0.63\%$
test_cql_speed 40.5719ms 36.9441ms 27.0680 Ops/s 27.2739 Ops/s $\color{#d91a1a}-0.76\%$
test_a2c_speed 10.1033ms 7.6116ms 131.3779 Ops/s 132.6672 Ops/s $\color{#d91a1a}-0.97\%$
test_ppo_speed 8.5277ms 7.8126ms 127.9980 Ops/s 126.9382 Ops/s $\color{#35bf28}+0.83\%$
test_reinforce_speed 7.7727ms 6.9036ms 144.8521 Ops/s 148.0792 Ops/s $\color{#d91a1a}-2.18\%$
test_iql_speed 33.6617ms 33.1158ms 30.1971 Ops/s 30.2802 Ops/s $\color{#d91a1a}-0.27\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 11.3760ms 3.6601ms 273.2187 Ops/s 281.5313 Ops/s $\color{#d91a1a}-2.95\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8934ms 0.5040ms 1.9843 KOps/s 1.9396 KOps/s $\color{#35bf28}+2.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7656ms 0.4804ms 2.0817 KOps/s 2.0771 KOps/s $\color{#35bf28}+0.22\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9139ms 3.6494ms 274.0195 Ops/s 284.5835 Ops/s $\color{#d91a1a}-3.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9307ms 0.5023ms 1.9907 KOps/s 2.0100 KOps/s $\color{#d91a1a}-0.96\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 2.2454ms 0.4871ms 2.0531 KOps/s 2.0920 KOps/s $\color{#d91a1a}-1.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.0710ms 1.2862ms 777.5070 Ops/s 794.2253 Ops/s $\color{#d91a1a}-2.10\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6703ms 1.2065ms 828.8616 Ops/s 842.5823 Ops/s $\color{#d91a1a}-1.63\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5705ms 3.6877ms 271.1685 Ops/s 270.9803 Ops/s $\color{#35bf28}+0.07\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9794ms 0.6379ms 1.5677 KOps/s 1.6085 KOps/s $\color{#d91a1a}-2.53\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7511ms 0.5960ms 1.6779 KOps/s 1.6831 KOps/s $\color{#d91a1a}-0.31\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.1162ms 3.5822ms 279.1565 Ops/s 281.3693 Ops/s $\color{#d91a1a}-0.79\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8105ms 0.5045ms 1.9823 KOps/s 1.9738 KOps/s $\color{#35bf28}+0.43\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.7688ms 0.4867ms 2.0545 KOps/s 2.0538 KOps/s $\color{#35bf28}+0.04\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.2831ms 3.5581ms 281.0524 Ops/s 279.3081 Ops/s $\color{#35bf28}+0.62\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7605ms 0.4991ms 2.0036 KOps/s 2.0091 KOps/s $\color{#d91a1a}-0.27\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6204ms 0.4729ms 2.1146 KOps/s 2.1025 KOps/s $\color{#35bf28}+0.57\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.4911ms 3.6509ms 273.9087 Ops/s 270.8404 Ops/s $\color{#35bf28}+1.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8679ms 0.6230ms 1.6050 KOps/s 1.6090 KOps/s $\color{#d91a1a}-0.25\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8727ms 0.5930ms 1.6862 KOps/s 1.6438 KOps/s $\color{#35bf28}+2.58\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1036s 5.6882ms 175.8039 Ops/s 125.5020 Ops/s $\textbf{\color{#35bf28}+40.08\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.9732ms 12.8846ms 77.6117 Ops/s 77.5694 Ops/s $\color{#35bf28}+0.05\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.2019ms 2.4868ms 402.1201 Ops/s 486.2555 Ops/s $\textbf{\color{#d91a1a}-17.30\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1015s 7.5287ms 132.8258 Ops/s 175.7936 Ops/s $\textbf{\color{#d91a1a}-24.44\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.8971ms 12.7481ms 78.4433 Ops/s 77.7531 Ops/s $\color{#35bf28}+0.89\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.3335ms 1.8152ms 550.8963 Ops/s 416.3103 Ops/s $\textbf{\color{#35bf28}+32.33\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1102s 6.3509ms 157.4589 Ops/s 124.2316 Ops/s $\textbf{\color{#35bf28}+26.75\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.9559ms 13.4011ms 74.6206 Ops/s 76.1325 Ops/s $\color{#d91a1a}-1.99\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.9096ms 2.7490ms 363.7724 Ops/s 378.7641 Ops/s $\color{#d91a1a}-3.96\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 3, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1177s 0.1175s 8.5082 Ops/s 8.3306 Ops/s $\color{#35bf28}+2.13\%$
test_sync 0.1051s 0.1039s 9.6217 Ops/s 9.4880 Ops/s $\color{#35bf28}+1.41\%$
test_async 0.1961s 95.3590ms 10.4867 Ops/s 10.9033 Ops/s $\color{#d91a1a}-3.82\%$
test_single_pixels 0.1304s 0.1283s 7.7961 Ops/s 7.7203 Ops/s $\color{#35bf28}+0.98\%$
test_sync_pixels 85.6050ms 83.0983ms 12.0339 Ops/s 11.9023 Ops/s $\color{#35bf28}+1.11\%$
test_async_pixels 0.1612s 69.5490ms 14.3783 Ops/s 14.2574 Ops/s $\color{#35bf28}+0.85\%$
test_simple 0.8844s 0.8249s 1.2122 Ops/s 1.1960 Ops/s $\color{#35bf28}+1.36\%$
test_transformed 1.1382s 1.0797s 0.9262 Ops/s 0.9116 Ops/s $\color{#35bf28}+1.61\%$
test_serial 2.5370s 2.4809s 0.4031 Ops/s 0.3960 Ops/s $\color{#35bf28}+1.79\%$
test_parallel 2.4134s 2.3485s 0.4258 Ops/s 0.4206 Ops/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-True-True-True-True] 0.1022ms 33.3833μs 29.9551 KOps/s 29.3629 KOps/s $\color{#35bf28}+2.02\%$
test_step_mdp_speed[True-True-True-True-False] 44.7100μs 19.7258μs 50.6949 KOps/s 49.2375 KOps/s $\color{#35bf28}+2.96\%$
test_step_mdp_speed[True-True-True-False-True] 40.3010μs 18.9958μs 52.6433 KOps/s 49.8905 KOps/s $\textbf{\color{#35bf28}+5.52\%}$
test_step_mdp_speed[True-True-True-False-False] 26.6400μs 11.3285μs 88.2730 KOps/s 84.6251 KOps/s $\color{#35bf28}+4.31\%$
test_step_mdp_speed[True-True-False-True-True] 63.1910μs 35.1030μs 28.4876 KOps/s 27.6114 KOps/s $\color{#35bf28}+3.17\%$
test_step_mdp_speed[True-True-False-True-False] 44.9400μs 21.8839μs 45.6957 KOps/s 45.3316 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-True-False-False-True] 39.3110μs 20.9046μs 47.8365 KOps/s 46.9420 KOps/s $\color{#35bf28}+1.91\%$
test_step_mdp_speed[True-True-False-False-False] 36.3810μs 13.4110μs 74.5658 KOps/s 74.1807 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-False-True-True-True] 58.8710μs 37.2974μs 26.8115 KOps/s 26.1407 KOps/s $\color{#35bf28}+2.57\%$
test_step_mdp_speed[True-False-True-True-False] 50.3210μs 23.7746μs 42.0617 KOps/s 41.0686 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[True-False-True-False-True] 46.6100μs 20.8853μs 47.8805 KOps/s 46.0152 KOps/s $\color{#35bf28}+4.05\%$
test_step_mdp_speed[True-False-True-False-False] 34.7300μs 13.3806μs 74.7350 KOps/s 74.5354 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-False-False-True-True] 61.5810μs 39.2912μs 25.4510 KOps/s 25.3874 KOps/s $\color{#35bf28}+0.25\%$
test_step_mdp_speed[True-False-False-True-False] 53.6410μs 25.9308μs 38.5642 KOps/s 38.9193 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[True-False-False-False-True] 48.7600μs 22.5325μs 44.3804 KOps/s 42.4007 KOps/s $\color{#35bf28}+4.67\%$
test_step_mdp_speed[True-False-False-False-False] 51.6910μs 15.2233μs 65.6887 KOps/s 64.4651 KOps/s $\color{#35bf28}+1.90\%$
test_step_mdp_speed[False-True-True-True-True] 64.2310μs 37.4017μs 26.7368 KOps/s 26.4352 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[False-True-True-True-False] 51.7500μs 23.9845μs 41.6937 KOps/s 41.2166 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-True-True-False-True] 51.9700μs 24.6033μs 40.6450 KOps/s 38.2890 KOps/s $\textbf{\color{#35bf28}+6.15\%}$
test_step_mdp_speed[False-True-True-False-False] 36.2400μs 15.0341μs 66.5153 KOps/s 64.4543 KOps/s $\color{#35bf28}+3.20\%$
test_step_mdp_speed[False-True-False-True-True] 65.4510μs 38.8674μs 25.7285 KOps/s 25.1531 KOps/s $\color{#35bf28}+2.29\%$
test_step_mdp_speed[False-True-False-True-False] 45.3800μs 25.6838μs 38.9350 KOps/s 38.0122 KOps/s $\color{#35bf28}+2.43\%$
test_step_mdp_speed[False-True-False-False-True] 48.7810μs 26.4615μs 37.7908 KOps/s 36.1538 KOps/s $\color{#35bf28}+4.53\%$
test_step_mdp_speed[False-True-False-False-False] 36.2900μs 16.7891μs 59.5625 KOps/s 58.0601 KOps/s $\color{#35bf28}+2.59\%$
test_step_mdp_speed[False-False-True-True-True] 64.7010μs 41.2743μs 24.2281 KOps/s 24.1535 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-True-True-False] 52.5410μs 27.4585μs 36.4186 KOps/s 35.6209 KOps/s $\color{#35bf28}+2.24\%$
test_step_mdp_speed[False-False-True-False-True] 47.3210μs 26.3604μs 37.9358 KOps/s 35.9710 KOps/s $\textbf{\color{#35bf28}+5.46\%}$
test_step_mdp_speed[False-False-True-False-False] 37.5510μs 16.7884μs 59.5649 KOps/s 58.1661 KOps/s $\color{#35bf28}+2.40\%$
test_step_mdp_speed[False-False-False-True-True] 57.8110μs 42.9904μs 23.2610 KOps/s 22.5930 KOps/s $\color{#35bf28}+2.96\%$
test_step_mdp_speed[False-False-False-True-False] 56.9010μs 29.5760μs 33.8112 KOps/s 33.5216 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-False-False-False-True] 60.7800μs 28.1459μs 35.5292 KOps/s 34.3773 KOps/s $\color{#35bf28}+3.35\%$
test_step_mdp_speed[False-False-False-False-False] 49.2410μs 18.6047μs 53.7498 KOps/s 53.0967 KOps/s $\color{#35bf28}+1.23\%$
test_values[generalized_advantage_estimate-True-True] 25.3206ms 24.7040ms 40.4793 Ops/s 38.9446 Ops/s $\color{#35bf28}+3.94\%$
test_values[vec_generalized_advantage_estimate-True-True] 81.2806ms 3.2038ms 312.1310 Ops/s 300.7793 Ops/s $\color{#35bf28}+3.77\%$
test_values[td0_return_estimate-False-False] 91.3310μs 64.8814μs 15.4127 KOps/s 15.0870 KOps/s $\color{#35bf28}+2.16\%$
test_values[td1_return_estimate-False-False] 53.9883ms 52.9850ms 18.8733 Ops/s 18.2585 Ops/s $\color{#35bf28}+3.37\%$
test_values[vec_td1_return_estimate-False-False] 2.0565ms 1.7766ms 562.8739 Ops/s 560.2954 Ops/s $\color{#35bf28}+0.46\%$
test_values[td_lambda_return_estimate-True-False] 85.4918ms 84.7664ms 11.7971 Ops/s 11.4134 Ops/s $\color{#35bf28}+3.36\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.1530ms 1.7756ms 563.1848 Ops/s 559.1379 Ops/s $\color{#35bf28}+0.72\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.7590ms 23.5196ms 42.5178 Ops/s 40.8184 Ops/s $\color{#35bf28}+4.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9536ms 0.7223ms 1.3845 KOps/s 1.3913 KOps/s $\color{#d91a1a}-0.48\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7279ms 0.6646ms 1.5048 KOps/s 1.4860 KOps/s $\color{#35bf28}+1.26\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4978ms 1.4618ms 684.0668 Ops/s 679.7693 Ops/s $\color{#35bf28}+0.63\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9479ms 0.6824ms 1.4654 KOps/s 1.4474 KOps/s $\color{#35bf28}+1.25\%$
test_dqn_speed 1.5728ms 1.4322ms 698.2281 Ops/s 691.8065 Ops/s $\color{#35bf28}+0.93\%$
test_ddpg_speed 3.3945ms 2.9366ms 340.5251 Ops/s 335.2328 Ops/s $\color{#35bf28}+1.58\%$
test_sac_speed 96.4613ms 9.2732ms 107.8380 Ops/s 116.8008 Ops/s $\textbf{\color{#d91a1a}-7.67\%}$
test_redq_speed 12.4731ms 10.6219ms 94.1447 Ops/s 84.5039 Ops/s $\textbf{\color{#35bf28}+11.41\%}$
test_redq_deprec_speed 11.9659ms 11.4847ms 87.0727 Ops/s 83.9303 Ops/s $\color{#35bf28}+3.74\%$
test_td3_speed 8.5100ms 8.3688ms 119.4913 Ops/s 117.7272 Ops/s $\color{#35bf28}+1.50\%$
test_cql_speed 27.0552ms 25.7723ms 38.8014 Ops/s 38.3657 Ops/s $\color{#35bf28}+1.14\%$
test_a2c_speed 6.2470ms 5.5643ms 179.7172 Ops/s 176.0353 Ops/s $\color{#35bf28}+2.09\%$
test_ppo_speed 6.9856ms 5.8525ms 170.8664 Ops/s 166.5777 Ops/s $\color{#35bf28}+2.57\%$
test_reinforce_speed 5.2094ms 4.5817ms 218.2587 Ops/s 212.8607 Ops/s $\color{#35bf28}+2.54\%$
test_iql_speed 20.3109ms 19.7327ms 50.6774 Ops/s 49.7848 Ops/s $\color{#35bf28}+1.79\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.7957ms 4.6790ms 213.7197 Ops/s 214.2861 Ops/s $\color{#d91a1a}-0.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.3800ms 0.5887ms 1.6986 KOps/s 1.4683 KOps/s $\textbf{\color{#35bf28}+15.68\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7451ms 0.5624ms 1.7780 KOps/s 1.7568 KOps/s $\color{#35bf28}+1.21\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.7693ms 4.5971ms 217.5296 Ops/s 214.4736 Ops/s $\color{#35bf28}+1.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3301ms 0.5797ms 1.7249 KOps/s 1.7000 KOps/s $\color{#35bf28}+1.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7019ms 0.5571ms 1.7950 KOps/s 1.7818 KOps/s $\color{#35bf28}+0.74\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8185ms 1.5201ms 657.8500 Ops/s 638.7253 Ops/s $\color{#35bf28}+2.99\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 6.5251ms 1.4583ms 685.7495 Ops/s 662.7569 Ops/s $\color{#35bf28}+3.47\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8971ms 4.7845ms 209.0076 Ops/s 209.1016 Ops/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7318ms 0.7092ms 1.4101 KOps/s 1.3911 KOps/s $\color{#35bf28}+1.36\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8878ms 0.6831ms 1.4639 KOps/s 1.4352 KOps/s $\color{#35bf28}+2.00\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.8026ms 4.6396ms 215.5343 Ops/s 212.7605 Ops/s $\color{#35bf28}+1.30\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5815ms 0.5908ms 1.6927 KOps/s 1.6741 KOps/s $\color{#35bf28}+1.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7023ms 0.5683ms 1.7595 KOps/s 1.7496 KOps/s $\color{#35bf28}+0.56\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.8209ms 4.6215ms 216.3815 Ops/s 214.8930 Ops/s $\color{#35bf28}+0.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6799ms 0.5798ms 1.7246 KOps/s 1.7032 KOps/s $\color{#35bf28}+1.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8372ms 0.5596ms 1.7871 KOps/s 1.7695 KOps/s $\color{#35bf28}+1.00\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8802ms 4.7773ms 209.3218 Ops/s 207.8802 Ops/s $\color{#35bf28}+0.69\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6727ms 0.7135ms 1.4016 KOps/s 1.3699 KOps/s $\color{#35bf28}+2.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8448ms 0.6879ms 1.4537 KOps/s 1.4062 KOps/s $\color{#35bf28}+3.37\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1386s 7.5409ms 132.6102 Ops/s 132.5380 Ops/s $\color{#35bf28}+0.05\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.2611ms 15.5462ms 64.3245 Ops/s 54.9399 Ops/s $\textbf{\color{#35bf28}+17.08\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.3880ms 1.2764ms 783.4352 Ops/s 769.3237 Ops/s $\color{#35bf28}+1.83\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1194s 9.4286ms 106.0606 Ops/s 136.8065 Ops/s $\textbf{\color{#d91a1a}-22.47\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.8203ms 15.5272ms 64.4032 Ops/s 63.5374 Ops/s $\color{#35bf28}+1.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.3609ms 1.3045ms 766.5895 Ops/s 764.0506 Ops/s $\color{#35bf28}+0.33\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1189s 7.5099ms 133.1578 Ops/s 130.9536 Ops/s $\color{#35bf28}+1.68\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.3277ms 15.7710ms 63.4077 Ops/s 61.5563 Ops/s $\color{#35bf28}+3.01\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.1833ms 1.7966ms 556.5996 Ops/s 601.1269 Ops/s $\textbf{\color{#d91a1a}-7.41\%}$

@vmoens vmoens mentioned this pull request Jun 3, 2024
Closed
3 tasks
@vmoens vmoens merged commit e5c3e32 into main Jun 4, 2024
36 of 47 checks passed
@vmoens vmoens deleted the fix-prioritized-slice-sampler branch June 4, 2024 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] outdated PrioritizedSliceSampler implementation
2 participants
@vmoens @facebook-github-bot