Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Doc] Improve PrioritizedSampler doc and get rid of np dependency as much as possible #1881

Merged
merged 8 commits into from
Feb 7, 2024
Merged

[Doc] Improve PrioritizedSampler doc and get rid of np dependency as much as possible #1881

merged 8 commits into from
Feb 7, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 6, 2024

cc @Cadene for #1875

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 6, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1881

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (4 Unrelated Failures)

As of commit f10c59b with merge base 1fe745a (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 6, 2024
@vmoens vmoens added documentation Improvements or additions to documentation Refactoring Refactoring of an existing feature Suitable for minor Suitable to be integrated in minor release (no new feature) labels Feb 6, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1313s 66.4727ms 15.0438 Ops/s 15.6606 Ops/s $\color{#d91a1a}-3.94\%$
test_sync 47.3163ms 36.2039ms 27.6214 Ops/s 30.2239 Ops/s $\textbf{\color{#d91a1a}-8.61\%}$
test_async 0.1384s 32.6734ms 30.6059 Ops/s 30.7287 Ops/s $\color{#d91a1a}-0.40\%$
test_simple 0.4948s 0.4333s 2.3079 Ops/s 2.2680 Ops/s $\color{#35bf28}+1.76\%$
test_transformed 0.6538s 0.6011s 1.6637 Ops/s 1.6431 Ops/s $\color{#35bf28}+1.26\%$
test_serial 1.4867s 1.4347s 0.6970 Ops/s 0.6824 Ops/s $\color{#35bf28}+2.14\%$
test_parallel 1.4529s 1.4053s 0.7116 Ops/s 0.7169 Ops/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-True-True-True-True] 0.1540ms 21.0847μs 47.4278 KOps/s 43.8161 KOps/s $\textbf{\color{#35bf28}+8.24\%}$
test_step_mdp_speed[True-True-True-True-False] 41.8490μs 12.8904μs 77.5773 KOps/s 76.1759 KOps/s $\color{#35bf28}+1.84\%$
test_step_mdp_speed[True-True-True-False-True] 49.3420μs 12.3363μs 81.0614 KOps/s 80.2605 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[True-True-True-False-False] 36.9990μs 7.5160μs 133.0502 KOps/s 133.3215 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[True-True-False-True-True] 52.6790μs 22.5412μs 44.3632 KOps/s 44.8838 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[True-True-False-True-False] 51.7570μs 14.1479μs 70.6816 KOps/s 70.4126 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-True-False-False-True] 33.6320μs 13.5726μs 73.6776 KOps/s 73.6256 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[True-True-False-False-False] 31.1090μs 8.8259μs 113.3031 KOps/s 113.8545 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-False-True-True-True] 48.7710μs 23.9460μs 41.7606 KOps/s 42.0334 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-False-True-True-False] 41.6480μs 15.6471μs 63.9096 KOps/s 63.6002 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[True-False-True-False-True] 43.1910μs 13.7422μs 72.7685 KOps/s 73.4316 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-False-True-False-False] 29.9260μs 8.8896μs 112.4908 KOps/s 114.0055 KOps/s $\color{#d91a1a}-1.33\%$
test_step_mdp_speed[True-False-False-True-True] 59.2610μs 24.9268μs 40.1175 KOps/s 40.0296 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[True-False-False-True-False] 37.4400μs 16.8265μs 59.4300 KOps/s 59.3879 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[True-False-False-False-True] 40.5660μs 14.7797μs 67.6605 KOps/s 67.9528 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-False-False-False-False] 28.9850μs 9.9847μs 100.1535 KOps/s 100.4539 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[False-True-True-True-True] 52.5780μs 23.7565μs 42.0937 KOps/s 41.5591 KOps/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[False-True-True-True-False] 36.2370μs 15.6163μs 64.0358 KOps/s 63.2229 KOps/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[False-True-True-False-True] 44.4830μs 15.9807μs 62.5753 KOps/s 62.9891 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-True-True-False-False] 38.0210μs 9.8976μs 101.0350 KOps/s 99.6852 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-True-False-True-True] 38.5420μs 25.0001μs 39.9998 KOps/s 39.1031 KOps/s $\color{#35bf28}+2.29\%$
test_step_mdp_speed[False-True-False-True-False] 40.2850μs 16.7623μs 59.6577 KOps/s 59.3257 KOps/s $\color{#35bf28}+0.56\%$
test_step_mdp_speed[False-True-False-False-True] 49.9130μs 16.7981μs 59.5306 KOps/s 58.3784 KOps/s $\color{#35bf28}+1.97\%$
test_step_mdp_speed[False-True-False-False-False] 41.4670μs 11.2682μs 88.7450 KOps/s 90.6089 KOps/s $\color{#d91a1a}-2.06\%$
test_step_mdp_speed[False-False-True-True-True] 50.8450μs 26.1321μs 38.2671 KOps/s 38.0363 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[False-False-True-True-False] 48.2500μs 17.8127μs 56.1396 KOps/s 54.9716 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[False-False-True-False-True] 40.2760μs 16.8887μs 59.2113 KOps/s 58.4993 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-False-True-False-False] 38.4020μs 11.1337μs 89.8177 KOps/s 89.1487 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-False-False-True-True] 54.5220μs 26.9529μs 37.1017 KOps/s 36.7295 KOps/s $\color{#35bf28}+1.01\%$
test_step_mdp_speed[False-False-False-True-False] 49.7530μs 18.9450μs 52.7844 KOps/s 51.9182 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[False-False-False-False-True] 42.8310μs 17.7888μs 56.2152 KOps/s 55.3032 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[False-False-False-False-False] 33.2220μs 12.2576μs 81.5818 KOps/s 82.4497 KOps/s $\color{#d91a1a}-1.05\%$
test_values[generalized_advantage_estimate-True-True] 10.0056ms 9.5824ms 104.3577 Ops/s 107.0159 Ops/s $\color{#d91a1a}-2.48\%$
test_values[vec_generalized_advantage_estimate-True-True] 38.7772ms 35.1194ms 28.4743 Ops/s 28.2975 Ops/s $\color{#35bf28}+0.62\%$
test_values[td0_return_estimate-False-False] 0.2346ms 0.1671ms 5.9839 KOps/s 6.1066 KOps/s $\color{#d91a1a}-2.01\%$
test_values[td1_return_estimate-False-False] 24.8667ms 23.8000ms 42.0168 Ops/s 42.9289 Ops/s $\color{#d91a1a}-2.12\%$
test_values[vec_td1_return_estimate-False-False] 36.4154ms 35.0844ms 28.5027 Ops/s 28.5470 Ops/s $\color{#d91a1a}-0.16\%$
test_values[td_lambda_return_estimate-True-False] 37.6566ms 34.5565ms 28.9381 Ops/s 29.2031 Ops/s $\color{#d91a1a}-0.91\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.3196ms 35.0908ms 28.4975 Ops/s 28.3890 Ops/s $\color{#35bf28}+0.38\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.6239ms 8.4104ms 118.9006 Ops/s 122.7363 Ops/s $\color{#d91a1a}-3.13\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4178ms 1.9838ms 504.0722 Ops/s 543.9239 Ops/s $\textbf{\color{#d91a1a}-7.33\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4623ms 0.3565ms 2.8047 KOps/s 2.8924 KOps/s $\color{#d91a1a}-3.03\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 63.5257ms 45.3062ms 22.0720 Ops/s 21.5848 Ops/s $\color{#35bf28}+2.26\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.5524ms 3.0299ms 330.0446 Ops/s 330.5331 Ops/s $\color{#d91a1a}-0.15\%$
test_dqn_speed 11.4075ms 5.9581ms 167.8393 Ops/s 169.9372 Ops/s $\color{#d91a1a}-1.23\%$
test_ddpg_speed 12.5036ms 11.5194ms 86.8098 Ops/s 86.1104 Ops/s $\color{#35bf28}+0.81\%$
test_sac_speed 99.9515ms 25.3588ms 39.4340 Ops/s 41.8948 Ops/s $\textbf{\color{#d91a1a}-5.87\%}$
test_redq_speed 15.4266ms 14.0541ms 71.1538 Ops/s 71.4842 Ops/s $\color{#d91a1a}-0.46\%$
test_redq_deprec_speed 23.1352ms 21.7205ms 46.0395 Ops/s 46.4583 Ops/s $\color{#d91a1a}-0.90\%$
test_td3_speed 25.0535ms 16.5043ms 60.5903 Ops/s 61.4028 Ops/s $\color{#d91a1a}-1.32\%$
test_cql_speed 74.8282ms 73.8979ms 13.5322 Ops/s 13.6243 Ops/s $\color{#d91a1a}-0.68\%$
test_a2c_speed 22.9896ms 21.9337ms 45.5919 Ops/s 45.6764 Ops/s $\color{#d91a1a}-0.18\%$
test_ppo_speed 24.2180ms 22.1397ms 45.1677 Ops/s 45.6637 Ops/s $\color{#d91a1a}-1.09\%$
test_reinforce_speed 22.7139ms 21.1360ms 47.3126 Ops/s 47.4374 Ops/s $\color{#d91a1a}-0.26\%$
test_iql_speed 57.9613ms 55.4999ms 18.0180 Ops/s 17.9813 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3972ms 2.8161ms 355.1024 Ops/s 375.7914 Ops/s $\textbf{\color{#d91a1a}-5.51\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7991ms 0.5084ms 1.9669 KOps/s 2.0024 KOps/s $\color{#d91a1a}-1.78\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8025ms 0.4784ms 2.0901 KOps/s 2.1152 KOps/s $\color{#d91a1a}-1.19\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0515ms 2.7112ms 368.8470 Ops/s 375.3196 Ops/s $\color{#d91a1a}-1.72\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8620ms 0.5019ms 1.9923 KOps/s 2.0399 KOps/s $\color{#d91a1a}-2.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7742ms 0.4695ms 2.1298 KOps/s 2.1236 KOps/s $\color{#35bf28}+0.29\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.9400ms 2.7445ms 364.3642 Ops/s 414.8365 Ops/s $\textbf{\color{#d91a1a}-12.17\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7319ms 0.6218ms 1.6083 KOps/s 1.6334 KOps/s $\color{#d91a1a}-1.54\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9563ms 0.6001ms 1.6664 KOps/s 1.7107 KOps/s $\color{#d91a1a}-2.59\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3272ms 2.6864ms 372.2493 Ops/s 376.9577 Ops/s $\color{#d91a1a}-1.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8582ms 0.5098ms 1.9616 KOps/s 2.0118 KOps/s $\color{#d91a1a}-2.49\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6505ms 0.4775ms 2.0942 KOps/s 2.1000 KOps/s $\color{#d91a1a}-0.28\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1862ms 2.7451ms 364.2819 Ops/s 370.3740 Ops/s $\color{#d91a1a}-1.64\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8887ms 0.5024ms 1.9906 KOps/s 2.0143 KOps/s $\color{#d91a1a}-1.18\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6374ms 0.4737ms 2.1112 KOps/s 2.0972 KOps/s $\color{#35bf28}+0.67\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0188ms 2.7904ms 358.3657 Ops/s 421.1015 Ops/s $\textbf{\color{#d91a1a}-14.90\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9664ms 0.6266ms 1.5958 KOps/s 1.6298 KOps/s $\color{#d91a1a}-2.09\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 90.0416ms 0.6995ms 1.4296 KOps/s 1.6908 KOps/s $\textbf{\color{#d91a1a}-15.45\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 99.9513ms 7.9482ms 125.8152 Ops/s 127.8814 Ops/s $\color{#d91a1a}-1.62\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.5511ms 13.3242ms 75.0511 Ops/s 74.8523 Ops/s $\color{#35bf28}+0.27\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.3982ms 2.5819ms 387.3086 Ops/s 397.7246 Ops/s $\color{#d91a1a}-2.62\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 99.3384ms 9.7478ms 102.5868 Ops/s 102.3844 Ops/s $\color{#35bf28}+0.20\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.4159ms 13.2923ms 75.2316 Ops/s 75.6463 Ops/s $\color{#d91a1a}-0.55\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.7940ms 2.5348ms 394.5082 Ops/s 395.5937 Ops/s $\color{#d91a1a}-0.27\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1005s 8.2387ms 121.3778 Ops/s 123.5857 Ops/s $\color{#d91a1a}-1.79\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.9098ms 13.5821ms 73.6261 Ops/s 73.9519 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.2687ms 2.7950ms 357.7782 Ops/s 362.6292 Ops/s $\color{#d91a1a}-1.34\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1194s 0.1183s 8.4558 Ops/s 8.4967 Ops/s $\color{#d91a1a}-0.48\%$
test_sync 96.5541ms 96.2731ms 10.3871 Ops/s 10.4280 Ops/s $\color{#d91a1a}-0.39\%$
test_async 0.1820s 92.0325ms 10.8657 Ops/s 10.8702 Ops/s $\color{#d91a1a}-0.04\%$
test_single_pixels 0.1411s 0.1396s 7.1622 Ops/s 7.3185 Ops/s $\color{#d91a1a}-2.14\%$
test_sync_pixels 83.1383ms 81.4671ms 12.2749 Ops/s 13.1090 Ops/s $\textbf{\color{#d91a1a}-6.36\%}$
test_async_pixels 0.1738s 75.7123ms 13.2079 Ops/s 13.2085 Ops/s $-0.00\%$
test_simple 0.9355s 0.8471s 1.1805 Ops/s 1.1978 Ops/s $\color{#d91a1a}-1.45\%$
test_transformed 1.1588s 1.1010s 0.9083 Ops/s 0.9222 Ops/s $\color{#d91a1a}-1.50\%$
test_serial 2.5154s 2.4657s 0.4056 Ops/s 0.4195 Ops/s $\color{#d91a1a}-3.32\%$
test_parallel 2.2876s 2.1456s 0.4661 Ops/s 0.4847 Ops/s $\color{#d91a1a}-3.84\%$
test_step_mdp_speed[True-True-True-True-True] 85.8510μs 33.4873μs 29.8621 KOps/s 30.3630 KOps/s $\color{#d91a1a}-1.65\%$
test_step_mdp_speed[True-True-True-True-False] 42.0610μs 20.1653μs 49.5903 KOps/s 50.9049 KOps/s $\color{#d91a1a}-2.58\%$
test_step_mdp_speed[True-True-True-False-True] 44.3610μs 19.1402μs 52.2460 KOps/s 54.5972 KOps/s $\color{#d91a1a}-4.31\%$
test_step_mdp_speed[True-True-True-False-False] 24.3400μs 11.2517μs 88.8755 KOps/s 90.5298 KOps/s $\color{#d91a1a}-1.83\%$
test_step_mdp_speed[True-True-False-True-True] 0.1071ms 35.0576μs 28.5245 KOps/s 28.9586 KOps/s $\color{#d91a1a}-1.50\%$
test_step_mdp_speed[True-True-False-True-False] 37.5410μs 21.3374μs 46.8660 KOps/s 47.7299 KOps/s $\color{#d91a1a}-1.81\%$
test_step_mdp_speed[True-True-False-False-True] 38.4100μs 20.5969μs 48.5510 KOps/s 49.7934 KOps/s $\color{#d91a1a}-2.50\%$
test_step_mdp_speed[True-True-False-False-False] 35.9510μs 13.0653μs 76.5385 KOps/s 77.6292 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[True-False-True-True-True] 54.5610μs 36.4235μs 27.4548 KOps/s 27.4445 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-False-True-True-False] 95.7810μs 23.5628μs 42.4399 KOps/s 43.6363 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[True-False-True-False-True] 36.2200μs 20.5830μs 48.5838 KOps/s 49.6300 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[True-False-True-False-False] 28.6700μs 13.0031μs 76.9046 KOps/s 76.4484 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-False-False-True-True] 56.0500μs 38.5721μs 25.9255 KOps/s 27.0354 KOps/s $\color{#d91a1a}-4.11\%$
test_step_mdp_speed[True-False-False-True-False] 46.4800μs 25.1563μs 39.7515 KOps/s 40.4465 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[True-False-False-False-True] 37.6910μs 22.7771μs 43.9037 KOps/s 45.3189 KOps/s $\color{#d91a1a}-3.12\%$
test_step_mdp_speed[True-False-False-False-False] 39.5110μs 15.0168μs 66.5921 KOps/s 69.2139 KOps/s $\color{#d91a1a}-3.79\%$
test_step_mdp_speed[False-True-True-True-True] 0.1151ms 37.0422μs 26.9963 KOps/s 27.9042 KOps/s $\color{#d91a1a}-3.25\%$
test_step_mdp_speed[False-True-True-True-False] 41.1910μs 23.5787μs 42.4112 KOps/s 43.5629 KOps/s $\color{#d91a1a}-2.64\%$
test_step_mdp_speed[False-True-True-False-True] 50.4600μs 25.0516μs 39.9176 KOps/s 41.1005 KOps/s $\color{#d91a1a}-2.88\%$
test_step_mdp_speed[False-True-True-False-False] 30.8900μs 14.8719μs 67.2408 KOps/s 68.0834 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-True-False-True-True] 97.0410μs 39.3604μs 25.4062 KOps/s 26.1326 KOps/s $\color{#d91a1a}-2.78\%$
test_step_mdp_speed[False-True-False-True-False] 49.6810μs 25.8963μs 38.6155 KOps/s 39.6134 KOps/s $\color{#d91a1a}-2.52\%$
test_step_mdp_speed[False-True-False-False-True] 45.4700μs 27.2306μs 36.7234 KOps/s 39.0799 KOps/s $\textbf{\color{#d91a1a}-6.03\%}$
test_step_mdp_speed[False-True-False-False-False] 40.6410μs 17.0272μs 58.7297 KOps/s 60.2434 KOps/s $\color{#d91a1a}-2.51\%$
test_step_mdp_speed[False-False-True-True-True] 66.6610μs 40.9729μs 24.4064 KOps/s 24.8186 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-False-True-True-False] 53.5100μs 27.7035μs 36.0966 KOps/s 37.1650 KOps/s $\color{#d91a1a}-2.87\%$
test_step_mdp_speed[False-False-True-False-True] 46.1600μs 26.4826μs 37.7607 KOps/s 39.0529 KOps/s $\color{#d91a1a}-3.31\%$
test_step_mdp_speed[False-False-True-False-False] 39.8710μs 16.7637μs 59.6526 KOps/s 60.0049 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-False-False-True-True] 58.5110μs 42.3435μs 23.6164 KOps/s 24.4073 KOps/s $\color{#d91a1a}-3.24\%$
test_step_mdp_speed[False-False-False-True-False] 55.5100μs 29.3560μs 34.0646 KOps/s 35.1458 KOps/s $\color{#d91a1a}-3.08\%$
test_step_mdp_speed[False-False-False-False-True] 76.5110μs 28.4047μs 35.2054 KOps/s 36.8039 KOps/s $\color{#d91a1a}-4.34\%$
test_step_mdp_speed[False-False-False-False-False] 34.9400μs 18.5484μs 53.9129 KOps/s 55.5251 KOps/s $\color{#d91a1a}-2.90\%$
test_values[generalized_advantage_estimate-True-True] 25.5802ms 24.7043ms 40.4788 Ops/s 40.4549 Ops/s $\color{#35bf28}+0.06\%$
test_values[vec_generalized_advantage_estimate-True-True] 85.5586ms 3.2805ms 304.8297 Ops/s 296.4844 Ops/s $\color{#35bf28}+2.81\%$
test_values[td0_return_estimate-False-False] 95.4910μs 62.2050μs 16.0759 KOps/s 16.1822 KOps/s $\color{#d91a1a}-0.66\%$
test_values[td1_return_estimate-False-False] 54.1102ms 53.2553ms 18.7775 Ops/s 17.6562 Ops/s $\textbf{\color{#35bf28}+6.35\%}$
test_values[vec_td1_return_estimate-False-False] 2.0866ms 1.7663ms 566.1547 Ops/s 563.2417 Ops/s $\color{#35bf28}+0.52\%$
test_values[td_lambda_return_estimate-True-False] 85.4599ms 85.0004ms 11.7646 Ops/s 11.0785 Ops/s $\textbf{\color{#35bf28}+6.19\%}$
test_values[vec_td_lambda_return_estimate-True-False] 4.1221ms 1.8029ms 554.6605 Ops/s 555.5647 Ops/s $\color{#d91a1a}-0.16\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.5898ms 23.4832ms 42.5836 Ops/s 40.3349 Ops/s $\textbf{\color{#35bf28}+5.57\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8879ms 0.7049ms 1.4186 KOps/s 1.4220 KOps/s $\color{#d91a1a}-0.24\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7445ms 0.6583ms 1.5191 KOps/s 1.5254 KOps/s $\color{#d91a1a}-0.41\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4957ms 1.4553ms 687.1309 Ops/s 686.1882 Ops/s $\color{#35bf28}+0.14\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9583ms 0.6760ms 1.4794 KOps/s 1.4858 KOps/s $\color{#d91a1a}-0.43\%$
test_dqn_speed 14.3466ms 7.5003ms 133.3288 Ops/s 136.7735 Ops/s $\color{#d91a1a}-2.52\%$
test_ddpg_speed 15.2901ms 14.3856ms 69.5140 Ops/s 71.7884 Ops/s $\color{#d91a1a}-3.17\%$
test_sac_speed 29.6768ms 28.8979ms 34.6046 Ops/s 36.0307 Ops/s $\color{#d91a1a}-3.96\%$
test_redq_speed 0.1017s 12.8793ms 77.6441 Ops/s 87.2702 Ops/s $\textbf{\color{#d91a1a}-11.03\%}$
test_redq_deprec_speed 23.7066ms 22.8610ms 43.7426 Ops/s 45.1507 Ops/s $\color{#d91a1a}-3.12\%$
test_td3_speed 19.4933ms 19.2758ms 51.8785 Ops/s 53.4393 Ops/s $\color{#d91a1a}-2.92\%$
test_cql_speed 77.2093ms 76.3158ms 13.1034 Ops/s 13.5946 Ops/s $\color{#d91a1a}-3.61\%$
test_a2c_speed 26.6618ms 25.3903ms 39.3851 Ops/s 40.8708 Ops/s $\color{#d91a1a}-3.64\%$
test_ppo_speed 26.2810ms 25.2503ms 39.6034 Ops/s 40.2285 Ops/s $\color{#d91a1a}-1.55\%$
test_reinforce_speed 25.8875ms 24.3678ms 41.0377 Ops/s 42.2655 Ops/s $\color{#d91a1a}-2.90\%$
test_iql_speed 51.5886ms 50.4135ms 19.8360 Ops/s 20.3128 Ops/s $\color{#d91a1a}-2.35\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7347ms 3.6099ms 277.0155 Ops/s 273.9740 Ops/s $\color{#35bf28}+1.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7118ms 0.5564ms 1.7971 KOps/s 1.8230 KOps/s $\color{#d91a1a}-1.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6671ms 0.5269ms 1.8980 KOps/s 1.9103 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8817ms 3.6618ms 273.0924 Ops/s 272.9259 Ops/s $\color{#35bf28}+0.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7131ms 0.5489ms 1.8218 KOps/s 1.8226 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6783ms 0.5230ms 1.9119 KOps/s 1.9198 KOps/s $\color{#d91a1a}-0.41\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.9294ms 3.7659ms 265.5417 Ops/s 301.6037 Ops/s $\textbf{\color{#d91a1a}-11.96\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7991ms 0.6790ms 1.4728 KOps/s 1.4903 KOps/s $\color{#d91a1a}-1.17\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7924ms 0.6510ms 1.5360 KOps/s 1.5460 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.6885ms 3.6062ms 277.2991 Ops/s 273.3294 Ops/s $\color{#35bf28}+1.45\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6929ms 0.5547ms 1.8027 KOps/s 1.8068 KOps/s $\color{#d91a1a}-0.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6708ms 0.5277ms 1.8952 KOps/s 1.9037 KOps/s $\color{#d91a1a}-0.45\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8445ms 3.6561ms 273.5183 Ops/s 273.2053 Ops/s $\color{#35bf28}+0.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6718ms 0.5519ms 1.8121 KOps/s 1.8190 KOps/s $\color{#d91a1a}-0.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6567ms 0.5245ms 1.9064 KOps/s 1.9219 KOps/s $\color{#d91a1a}-0.81\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.8712ms 3.7486ms 266.7665 Ops/s 301.4348 Ops/s $\textbf{\color{#d91a1a}-11.50\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8414ms 0.6796ms 1.4714 KOps/s 1.4668 KOps/s $\color{#35bf28}+0.32\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7921ms 0.6548ms 1.5272 KOps/s 1.2929 KOps/s $\textbf{\color{#35bf28}+18.12\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1342s 10.5098ms 95.1494 Ops/s 97.5840 Ops/s $\color{#d91a1a}-2.49\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.4966ms 16.5417ms 60.4534 Ops/s 61.7413 Ops/s $\color{#d91a1a}-2.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.4911ms 3.1431ms 318.1582 Ops/s 334.5697 Ops/s $\color{#d91a1a}-4.91\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1207s 10.1146ms 98.8672 Ops/s 98.7809 Ops/s $\color{#35bf28}+0.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.0196ms 16.4659ms 60.7315 Ops/s 61.8733 Ops/s $\color{#d91a1a}-1.85\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.6243ms 3.0579ms 327.0208 Ops/s 329.8183 Ops/s $\color{#d91a1a}-0.85\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1215s 12.7072ms 78.6954 Ops/s 95.5657 Ops/s $\textbf{\color{#d91a1a}-17.65\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.5969ms 16.8964ms 59.1842 Ops/s 60.9244 Ops/s $\color{#d91a1a}-2.86\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.3241ms 3.3545ms 298.1112 Ops/s 307.6789 Ops/s $\color{#d91a1a}-3.11\%$

@vmoens vmoens merged commit 144f547 into main Feb 7, 2024
63 of 67 checks passed
@vmoens vmoens deleted the sampler-doc branch February 7, 2024 09:32
vmoens added a commit that referenced this pull request Feb 28, 2024
@vmoens
[Doc] Improve PrioritizedSampler doc and get rid of np dependency as …
a7b359a 
…much as possible (#1881)
@vmoens vmoens mentioned this pull request Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. documentation Improvements or additions to documentation Refactoring Refactoring of an existing feature Suitable for minor Suitable to be integrated in minor release (no new feature)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot