Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix update in serial / parallel env #1866

Merged
merged 35 commits into from
Feb 5, 2024
Merged

[BugFix] Fix update in serial / parallel env #1866

merged 35 commits into from
Feb 5, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 1, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 1, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1866

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (8 Unrelated Failures)

As of commit b97781b with merge base 80fc87f (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 1, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 1, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1308s 67.0091ms 14.9233 Ops/s 16.6600 Ops/s $\textbf{\color{#d91a1a}-10.42\%}$
test_sync 33.1541ms 32.8193ms 30.4698 Ops/s 30.1025 Ops/s $\color{#35bf28}+1.22\%$
test_async 0.1364s 32.6257ms 30.6507 Ops/s 31.0536 Ops/s $\color{#d91a1a}-1.30\%$
test_simple 0.4921s 0.4336s 2.3061 Ops/s 2.3885 Ops/s $\color{#d91a1a}-3.45\%$
test_transformed 0.6473s 0.5932s 1.6858 Ops/s 1.7410 Ops/s $\color{#d91a1a}-3.17\%$
test_serial 1.4718s 1.4169s 0.7057 Ops/s 0.7675 Ops/s $\textbf{\color{#d91a1a}-8.05\%}$
test_parallel 1.4280s 1.3665s 0.7318 Ops/s 0.8209 Ops/s $\textbf{\color{#d91a1a}-10.85\%}$
test_step_mdp_speed[True-True-True-True-True] 0.1332ms 21.0878μs 47.4207 KOps/s 48.4830 KOps/s $\color{#d91a1a}-2.19\%$
test_step_mdp_speed[True-True-True-True-False] 34.7250μs 12.9231μs 77.3808 KOps/s 76.4331 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-True-True-False-True] 49.8330μs 12.3592μs 80.9116 KOps/s 82.5108 KOps/s $\color{#d91a1a}-1.94\%$
test_step_mdp_speed[True-True-True-False-False] 36.1670μs 7.4835μs 133.6272 KOps/s 134.7409 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-True-False-True-True] 46.4160μs 22.4305μs 44.5822 KOps/s 44.8286 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[True-True-False-True-False] 37.4800μs 14.1502μs 70.6703 KOps/s 71.5131 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-True-False-False-True] 52.6580μs 13.4552μs 74.3206 KOps/s 74.8371 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[True-True-False-False-False] 33.8230μs 8.6671μs 115.3794 KOps/s 115.2069 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[True-False-True-True-True] 52.4180μs 23.9313μs 41.7863 KOps/s 42.3889 KOps/s $\color{#d91a1a}-1.42\%$
test_step_mdp_speed[True-False-True-True-False] 44.3220μs 15.5645μs 64.2488 KOps/s 65.3697 KOps/s $\color{#d91a1a}-1.71\%$
test_step_mdp_speed[True-False-True-False-True] 40.5060μs 13.7090μs 72.9450 KOps/s 74.9139 KOps/s $\color{#d91a1a}-2.63\%$
test_step_mdp_speed[True-False-True-False-False] 43.5410μs 8.7219μs 114.6546 KOps/s 116.3669 KOps/s $\color{#d91a1a}-1.47\%$
test_step_mdp_speed[True-False-False-True-True] 69.5900μs 25.0440μs 39.9297 KOps/s 40.5457 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-False-False-True-False] 63.2320μs 16.5529μs 60.4124 KOps/s 60.1232 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[True-False-False-False-True] 37.4190μs 14.6236μs 68.3825 KOps/s 68.4529 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[True-False-False-False-False] 36.5580μs 9.8020μs 102.0201 KOps/s 100.9359 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-True-True-True-True] 59.3200μs 23.6660μs 42.2547 KOps/s 42.3705 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-True-True-True-False] 42.8600μs 15.5644μs 64.2491 KOps/s 65.7875 KOps/s $\color{#d91a1a}-2.34\%$
test_step_mdp_speed[False-True-True-False-True] 45.6450μs 15.7731μs 63.3991 KOps/s 64.6577 KOps/s $\color{#d91a1a}-1.95\%$
test_step_mdp_speed[False-True-True-False-False] 33.3620μs 9.9410μs 100.5931 KOps/s 101.9841 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-True-False-True-True] 54.9720μs 24.8023μs 40.3189 KOps/s 41.5766 KOps/s $\color{#d91a1a}-3.03\%$
test_step_mdp_speed[False-True-False-True-False] 43.2010μs 16.7027μs 59.8707 KOps/s 60.6333 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-True-False-False-True] 40.4960μs 16.9265μs 59.0789 KOps/s 60.2061 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[False-True-False-False-False] 43.7320μs 11.1108μs 90.0024 KOps/s 90.7359 KOps/s $\color{#d91a1a}-0.81\%$
test_step_mdp_speed[False-False-True-True-True] 53.3700μs 26.2957μs 38.0290 KOps/s 38.6782 KOps/s $\color{#d91a1a}-1.68\%$
test_step_mdp_speed[False-False-True-True-False] 54.7020μs 17.9996μs 55.5569 KOps/s 55.6799 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-False-True-False-True] 42.3390μs 17.0322μs 58.7123 KOps/s 59.7284 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[False-False-True-False-False] 35.4660μs 11.0707μs 90.3282 KOps/s 90.1433 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-False-False-True-True] 61.3040μs 27.3256μs 36.5958 KOps/s 37.2626 KOps/s $\color{#d91a1a}-1.79\%$
test_step_mdp_speed[False-False-False-True-False] 60.2020μs 19.0444μs 52.5088 KOps/s 53.3452 KOps/s $\color{#d91a1a}-1.57\%$
test_step_mdp_speed[False-False-False-False-True] 54.5510μs 18.0184μs 55.4990 KOps/s 56.2227 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[False-False-False-False-False] 38.6120μs 12.2361μs 81.7256 KOps/s 82.9548 KOps/s $\color{#d91a1a}-1.48\%$
test_values[generalized_advantage_estimate-True-True] 9.6361ms 9.1724ms 109.0229 Ops/s 109.0914 Ops/s $\color{#d91a1a}-0.06\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.1658ms 33.3113ms 30.0199 Ops/s 28.6646 Ops/s $\color{#35bf28}+4.73\%$
test_values[td0_return_estimate-False-False] 0.2170ms 0.1659ms 6.0285 KOps/s 6.0764 KOps/s $\color{#d91a1a}-0.79\%$
test_values[td1_return_estimate-False-False] 25.8222ms 22.6418ms 44.1661 Ops/s 43.8789 Ops/s $\color{#35bf28}+0.65\%$
test_values[vec_td1_return_estimate-False-False] 34.7828ms 33.4204ms 29.9218 Ops/s 28.4687 Ops/s $\textbf{\color{#35bf28}+5.10\%}$
test_values[td_lambda_return_estimate-True-False] 35.7349ms 32.5851ms 30.6888 Ops/s 30.7137 Ops/s $\color{#d91a1a}-0.08\%$
test_values[vec_td_lambda_return_estimate-True-False] 34.7649ms 33.3460ms 29.9886 Ops/s 28.4843 Ops/s $\textbf{\color{#35bf28}+5.28\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.1831ms 7.9643ms 125.5610 Ops/s 124.0983 Ops/s $\color{#35bf28}+1.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3441ms 1.9685ms 508.0018 Ops/s 533.7615 Ops/s $\color{#d91a1a}-4.83\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4220ms 0.3482ms 2.8721 KOps/s 2.9435 KOps/s $\color{#d91a1a}-2.43\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 43.2834ms 40.7905ms 24.5155 Ops/s 21.9789 Ops/s $\textbf{\color{#35bf28}+11.54\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.5076ms 3.0163ms 331.5366 Ops/s 333.0927 Ops/s $\color{#d91a1a}-0.47\%$
test_dqn_speed 7.4812ms 6.0192ms 166.1359 Ops/s 170.3835 Ops/s $\color{#d91a1a}-2.49\%$
test_ddpg_speed 12.8183ms 11.7355ms 85.2118 Ops/s 87.2756 Ops/s $\color{#d91a1a}-2.36\%$
test_sac_speed 26.0658ms 23.7451ms 42.1140 Ops/s 43.1239 Ops/s $\color{#d91a1a}-2.34\%$
test_redq_speed 15.4778ms 13.7441ms 72.7587 Ops/s 71.8302 Ops/s $\color{#35bf28}+1.29\%$
test_redq_deprec_speed 0.1122s 24.0622ms 41.5589 Ops/s 46.6134 Ops/s $\textbf{\color{#d91a1a}-10.84\%}$
test_td3_speed 27.0886ms 16.5136ms 60.5560 Ops/s 61.3222 Ops/s $\color{#d91a1a}-1.25\%$
test_cql_speed 75.7594ms 74.2482ms 13.4683 Ops/s 13.6388 Ops/s $\color{#d91a1a}-1.25\%$
test_a2c_speed 23.0992ms 22.0556ms 45.3400 Ops/s 46.1650 Ops/s $\color{#d91a1a}-1.79\%$
test_ppo_speed 24.8921ms 22.4516ms 44.5403 Ops/s 45.6507 Ops/s $\color{#d91a1a}-2.43\%$
test_reinforce_speed 22.3049ms 21.4292ms 46.6653 Ops/s 47.4697 Ops/s $\color{#d91a1a}-1.69\%$
test_iql_speed 57.6199ms 55.7489ms 17.9376 Ops/s 18.0883 Ops/s $\color{#d91a1a}-0.83\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0211ms 2.6325ms 379.8611 Ops/s 384.4377 Ops/s $\color{#d91a1a}-1.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8394ms 0.5033ms 1.9871 KOps/s 1.9996 KOps/s $\color{#d91a1a}-0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.0969ms 0.5067ms 1.9734 KOps/s 2.1014 KOps/s $\textbf{\color{#d91a1a}-6.09\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.1833ms 2.6705ms 374.4573 Ops/s 379.3250 Ops/s $\color{#d91a1a}-1.28\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9150ms 0.5002ms 1.9993 KOps/s 2.0142 KOps/s $\color{#d91a1a}-0.74\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7436ms 0.4710ms 2.1233 KOps/s 2.1127 KOps/s $\color{#35bf28}+0.50\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.7136ms 2.4546ms 407.3957 Ops/s 422.7567 Ops/s $\color{#d91a1a}-3.63\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9136ms 0.6206ms 1.6113 KOps/s 1.6396 KOps/s $\color{#d91a1a}-1.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.2488ms 0.5983ms 1.6715 KOps/s 1.7014 KOps/s $\color{#d91a1a}-1.76\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.2045ms 2.6233ms 381.1975 Ops/s 348.5186 Ops/s $\textbf{\color{#35bf28}+9.38\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8246ms 0.5052ms 1.9796 KOps/s 1.9900 KOps/s $\color{#d91a1a}-0.52\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5854ms 0.4774ms 2.0949 KOps/s 2.0999 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.4983ms 2.6741ms 373.9538 Ops/s 376.5038 Ops/s $\color{#d91a1a}-0.68\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8070ms 0.5014ms 1.9942 KOps/s 2.0097 KOps/s $\color{#d91a1a}-0.77\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5775ms 0.4739ms 2.1102 KOps/s 2.1206 KOps/s $\color{#d91a1a}-0.49\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.6846ms 2.3964ms 417.2889 Ops/s 429.8491 Ops/s $\color{#d91a1a}-2.92\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0499ms 0.6189ms 1.6158 KOps/s 1.6334 KOps/s $\color{#d91a1a}-1.08\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7029ms 0.5885ms 1.6992 KOps/s 1.6887 KOps/s $\color{#35bf28}+0.62\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1991s 11.7751ms 84.9249 Ops/s 82.4959 Ops/s $\color{#35bf28}+2.94\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.8602ms 13.0093ms 76.8681 Ops/s 76.6106 Ops/s $\color{#35bf28}+0.34\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.0800ms 2.5169ms 397.3170 Ops/s 404.5807 Ops/s $\color{#d91a1a}-1.80\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.2063s 8.1120ms 123.2735 Ops/s 126.4355 Ops/s $\color{#d91a1a}-2.50\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.1336ms 13.1472ms 76.0620 Ops/s 76.8987 Ops/s $\color{#d91a1a}-1.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.9991ms 2.5087ms 398.6166 Ops/s 396.7170 Ops/s $\color{#35bf28}+0.48\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.2012s 8.1767ms 122.2987 Ops/s 124.5280 Ops/s $\color{#d91a1a}-1.79\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.5288ms 13.2615ms 75.4065 Ops/s 75.3856 Ops/s $\color{#35bf28}+0.03\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.2341ms 2.6641ms 375.3621 Ops/s 361.6131 Ops/s $\color{#35bf28}+3.80\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 1, 2024 •
edited
Loading

$\color{#35bf28}\textsf{\Large✔\kern{0.2cm}\normalsize OK}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}0$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1184s 0.1178s 8.4914 Ops/s 8.1684 Ops/s $\color{#35bf28}+3.95\%$
test_sync 96.2879ms 96.2007ms 10.3949 Ops/s 7.2176 Ops/s $\textbf{\color{#35bf28}+44.02\%}$
test_async 0.2519s 92.4397ms 10.8179 Ops/s 7.5052 Ops/s $\textbf{\color{#35bf28}+44.14\%}$
test_single_pixels 0.1293s 0.1290s 7.7516 Ops/s 7.2995 Ops/s $\textbf{\color{#35bf28}+6.19\%}$
test_sync_pixels 83.4619ms 82.3007ms 12.1506 Ops/s 9.6261 Ops/s $\textbf{\color{#35bf28}+26.23\%}$
test_async_pixels 0.1539s 76.5729ms 13.0595 Ops/s 10.0805 Ops/s $\textbf{\color{#35bf28}+29.55\%}$
test_simple 0.9087s 0.8460s 1.1820 Ops/s 1.1653 Ops/s $\color{#35bf28}+1.43\%$
test_transformed 1.1574s 1.0924s 0.9154 Ops/s 0.8908 Ops/s $\color{#35bf28}+2.76\%$
test_serial 2.5399s 2.4681s 0.4052 Ops/s 0.4179 Ops/s $\color{#d91a1a}-3.04\%$
test_parallel 2.1982s 2.1192s 0.4719 Ops/s 0.3103 Ops/s $\textbf{\color{#35bf28}+52.06\%}$
test_step_mdp_speed[True-True-True-True-True] 97.7520μs 33.7373μs 29.6408 KOps/s 30.1112 KOps/s $\color{#d91a1a}-1.56\%$
test_step_mdp_speed[True-True-True-True-False] 41.2810μs 19.8679μs 50.3324 KOps/s 50.2210 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[True-True-True-False-True] 43.6400μs 18.8493μs 53.0524 KOps/s 55.0863 KOps/s $\color{#d91a1a}-3.69\%$
test_step_mdp_speed[True-True-True-False-False] 30.1610μs 11.2205μs 89.1223 KOps/s 88.5313 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[True-True-False-True-True] 62.0410μs 34.9220μs 28.6353 KOps/s 28.8108 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[True-True-False-True-False] 47.0800μs 21.3439μs 46.8518 KOps/s 46.1971 KOps/s $\color{#35bf28}+1.42\%$
test_step_mdp_speed[True-True-False-False-True] 39.0000μs 20.3007μs 49.2593 KOps/s 49.5977 KOps/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[True-True-False-False-False] 29.6210μs 12.9901μs 76.9814 KOps/s 75.0725 KOps/s $\color{#35bf28}+2.54\%$
test_step_mdp_speed[True-False-True-True-True] 60.0810μs 36.5424μs 27.3654 KOps/s 27.5194 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[True-False-True-True-False] 43.0710μs 23.6549μs 42.2746 KOps/s 42.5049 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-False-True-False-True] 45.6110μs 20.4705μs 48.8509 KOps/s 50.1716 KOps/s $\color{#d91a1a}-2.63\%$
test_step_mdp_speed[True-False-True-False-False] 29.5910μs 13.1323μs 76.1480 KOps/s 76.1470 KOps/s $+0.00\%$
test_step_mdp_speed[True-False-False-True-True] 61.0520μs 38.4625μs 25.9994 KOps/s 26.3618 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-False-False-True-False] 49.7900μs 25.6279μs 39.0200 KOps/s 39.0300 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[True-False-False-False-True] 46.6810μs 22.1588μs 45.1287 KOps/s 45.5496 KOps/s $\color{#d91a1a}-0.92\%$
test_step_mdp_speed[True-False-False-False-False] 28.8110μs 14.9806μs 66.7531 KOps/s 67.1742 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-True-True-True-True] 62.9810μs 37.0642μs 26.9802 KOps/s 27.5975 KOps/s $\color{#d91a1a}-2.24\%$
test_step_mdp_speed[False-True-True-True-False] 77.5010μs 23.8880μs 41.8620 KOps/s 42.7149 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[False-True-True-False-True] 42.4210μs 24.7115μs 40.4669 KOps/s 42.1201 KOps/s $\color{#d91a1a}-3.93\%$
test_step_mdp_speed[False-True-True-False-False] 39.4000μs 15.0619μs 66.3925 KOps/s 66.0060 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-True-False-True-True] 63.0210μs 38.9400μs 25.6805 KOps/s 26.3133 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[False-True-False-True-False] 43.1710μs 25.6174μs 39.0360 KOps/s 39.6063 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[False-True-False-False-True] 52.6310μs 26.5439μs 37.6734 KOps/s 38.8495 KOps/s $\color{#d91a1a}-3.03\%$
test_step_mdp_speed[False-True-False-False-False] 31.2010μs 16.6217μs 60.1623 KOps/s 59.0336 KOps/s $\color{#35bf28}+1.91\%$
test_step_mdp_speed[False-False-True-True-True] 61.1310μs 40.8147μs 24.5010 KOps/s 25.1800 KOps/s $\color{#d91a1a}-2.70\%$
test_step_mdp_speed[False-False-True-True-False] 49.8710μs 27.4702μs 36.4031 KOps/s 37.1334 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[False-False-True-False-True] 49.8310μs 26.3332μs 37.9748 KOps/s 38.7950 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[False-False-True-False-False] 43.3910μs 16.6769μs 59.9634 KOps/s 59.4348 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-False-False-True-True] 62.1110μs 42.0423μs 23.7856 KOps/s 23.8403 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-False-False-True-False] 42.1510μs 29.1841μs 34.2653 KOps/s 34.3066 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[False-False-False-False-True] 51.4610μs 27.7215μs 36.0730 KOps/s 36.2087 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-False-False-False-False] 41.8210μs 18.3396μs 54.5270 KOps/s 53.3990 KOps/s $\color{#35bf28}+2.11\%$
test_values[generalized_advantage_estimate-True-True] 24.7766ms 24.1993ms 41.3235 Ops/s 41.2655 Ops/s $\color{#35bf28}+0.14\%$
test_values[vec_generalized_advantage_estimate-True-True] 85.2189ms 3.2675ms 306.0489 Ops/s 303.3669 Ops/s $\color{#35bf28}+0.88\%$
test_values[td0_return_estimate-False-False] 87.2020μs 61.1048μs 16.3653 KOps/s 16.6308 KOps/s $\color{#d91a1a}-1.60\%$
test_values[td1_return_estimate-False-False] 52.3954ms 51.8068ms 19.3025 Ops/s 19.0836 Ops/s $\color{#35bf28}+1.15\%$
test_values[vec_td1_return_estimate-False-False] 2.1125ms 1.7609ms 567.8887 Ops/s 565.0974 Ops/s $\color{#35bf28}+0.49\%$
test_values[td_lambda_return_estimate-True-False] 83.2811ms 82.8873ms 12.0646 Ops/s 12.0869 Ops/s $\color{#d91a1a}-0.18\%$
test_values[vec_td_lambda_return_estimate-True-False] 4.0346ms 1.7999ms 555.5972 Ops/s 553.4889 Ops/s $\color{#35bf28}+0.38\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.1555ms 22.9794ms 43.5172 Ops/s 40.7028 Ops/s $\textbf{\color{#35bf28}+6.91\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8895ms 0.7016ms 1.4253 KOps/s 1.4301 KOps/s $\color{#d91a1a}-0.34\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7376ms 0.6522ms 1.5333 KOps/s 1.5263 KOps/s $\color{#35bf28}+0.46\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5622ms 1.4562ms 686.7319 Ops/s 688.9544 Ops/s $\color{#d91a1a}-0.32\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9467ms 0.6785ms 1.4738 KOps/s 1.4881 KOps/s $\color{#d91a1a}-0.96\%$
test_dqn_speed 14.3317ms 7.6349ms 130.9775 Ops/s 131.3424 Ops/s $\color{#d91a1a}-0.28\%$
test_ddpg_speed 15.9552ms 14.6575ms 68.2244 Ops/s 69.4612 Ops/s $\color{#d91a1a}-1.78\%$
test_sac_speed 29.7355ms 28.9542ms 34.5373 Ops/s 34.9295 Ops/s $\color{#d91a1a}-1.12\%$
test_redq_speed 12.5433ms 11.4308ms 87.4833 Ops/s 87.0504 Ops/s $\color{#35bf28}+0.50\%$
test_redq_deprec_speed 23.7439ms 22.6589ms 44.1327 Ops/s 43.8605 Ops/s $\color{#35bf28}+0.62\%$
test_td3_speed 31.2824ms 19.3774ms 51.6065 Ops/s 51.7814 Ops/s $\color{#d91a1a}-0.34\%$
test_cql_speed 76.2237ms 75.2750ms 13.2846 Ops/s 13.1320 Ops/s $\color{#35bf28}+1.16\%$
test_a2c_speed 26.1727ms 25.0201ms 39.9679 Ops/s 39.3357 Ops/s $\color{#35bf28}+1.61\%$
test_ppo_speed 26.3449ms 25.4289ms 39.3253 Ops/s 38.7784 Ops/s $\color{#35bf28}+1.41\%$
test_reinforce_speed 25.0885ms 24.1533ms 41.4022 Ops/s 40.6630 Ops/s $\color{#35bf28}+1.82\%$
test_iql_speed 51.4441ms 50.4318ms 19.8288 Ops/s 19.6716 Ops/s $\color{#35bf28}+0.80\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7505ms 3.6146ms 276.6535 Ops/s 276.7714 Ops/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7401ms 0.5450ms 1.8349 KOps/s 1.7919 KOps/s $\color{#35bf28}+2.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6689ms 0.5174ms 1.9329 KOps/s 1.8915 KOps/s $\color{#35bf28}+2.19\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9175ms 3.6650ms 272.8521 Ops/s 275.4071 Ops/s $\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6879ms 0.5427ms 1.8427 KOps/s 1.7948 KOps/s $\color{#35bf28}+2.66\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6810ms 0.5152ms 1.9410 KOps/s 1.8957 KOps/s $\color{#35bf28}+2.39\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5322ms 3.3582ms 297.7763 Ops/s 297.6936 Ops/s $\color{#35bf28}+0.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7949ms 0.6694ms 1.4939 KOps/s 1.4498 KOps/s $\color{#35bf28}+3.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7941ms 0.6425ms 1.5563 KOps/s 1.5215 KOps/s $\color{#35bf28}+2.29\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7585ms 3.6495ms 274.0119 Ops/s 274.9259 Ops/s $\color{#d91a1a}-0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7098ms 0.5470ms 1.8282 KOps/s 1.8009 KOps/s $\color{#35bf28}+1.52\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7247ms 0.5178ms 1.9313 KOps/s 1.8762 KOps/s $\color{#35bf28}+2.94\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8219ms 3.6595ms 273.2589 Ops/s 271.7556 Ops/s $\color{#35bf28}+0.55\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6687ms 0.5425ms 1.8434 KOps/s 1.8048 KOps/s $\color{#35bf28}+2.14\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7798ms 0.5156ms 1.9394 KOps/s 1.8985 KOps/s $\color{#35bf28}+2.16\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4875ms 3.2991ms 303.1130 Ops/s 295.8466 Ops/s $\color{#35bf28}+2.46\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7812ms 0.6656ms 1.5024 KOps/s 1.4591 KOps/s $\color{#35bf28}+2.97\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7640ms 0.6443ms 1.5521 KOps/s 1.5133 KOps/s $\color{#35bf28}+2.56\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2341s 14.4797ms 69.0620 Ops/s 68.8537 Ops/s $\color{#35bf28}+0.30\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.5098ms 16.2526ms 61.5285 Ops/s 61.0757 Ops/s $\color{#35bf28}+0.74\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.5883ms 3.0490ms 327.9717 Ops/s 333.7040 Ops/s $\color{#d91a1a}-1.72\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.2061s 9.5502ms 104.7095 Ops/s 102.9156 Ops/s $\color{#35bf28}+1.74\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.6435ms 16.2346ms 61.5969 Ops/s 60.3687 Ops/s $\color{#35bf28}+2.03\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 8.1445ms 3.1063ms 321.9236 Ops/s 331.6069 Ops/s $\color{#d91a1a}-2.92\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.2069s 9.7463ms 102.6029 Ops/s 100.7240 Ops/s $\color{#35bf28}+1.87\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.6075ms 16.3505ms 61.1602 Ops/s 59.9792 Ops/s $\color{#35bf28}+1.97\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.1965ms 3.2097ms 311.5598 Ops/s 311.1929 Ops/s $\color{#35bf28}+0.12\%$

@vmoens vmoens added the bug Something isn't working label Feb 2, 2024
@vmoens vmoens linked an issue Feb 2, 2024 that may be closed by this pull request
[BUG] Problems with BatchedEnv on accelerated device with single envs on cpu #1864
Open
@skandermoalla skandermoalla mentioned this pull request Feb 2, 2024
Open
@vmoens vmoens changed the title [BugFix] Fix update in serial env [BugFix] Fix update in serial / parallel env Feb 5, 2024
@vmoens
Copy link
Contributor Author

vmoens commented Feb 5, 2024

@skandermoalla I updated a test in collectors that will check that everything's working for parallel env. It fails on main

@vmoens vmoens merged commit 19a920e into main Feb 5, 2024
46 of 54 checks passed
@vmoens vmoens deleted the fix-stepcount branch February 5, 2024 20:22
vmoens added a commit that referenced this pull request Feb 6, 2024
@vmoens
[BugFix] Fix update in serial / parallel env (#1866)
cd2c50f
@vmoens vmoens mentioned this pull request Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Problems with BatchedEnv on accelerated device with single envs on cpu
2 participants
@vmoens @facebook-github-bot