Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Extend with a list of tensordicts #2032

Merged
merged 2 commits into from
Mar 21, 2024
Merged

[BugFix] Extend with a list of tensordicts #2032

merged 2 commits into from
Mar 21, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 21, 2024

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 21, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2032

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 2e0b5fe with merge base 660d827 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 21, 2024
@vmoens vmoens added the bug Something isn't working label Mar 21, 2024
@vmoens vmoens merged commit a69c667 into main Mar 21, 2024
60 of 62 checks passed
@vmoens vmoens deleted the fix-flip-list branch March 21, 2024 16:37
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1062s 0.1042s 9.5994 Ops/s 9.1788 Ops/s $\color{#35bf28}+4.58\%$
test_sync 93.7146ms 91.9629ms 10.8739 Ops/s 11.2485 Ops/s $\color{#d91a1a}-3.33\%$
test_async 0.1800s 89.8478ms 11.1299 Ops/s 11.3945 Ops/s $\color{#d91a1a}-2.32\%$
test_single_pixels 0.1133s 0.1125s 8.8915 Ops/s 8.9548 Ops/s $\color{#d91a1a}-0.71\%$
test_sync_pixels 69.2141ms 67.1014ms 14.9028 Ops/s 14.7465 Ops/s $\color{#35bf28}+1.06\%$
test_async_pixels 0.1106s 62.1812ms 16.0820 Ops/s 17.8260 Ops/s $\textbf{\color{#d91a1a}-9.78\%}$
test_simple 0.7415s 0.6771s 1.4770 Ops/s 1.4889 Ops/s $\color{#d91a1a}-0.80\%$
test_transformed 0.9483s 0.8824s 1.1332 Ops/s 1.1345 Ops/s $\color{#d91a1a}-0.11\%$
test_serial 2.1692s 2.1031s 0.4755 Ops/s 0.4791 Ops/s $\color{#d91a1a}-0.75\%$
test_parallel 1.8781s 1.8517s 0.5400 Ops/s 0.5559 Ops/s $\color{#d91a1a}-2.85\%$
test_step_mdp_speed[True-True-True-True-True] 0.1271ms 32.4766μs 30.7914 KOps/s 29.9466 KOps/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[True-True-True-True-False] 52.7410μs 19.2923μs 51.8341 KOps/s 50.8558 KOps/s $\color{#35bf28}+1.92\%$
test_step_mdp_speed[True-True-True-False-True] 44.9200μs 18.2327μs 54.8466 KOps/s 52.4010 KOps/s $\color{#35bf28}+4.67\%$
test_step_mdp_speed[True-True-True-False-False] 34.9810μs 11.0048μs 90.8691 KOps/s 88.9342 KOps/s $\color{#35bf28}+2.18\%$
test_step_mdp_speed[True-True-False-True-True] 64.2420μs 34.0673μs 29.3537 KOps/s 28.2126 KOps/s $\color{#35bf28}+4.04\%$
test_step_mdp_speed[True-True-False-True-False] 50.0520μs 20.9240μs 47.7920 KOps/s 46.5166 KOps/s $\color{#35bf28}+2.74\%$
test_step_mdp_speed[True-True-False-False-True] 51.8100μs 19.7613μs 50.6040 KOps/s 47.5400 KOps/s $\textbf{\color{#35bf28}+6.44\%}$
test_step_mdp_speed[True-True-False-False-False] 45.0510μs 12.7202μs 78.6152 KOps/s 76.4113 KOps/s $\color{#35bf28}+2.88\%$
test_step_mdp_speed[True-False-True-True-True] 66.4310μs 36.1844μs 27.6362 KOps/s 27.1611 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-False-True-True-False] 57.7610μs 23.0868μs 43.3148 KOps/s 42.4656 KOps/s $\color{#35bf28}+2.00\%$
test_step_mdp_speed[True-False-True-False-True] 45.2900μs 20.0730μs 49.8182 KOps/s 47.6321 KOps/s $\color{#35bf28}+4.59\%$
test_step_mdp_speed[True-False-True-False-False] 39.0800μs 12.8419μs 77.8703 KOps/s 76.2062 KOps/s $\color{#35bf28}+2.18\%$
test_step_mdp_speed[True-False-False-True-True] 66.5510μs 37.9717μs 26.3354 KOps/s 26.0427 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[True-False-False-True-False] 51.0900μs 24.7721μs 40.3680 KOps/s 39.8979 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-False-False-False-True] 48.3920μs 21.4280μs 46.6680 KOps/s 44.5657 KOps/s $\color{#35bf28}+4.72\%$
test_step_mdp_speed[True-False-False-False-False] 38.6810μs 14.5148μs 68.8953 KOps/s 67.1371 KOps/s $\color{#35bf28}+2.62\%$
test_step_mdp_speed[False-True-True-True-True] 63.3200μs 35.2076μs 28.4030 KOps/s 27.0621 KOps/s $\color{#35bf28}+4.95\%$
test_step_mdp_speed[False-True-True-True-False] 48.2610μs 22.8349μs 43.7927 KOps/s 42.4759 KOps/s $\color{#35bf28}+3.10\%$
test_step_mdp_speed[False-True-True-False-True] 52.1510μs 24.0360μs 41.6043 KOps/s 39.6316 KOps/s $\color{#35bf28}+4.98\%$
test_step_mdp_speed[False-True-True-False-False] 35.7210μs 14.4141μs 69.3765 KOps/s 66.2835 KOps/s $\color{#35bf28}+4.67\%$
test_step_mdp_speed[False-True-False-True-True] 93.6710μs 37.3325μs 26.7863 KOps/s 25.7501 KOps/s $\color{#35bf28}+4.02\%$
test_step_mdp_speed[False-True-False-True-False] 49.3400μs 24.9277μs 40.1161 KOps/s 39.4997 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[False-True-False-False-True] 54.3900μs 25.4224μs 39.3353 KOps/s 37.0335 KOps/s $\textbf{\color{#35bf28}+6.22\%}$
test_step_mdp_speed[False-True-False-False-False] 45.2800μs 16.2496μs 61.5401 KOps/s 60.5924 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[False-False-True-True-True] 79.5220μs 39.3479μs 25.4143 KOps/s 24.4737 KOps/s $\color{#35bf28}+3.84\%$
test_step_mdp_speed[False-False-True-True-False] 53.3310μs 26.7228μs 37.4212 KOps/s 36.6979 KOps/s $\color{#35bf28}+1.97\%$
test_step_mdp_speed[False-False-True-False-True] 49.8720μs 25.6670μs 38.9606 KOps/s 37.1971 KOps/s $\color{#35bf28}+4.74\%$
test_step_mdp_speed[False-False-True-False-False] 47.7200μs 16.1682μs 61.8498 KOps/s 59.6454 KOps/s $\color{#35bf28}+3.70\%$
test_step_mdp_speed[False-False-False-True-True] 74.0710μs 40.2313μs 24.8563 KOps/s 24.0998 KOps/s $\color{#35bf28}+3.14\%$
test_step_mdp_speed[False-False-False-True-False] 55.5000μs 28.5548μs 35.0204 KOps/s 34.7819 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[False-False-False-False-True] 65.1700μs 27.6216μs 36.2036 KOps/s 35.5341 KOps/s $\color{#35bf28}+1.88\%$
test_step_mdp_speed[False-False-False-False-False] 42.1610μs 18.1089μs 55.2215 KOps/s 54.5022 KOps/s $\color{#35bf28}+1.32\%$
test_values[generalized_advantage_estimate-True-True] 25.2030ms 24.7312ms 40.4347 Ops/s 42.5914 Ops/s $\textbf{\color{#d91a1a}-5.06\%}$
test_values[vec_generalized_advantage_estimate-True-True] 82.9902ms 3.2367ms 308.9598 Ops/s 313.6983 Ops/s $\color{#d91a1a}-1.51\%$
test_values[td0_return_estimate-False-False] 95.4810μs 67.2358μs 14.8730 KOps/s 15.6960 KOps/s $\textbf{\color{#d91a1a}-5.24\%}$
test_values[td1_return_estimate-False-False] 56.1998ms 55.3747ms 18.0588 Ops/s 18.8306 Ops/s $\color{#d91a1a}-4.10\%$
test_values[vec_td1_return_estimate-False-False] 2.0881ms 1.7707ms 564.7400 Ops/s 568.0593 Ops/s $\color{#d91a1a}-0.58\%$
test_values[td_lambda_return_estimate-True-False] 89.5983ms 87.1179ms 11.4787 Ops/s 11.8977 Ops/s $\color{#d91a1a}-3.52\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.1159ms 1.7676ms 565.7532 Ops/s 569.2097 Ops/s $\color{#d91a1a}-0.61\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.6688ms 24.2128ms 41.3005 Ops/s 42.9419 Ops/s $\color{#d91a1a}-3.82\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8952ms 0.7161ms 1.3965 KOps/s 1.4382 KOps/s $\color{#d91a1a}-2.90\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7273ms 0.6597ms 1.5159 KOps/s 1.5530 KOps/s $\color{#d91a1a}-2.39\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5119ms 1.4657ms 682.2729 Ops/s 690.0037 Ops/s $\color{#d91a1a}-1.12\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9381ms 0.6707ms 1.4911 KOps/s 1.5084 KOps/s $\color{#d91a1a}-1.15\%$
test_dqn_speed 1.6755ms 1.4564ms 686.6012 Ops/s 683.5157 Ops/s $\color{#35bf28}+0.45\%$
test_ddpg_speed 2.9400ms 2.7747ms 360.4050 Ops/s 362.9824 Ops/s $\color{#d91a1a}-0.71\%$
test_sac_speed 8.6164ms 8.1308ms 122.9891 Ops/s 124.3101 Ops/s $\color{#d91a1a}-1.06\%$
test_redq_speed 11.4795ms 10.5118ms 95.1316 Ops/s 98.1507 Ops/s $\color{#d91a1a}-3.08\%$
test_redq_deprec_speed 11.9409ms 11.4018ms 87.7058 Ops/s 91.1206 Ops/s $\color{#d91a1a}-3.75\%$
test_td3_speed 8.2416ms 8.0480ms 124.2541 Ops/s 124.9562 Ops/s $\color{#d91a1a}-0.56\%$
test_cql_speed 27.1965ms 25.5154ms 39.1920 Ops/s 39.7940 Ops/s $\color{#d91a1a}-1.51\%$
test_a2c_speed 5.9275ms 5.6252ms 177.7701 Ops/s 181.5353 Ops/s $\color{#d91a1a}-2.07\%$
test_ppo_speed 6.2778ms 5.9839ms 167.1148 Ops/s 155.2089 Ops/s $\textbf{\color{#35bf28}+7.67\%}$
test_reinforce_speed 4.8271ms 4.5712ms 218.7618 Ops/s 221.3028 Ops/s $\color{#d91a1a}-1.15\%$
test_iql_speed 20.2417ms 19.5416ms 51.1730 Ops/s 51.8531 Ops/s $\color{#d91a1a}-1.31\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1272ms 2.9079ms 343.8964 Ops/s 340.8649 Ops/s $\color{#35bf28}+0.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7995ms 0.5466ms 1.8296 KOps/s 1.8538 KOps/s $\color{#d91a1a}-1.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.4133ms 0.5279ms 1.8944 KOps/s 1.9091 KOps/s $\color{#d91a1a}-0.77\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1070ms 2.8977ms 345.1010 Ops/s 341.8101 Ops/s $\color{#35bf28}+0.96\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1418ms 0.5373ms 1.8611 KOps/s 1.8745 KOps/s $\color{#d91a1a}-0.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1175s 0.6218ms 1.6081 KOps/s 1.9574 KOps/s $\textbf{\color{#d91a1a}-17.84\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7166ms 1.5317ms 652.8815 Ops/s 665.0727 Ops/s $\color{#d91a1a}-1.83\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.2381ms 1.4771ms 677.0192 Ops/s 699.8169 Ops/s $\color{#d91a1a}-3.26\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1474ms 3.0282ms 330.2308 Ops/s 328.5531 Ops/s $\color{#35bf28}+0.51\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6996ms 0.6708ms 1.4907 KOps/s 1.5110 KOps/s $\color{#d91a1a}-1.34\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8320ms 0.6472ms 1.5452 KOps/s 1.5605 KOps/s $\color{#d91a1a}-0.98\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0065ms 2.8848ms 346.6498 Ops/s 343.3942 Ops/s $\color{#35bf28}+0.95\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5030ms 0.5458ms 1.8322 KOps/s 1.8532 KOps/s $\color{#d91a1a}-1.13\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7270ms 0.5273ms 1.8963 KOps/s 1.9266 KOps/s $\color{#d91a1a}-1.57\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0933ms 2.9191ms 342.5758 Ops/s 340.6105 Ops/s $\color{#35bf28}+0.58\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8202ms 0.5410ms 1.8484 KOps/s 1.8729 KOps/s $\color{#d91a1a}-1.31\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.5751ms 0.5212ms 1.9187 KOps/s 1.9566 KOps/s $\color{#d91a1a}-1.94\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1038ms 3.0204ms 331.0771 Ops/s 328.4578 Ops/s $\color{#35bf28}+0.80\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6145ms 0.6760ms 1.4793 KOps/s 1.5080 KOps/s $\color{#d91a1a}-1.91\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8161ms 0.6525ms 1.5327 KOps/s 1.5537 KOps/s $\color{#d91a1a}-1.35\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1350s 7.4135ms 134.8885 Ops/s 137.1998 Ops/s $\color{#d91a1a}-1.68\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.0765ms 15.4206ms 64.8482 Ops/s 65.2655 Ops/s $\color{#d91a1a}-0.64\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.5455ms 1.2158ms 822.5370 Ops/s 944.0558 Ops/s $\textbf{\color{#d91a1a}-12.87\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1173s 7.0802ms 141.2386 Ops/s 140.3629 Ops/s $\color{#35bf28}+0.62\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.8813ms 15.4163ms 64.8666 Ops/s 57.1020 Ops/s $\textbf{\color{#35bf28}+13.60\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.4762ms 1.1674ms 856.6065 Ops/s 942.8665 Ops/s $\textbf{\color{#d91a1a}-9.15\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1172s 9.6815ms 103.2901 Ops/s 134.3322 Ops/s $\textbf{\color{#d91a1a}-23.11\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.3158ms 15.6488ms 63.9025 Ops/s 63.5904 Ops/s $\color{#35bf28}+0.49\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.8765ms 1.6676ms 599.6797 Ops/s 641.4027 Ops/s $\textbf{\color{#d91a1a}-6.50\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 57.7864ms 55.8873ms 17.8931 Ops/s 17.1074 Ops/s $\color{#35bf28}+4.59\%$
test_sync 32.2654ms 30.1083ms 33.2134 Ops/s 32.6063 Ops/s $\color{#35bf28}+1.86\%$
test_async 51.5128ms 28.3569ms 35.2648 Ops/s 35.8007 Ops/s $\color{#d91a1a}-1.50\%$
test_simple 0.4320s 0.3561s 2.8079 Ops/s 2.8145 Ops/s $\color{#d91a1a}-0.23\%$
test_transformed 0.5588s 0.4938s 2.0253 Ops/s 2.0442 Ops/s $\color{#d91a1a}-0.93\%$
test_serial 1.2745s 1.2163s 0.8222 Ops/s 0.8255 Ops/s $\color{#d91a1a}-0.41\%$
test_parallel 1.1412s 1.0639s 0.9399 Ops/s 0.9538 Ops/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-True-True-True-True] 0.1686ms 21.2400μs 47.0809 KOps/s 46.8930 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-True-True-True-False] 43.3210μs 12.9957μs 76.9483 KOps/s 77.1666 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-True-True-False-True] 42.6390μs 12.3299μs 81.1038 KOps/s 80.0273 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[True-True-True-False-False] 52.2480μs 7.4502μs 134.2238 KOps/s 132.1111 KOps/s $\color{#35bf28}+1.60\%$
test_step_mdp_speed[True-True-False-True-True] 73.5570μs 22.7947μs 43.8698 KOps/s 44.1945 KOps/s $\color{#d91a1a}-0.73\%$
test_step_mdp_speed[True-True-False-True-False] 41.5470μs 14.2126μs 70.3603 KOps/s 70.2671 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[True-True-False-False-True] 45.5450μs 13.5913μs 73.5767 KOps/s 73.8796 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[True-True-False-False-False] 68.1370μs 8.7588μs 114.1714 KOps/s 114.1766 KOps/s $-0.00\%$
test_step_mdp_speed[True-False-True-True-True] 57.2270μs 23.9489μs 41.7557 KOps/s 41.5920 KOps/s $\color{#35bf28}+0.39\%$
test_step_mdp_speed[True-False-True-True-False] 43.5110μs 15.5131μs 64.4618 KOps/s 64.8337 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-False-True-False-True] 42.4800μs 13.5588μs 73.7528 KOps/s 73.0456 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[True-False-True-False-False] 56.0770μs 8.5466μs 117.0058 KOps/s 113.6758 KOps/s $\color{#35bf28}+2.93\%$
test_step_mdp_speed[True-False-False-True-True] 73.5980μs 25.0396μs 39.9367 KOps/s 40.1192 KOps/s $\color{#d91a1a}-0.46\%$
test_step_mdp_speed[True-False-False-True-False] 69.2070μs 16.4240μs 60.8867 KOps/s 59.9743 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-False-False-False-True] 46.5270μs 14.5865μs 68.5564 KOps/s 67.5323 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-False-False-False-False] 54.8320μs 9.7908μs 102.1371 KOps/s 99.2449 KOps/s $\color{#35bf28}+2.91\%$
test_step_mdp_speed[False-True-True-True-True] 78.0750μs 23.5293μs 42.5002 KOps/s 41.4547 KOps/s $\color{#35bf28}+2.52\%$
test_step_mdp_speed[False-True-True-True-False] 54.5820μs 15.2783μs 65.4522 KOps/s 64.1482 KOps/s $\color{#35bf28}+2.03\%$
test_step_mdp_speed[False-True-True-False-True] 53.2600μs 15.4642μs 64.6656 KOps/s 63.9267 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-True-True-False-False] 0.3803ms 9.7783μs 102.2672 KOps/s 100.2578 KOps/s $\color{#35bf28}+2.00\%$
test_step_mdp_speed[False-True-False-True-True] 39.0620μs 25.2899μs 39.5415 KOps/s 39.3283 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-True-False-True-False] 59.4520μs 16.5384μs 60.4653 KOps/s 60.0566 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[False-True-False-False-True] 46.8870μs 16.8881μs 59.2135 KOps/s 59.0817 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-True-False-False-False] 42.3790μs 11.0139μs 90.7943 KOps/s 88.8041 KOps/s $\color{#35bf28}+2.24\%$
test_step_mdp_speed[False-False-True-True-True] 61.1040μs 26.2709μs 38.0649 KOps/s 38.1666 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-False-True-True-False] 73.1670μs 17.8347μs 56.0705 KOps/s 55.5301 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-False-True-False-True] 50.1640μs 16.6955μs 59.8962 KOps/s 59.4881 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[False-False-True-False-False] 45.2940μs 11.0110μs 90.8186 KOps/s 89.2326 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[False-False-False-True-True] 0.1015ms 27.3515μs 36.5611 KOps/s 36.7148 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[False-False-False-True-False] 70.2810μs 18.8085μs 53.1675 KOps/s 53.1314 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-False-False-False-True] 70.8460μs 17.7891μs 56.2141 KOps/s 56.1228 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[False-False-False-False-False] 43.7220μs 11.9974μs 83.3512 KOps/s 82.3201 KOps/s $\color{#35bf28}+1.25\%$
test_values[generalized_advantage_estimate-True-True] 10.6574ms 10.0070ms 99.9305 Ops/s 102.7069 Ops/s $\color{#d91a1a}-2.70\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.0450ms 33.6935ms 29.6793 Ops/s 27.9687 Ops/s $\textbf{\color{#35bf28}+6.12\%}$
test_values[td0_return_estimate-False-False] 0.2335ms 0.1860ms 5.3750 KOps/s 5.5665 KOps/s $\color{#d91a1a}-3.44\%$
test_values[td1_return_estimate-False-False] 25.5983ms 24.0987ms 41.4959 Ops/s 41.6270 Ops/s $\color{#d91a1a}-0.31\%$
test_values[vec_td1_return_estimate-False-False] 34.5195ms 33.5188ms 29.8340 Ops/s 25.8235 Ops/s $\textbf{\color{#35bf28}+15.53\%}$
test_values[td_lambda_return_estimate-True-False] 37.2492ms 34.0075ms 29.4053 Ops/s 29.1639 Ops/s $\color{#35bf28}+0.83\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.0142ms 33.5747ms 29.7844 Ops/s 28.1453 Ops/s $\textbf{\color{#35bf28}+5.82\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.7984ms 8.3390ms 119.9178 Ops/s 121.3861 Ops/s $\color{#d91a1a}-1.21\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2477ms 1.9526ms 512.1446 Ops/s 489.1029 Ops/s $\color{#35bf28}+4.71\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5077ms 0.3547ms 2.8190 KOps/s 2.8268 KOps/s $\color{#d91a1a}-0.28\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 44.6177ms 42.4006ms 23.5846 Ops/s 21.3717 Ops/s $\textbf{\color{#35bf28}+10.35\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.7506ms 3.0416ms 328.7795 Ops/s 327.4273 Ops/s $\color{#35bf28}+0.41\%$
test_dqn_speed 6.5038ms 1.3637ms 733.3175 Ops/s 728.3053 Ops/s $\color{#35bf28}+0.69\%$
test_ddpg_speed 3.4706ms 2.7324ms 365.9759 Ops/s 368.4158 Ops/s $\color{#d91a1a}-0.66\%$
test_sac_speed 9.5557ms 8.3109ms 120.3238 Ops/s 110.6991 Ops/s $\textbf{\color{#35bf28}+8.69\%}$
test_redq_speed 14.1370ms 13.3297ms 75.0205 Ops/s 74.5515 Ops/s $\color{#35bf28}+0.63\%$
test_redq_deprec_speed 14.4453ms 13.4781ms 74.1947 Ops/s 73.3969 Ops/s $\color{#35bf28}+1.09\%$
test_td3_speed 16.8699ms 8.3081ms 120.3646 Ops/s 119.5693 Ops/s $\color{#35bf28}+0.67\%$
test_cql_speed 38.2845ms 36.8943ms 27.1045 Ops/s 27.3296 Ops/s $\color{#d91a1a}-0.82\%$
test_a2c_speed 8.3269ms 7.4977ms 133.3745 Ops/s 132.1094 Ops/s $\color{#35bf28}+0.96\%$
test_ppo_speed 8.6568ms 7.7272ms 129.4132 Ops/s 128.1674 Ops/s $\color{#35bf28}+0.97\%$
test_reinforce_speed 9.1562ms 6.9062ms 144.7972 Ops/s 148.0648 Ops/s $\color{#d91a1a}-2.21\%$
test_iql_speed 33.6376ms 32.8821ms 30.4117 Ops/s 30.0946 Ops/s $\color{#35bf28}+1.05\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.7071ms 2.3043ms 433.9766 Ops/s 422.0789 Ops/s $\color{#35bf28}+2.82\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 97.3875ms 0.5752ms 1.7386 KOps/s 1.9678 KOps/s $\textbf{\color{#d91a1a}-11.65\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7839ms 0.4829ms 2.0709 KOps/s 2.0559 KOps/s $\color{#35bf28}+0.73\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.6773ms 2.4098ms 414.9794 Ops/s 408.9960 Ops/s $\color{#35bf28}+1.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0943ms 0.5064ms 1.9747 KOps/s 1.9932 KOps/s $\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8425ms 0.4812ms 2.0780 KOps/s 2.0687 KOps/s $\color{#35bf28}+0.45\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.4233ms 1.3006ms 768.8892 Ops/s 762.9682 Ops/s $\color{#35bf28}+0.78\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4747ms 1.2343ms 810.1656 Ops/s 814.2878 Ops/s $\color{#d91a1a}-0.51\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5311ms 2.4513ms 407.9491 Ops/s 412.9748 Ops/s $\color{#d91a1a}-1.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0470ms 0.6317ms 1.5829 KOps/s 1.6005 KOps/s $\color{#d91a1a}-1.10\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9586ms 0.6017ms 1.6620 KOps/s 1.6706 KOps/s $\color{#d91a1a}-0.51\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.7176ms 2.3124ms 432.4601 Ops/s 414.9276 Ops/s $\color{#35bf28}+4.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6431ms 0.5105ms 1.9589 KOps/s 1.9529 KOps/s $\color{#35bf28}+0.31\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.6824ms 0.4933ms 2.0270 KOps/s 2.0320 KOps/s $\color{#d91a1a}-0.25\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6541ms 2.3403ms 427.2979 Ops/s 414.4882 Ops/s $\color{#35bf28}+3.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.4422ms 0.5105ms 1.9590 KOps/s 1.9928 KOps/s $\color{#d91a1a}-1.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9269ms 0.4787ms 2.0890 KOps/s 2.0771 KOps/s $\color{#35bf28}+0.58\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.8150ms 2.4259ms 412.2163 Ops/s 413.4694 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0594ms 0.6458ms 1.5486 KOps/s 1.5880 KOps/s $\color{#d91a1a}-2.49\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.5585ms 0.6133ms 1.6304 KOps/s 1.6798 KOps/s $\color{#d91a1a}-2.94\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1212s 8.0337ms 124.4759 Ops/s 124.8176 Ops/s $\color{#d91a1a}-0.27\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.0327ms 12.3203ms 81.1672 Ops/s 83.3477 Ops/s $\color{#d91a1a}-2.62\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.4987ms 1.0222ms 978.2360 Ops/s 963.5605 Ops/s $\color{#35bf28}+1.52\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1056s 5.6126ms 178.1707 Ops/s 173.1300 Ops/s $\color{#35bf28}+2.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1167s 14.4602ms 69.1556 Ops/s 82.8796 Ops/s $\textbf{\color{#d91a1a}-16.56\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.5856ms 1.0294ms 971.3994 Ops/s 947.4186 Ops/s $\color{#35bf28}+2.53\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1169s 6.3170ms 158.3036 Ops/s 121.3408 Ops/s $\textbf{\color{#35bf28}+30.46\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.1054ms 12.7862ms 78.2093 Ops/s 79.2984 Ops/s $\color{#d91a1a}-1.37\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9103ms 1.3402ms 746.1527 Ops/s 735.2683 Ops/s $\color{#35bf28}+1.48\%$

vmoens added a commit that referenced this pull request Mar 25, 2024
@vmoens
[BugFix] Extend with a list of tensordicts (#2032)
01301ca 
(cherry picked from commit a69c667)
vmoens added a commit that referenced this pull request Apr 7, 2024
@vmoens
[BugFix] Extend with a list of tensordicts (#2032)
fd41b39 
(cherry picked from commit a69c667)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot