Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Use setdefault in _cache_values #1910

Merged
merged 1 commit into from
Feb 15, 2024
Merged

[BugFix] Use setdefault in _cache_values #1910

merged 1 commit into from
Feb 15, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 15, 2024 •
edited
Loading

Closes #1907

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 15, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1910

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (20 Unrelated Failures)

As of commit d510f28 with merge base 6c407bd (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 15, 2024
@vmoens vmoens added bug Something isn't working Suitable for minor Suitable to be integrated in minor release (no new feature) labels Feb 15, 2024
@vmoens vmoens merged commit 45764b5 into main Feb 15, 2024
32 of 52 checks passed
@vmoens vmoens deleted the fix-cache-loss branch February 15, 2024 13:43
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 66.4795ms 64.7454ms 15.4451 Ops/s 15.2166 Ops/s $\color{#35bf28}+1.50\%$
test_sync 44.6464ms 36.1186ms 27.6866 Ops/s 28.4697 Ops/s $\color{#d91a1a}-2.75\%$
test_async 0.1051s 34.9896ms 28.5799 Ops/s 28.2624 Ops/s $\color{#35bf28}+1.12\%$
test_simple 0.5167s 0.4462s 2.2409 Ops/s 2.2189 Ops/s $\color{#35bf28}+0.99\%$
test_transformed 0.6809s 0.6179s 1.6184 Ops/s 1.5843 Ops/s $\color{#35bf28}+2.15\%$
test_serial 1.5059s 1.4469s 0.6911 Ops/s 0.6798 Ops/s $\color{#35bf28}+1.66\%$
test_parallel 1.4973s 1.4246s 0.7020 Ops/s 0.7005 Ops/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[True-True-True-True-True] 0.2094ms 21.0897μs 47.4164 KOps/s 46.7877 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-True-True-True-False] 53.5000μs 12.9170μs 77.4173 KOps/s 76.3792 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[True-True-True-False-True] 71.1530μs 12.1458μs 82.3330 KOps/s 79.3916 KOps/s $\color{#35bf28}+3.70\%$
test_step_mdp_speed[True-True-True-False-False] 34.1940μs 7.4203μs 134.7662 KOps/s 131.1265 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[True-True-False-True-True] 81.0720μs 22.2406μs 44.9628 KOps/s 43.7247 KOps/s $\color{#35bf28}+2.83\%$
test_step_mdp_speed[True-True-False-True-False] 44.8240μs 14.1606μs 70.6185 KOps/s 69.6759 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[True-True-False-False-True] 43.6220μs 13.5060μs 74.0413 KOps/s 71.5442 KOps/s $\color{#35bf28}+3.49\%$
test_step_mdp_speed[True-True-False-False-False] 30.9880μs 8.7201μs 114.6779 KOps/s 113.3043 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-False-True-True-True] 76.7950μs 23.5442μs 42.4734 KOps/s 40.9636 KOps/s $\color{#35bf28}+3.69\%$
test_step_mdp_speed[True-False-True-True-False] 47.6800μs 15.4039μs 64.9188 KOps/s 63.4295 KOps/s $\color{#35bf28}+2.35\%$
test_step_mdp_speed[True-False-True-False-True] 40.3160μs 13.4407μs 74.4009 KOps/s 71.6833 KOps/s $\color{#35bf28}+3.79\%$
test_step_mdp_speed[True-False-True-False-False] 44.7140μs 8.8963μs 112.4065 KOps/s 112.0994 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-False-False-True-True] 97.5530μs 24.8780μs 40.1961 KOps/s 38.9413 KOps/s $\color{#35bf28}+3.22\%$
test_step_mdp_speed[True-False-False-True-False] 67.4570μs 16.6661μs 60.0021 KOps/s 59.2208 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-False-False-False-True] 62.9280μs 14.6801μs 68.1195 KOps/s 66.0470 KOps/s $\color{#35bf28}+3.14\%$
test_step_mdp_speed[True-False-False-False-False] 59.8720μs 9.8535μs 101.4872 KOps/s 99.4049 KOps/s $\color{#35bf28}+2.09\%$
test_step_mdp_speed[False-True-True-True-True] 79.1780μs 23.6472μs 42.2882 KOps/s 40.9741 KOps/s $\color{#35bf28}+3.21\%$
test_step_mdp_speed[False-True-True-True-False] 50.7960μs 15.4496μs 64.7267 KOps/s 64.2180 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-True-True-False-True] 43.7620μs 15.6847μs 63.7565 KOps/s 61.6848 KOps/s $\color{#35bf28}+3.36\%$
test_step_mdp_speed[False-True-True-False-False] 45.0240μs 9.8942μs 101.0695 KOps/s 99.4845 KOps/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[False-True-False-True-True] 46.7980μs 25.4998μs 39.2160 KOps/s 38.5003 KOps/s $\color{#35bf28}+1.86\%$
test_step_mdp_speed[False-True-False-True-False] 53.9310μs 16.6946μs 59.8995 KOps/s 59.8701 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-True-False-False-True] 47.0890μs 17.0299μs 58.7201 KOps/s 57.8075 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[False-True-False-False-False] 51.7870μs 11.1609μs 89.5983 KOps/s 88.1304 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[False-False-True-True-True] 67.7070μs 26.3661μs 37.9275 KOps/s 36.9607 KOps/s $\color{#35bf28}+2.62\%$
test_step_mdp_speed[False-False-True-True-False] 46.9280μs 17.9833μs 55.6073 KOps/s 55.7476 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[False-False-True-False-True] 61.8760μs 16.7578μs 59.6737 KOps/s 57.8173 KOps/s $\color{#35bf28}+3.21\%$
test_step_mdp_speed[False-False-True-False-False] 48.5310μs 11.0083μs 90.8407 KOps/s 88.4897 KOps/s $\color{#35bf28}+2.66\%$
test_step_mdp_speed[False-False-False-True-True] 94.1270μs 27.0333μs 36.9914 KOps/s 35.9840 KOps/s $\color{#35bf28}+2.80\%$
test_step_mdp_speed[False-False-False-True-False] 60.0530μs 19.1397μs 52.2475 KOps/s 51.3303 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[False-False-False-False-True] 60.0530μs 17.8385μs 56.0584 KOps/s 54.8230 KOps/s $\color{#35bf28}+2.25\%$
test_step_mdp_speed[False-False-False-False-False] 53.4000μs 12.2061μs 81.9259 KOps/s 81.2851 KOps/s $\color{#35bf28}+0.79\%$
test_values[generalized_advantage_estimate-True-True] 10.0441ms 9.4456ms 105.8693 Ops/s 106.1859 Ops/s $\color{#d91a1a}-0.30\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.7927ms 33.4543ms 29.8915 Ops/s 29.9599 Ops/s $\color{#d91a1a}-0.23\%$
test_values[td0_return_estimate-False-False] 0.2391ms 0.1663ms 6.0138 KOps/s 5.9398 KOps/s $\color{#35bf28}+1.24\%$
test_values[td1_return_estimate-False-False] 26.8515ms 23.5664ms 42.4333 Ops/s 42.0175 Ops/s $\color{#35bf28}+0.99\%$
test_values[vec_td1_return_estimate-False-False] 34.6179ms 33.4063ms 29.9345 Ops/s 29.7860 Ops/s $\color{#35bf28}+0.50\%$
test_values[td_lambda_return_estimate-True-False] 36.7024ms 33.9267ms 29.4753 Ops/s 29.1559 Ops/s $\color{#35bf28}+1.10\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.0478ms 33.3909ms 29.9483 Ops/s 29.7505 Ops/s $\color{#35bf28}+0.66\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.4093ms 8.1875ms 122.1372 Ops/s 123.1188 Ops/s $\color{#d91a1a}-0.80\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5341ms 1.9023ms 525.6674 Ops/s 507.5419 Ops/s $\color{#35bf28}+3.57\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5468ms 0.3571ms 2.8004 KOps/s 2.8053 KOps/s $\color{#d91a1a}-0.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 40.6076ms 39.2618ms 25.4700 Ops/s 23.0842 Ops/s $\textbf{\color{#35bf28}+10.34\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6999ms 3.0496ms 327.9072 Ops/s 329.0132 Ops/s $\color{#d91a1a}-0.34\%$
test_dqn_speed 6.8789ms 1.4048ms 711.8665 Ops/s 707.5981 Ops/s $\color{#35bf28}+0.60\%$
test_ddpg_speed 88.2887ms 3.0629ms 326.4930 Ops/s 354.8704 Ops/s $\textbf{\color{#d91a1a}-8.00\%}$
test_sac_speed 10.0216ms 8.9437ms 111.8104 Ops/s 108.7075 Ops/s $\color{#35bf28}+2.85\%$
test_redq_speed 14.4594ms 13.6467ms 73.2780 Ops/s 72.6757 Ops/s $\color{#35bf28}+0.83\%$
test_redq_deprec_speed 15.2334ms 14.2581ms 70.1356 Ops/s 67.0668 Ops/s $\color{#35bf28}+4.58\%$
test_td3_speed 10.3572ms 9.1471ms 109.3244 Ops/s 107.1157 Ops/s $\color{#35bf28}+2.06\%$
test_cql_speed 38.7014ms 37.0985ms 26.9552 Ops/s 26.7722 Ops/s $\color{#35bf28}+0.68\%$
test_a2c_speed 8.5783ms 7.6956ms 129.9450 Ops/s 126.3973 Ops/s $\color{#35bf28}+2.81\%$
test_ppo_speed 9.5219ms 7.9468ms 125.8371 Ops/s 121.3755 Ops/s $\color{#35bf28}+3.68\%$
test_reinforce_speed 8.7019ms 6.9310ms 144.2790 Ops/s 141.9642 Ops/s $\color{#35bf28}+1.63\%$
test_iql_speed 35.1899ms 33.7010ms 29.6727 Ops/s 29.0080 Ops/s $\color{#35bf28}+2.29\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.1163ms 3.1045ms 322.1142 Ops/s 326.2140 Ops/s $\color{#d91a1a}-1.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7403ms 0.5327ms 1.8773 KOps/s 1.8570 KOps/s $\color{#35bf28}+1.10\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8287ms 0.5030ms 1.9879 KOps/s 1.9530 KOps/s $\color{#35bf28}+1.79\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8270ms 3.1484ms 317.6190 Ops/s 327.7118 Ops/s $\color{#d91a1a}-3.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8235ms 0.5355ms 1.8673 KOps/s 1.8832 KOps/s $\color{#d91a1a}-0.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6970ms 0.5016ms 1.9935 KOps/s 1.9778 KOps/s $\color{#35bf28}+0.79\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.4550ms 3.1379ms 318.6809 Ops/s 322.4189 Ops/s $\color{#d91a1a}-1.16\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9685ms 0.6543ms 1.5283 KOps/s 1.5213 KOps/s $\color{#35bf28}+0.47\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8483ms 0.6257ms 1.5981 KOps/s 1.5851 KOps/s $\color{#35bf28}+0.82\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.6495ms 3.0142ms 331.7581 Ops/s 331.7186 Ops/s $\color{#35bf28}+0.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7990ms 0.5375ms 1.8605 KOps/s 1.9058 KOps/s $\color{#d91a1a}-2.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6877ms 0.5052ms 1.9795 KOps/s 1.9517 KOps/s $\color{#35bf28}+1.43\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.1966ms 3.0728ms 325.4377 Ops/s 326.7345 Ops/s $\color{#d91a1a}-0.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7644ms 0.5314ms 1.8820 KOps/s 1.8767 KOps/s $\color{#35bf28}+0.28\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8474ms 0.4998ms 2.0007 KOps/s 1.9608 KOps/s $\color{#35bf28}+2.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.9470ms 3.2272ms 309.8623 Ops/s 312.2920 Ops/s $\color{#d91a1a}-0.78\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8926ms 0.6591ms 1.5171 KOps/s 1.5325 KOps/s $\color{#d91a1a}-1.00\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9869ms 0.6300ms 1.5873 KOps/s 1.5903 KOps/s $\color{#d91a1a}-0.19\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1177s 8.7971ms 113.6744 Ops/s 116.0645 Ops/s $\color{#d91a1a}-2.06\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.8498ms 13.7119ms 72.9292 Ops/s 73.3172 Ops/s $\color{#d91a1a}-0.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.1510ms 2.5891ms 386.2409 Ops/s 385.8058 Ops/s $\color{#35bf28}+0.11\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1154s 10.6967ms 93.4868 Ops/s 114.7361 Ops/s $\textbf{\color{#d91a1a}-18.52\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.6405ms 13.7764ms 72.5881 Ops/s 61.3496 Ops/s $\textbf{\color{#35bf28}+18.32\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.7332ms 2.5806ms 387.5083 Ops/s 352.4488 Ops/s $\textbf{\color{#35bf28}+9.95\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1165s 8.9039ms 112.3099 Ops/s 112.7941 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.2234ms 14.0162ms 71.3459 Ops/s 71.9269 Ops/s $\color{#d91a1a}-0.81\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.0697ms 2.8821ms 346.9705 Ops/s 350.4439 Ops/s $\color{#d91a1a}-0.99\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1175s 0.1153s 8.6730 Ops/s 8.7508 Ops/s $\color{#d91a1a}-0.89\%$
test_sync 0.1793s 0.1033s 9.6776 Ops/s 9.6521 Ops/s $\color{#35bf28}+0.26\%$
test_async 0.2639s 91.7366ms 10.9008 Ops/s 10.9301 Ops/s $\color{#d91a1a}-0.27\%$
test_single_pixels 0.1379s 0.1377s 7.2640 Ops/s 7.8495 Ops/s $\textbf{\color{#d91a1a}-7.46\%}$
test_sync_pixels 82.0068ms 77.6308ms 12.8815 Ops/s 12.3326 Ops/s $\color{#35bf28}+4.45\%$
test_async_pixels 0.1352s 72.0449ms 13.8802 Ops/s 13.2704 Ops/s $\color{#35bf28}+4.60\%$
test_simple 0.8997s 0.8255s 1.2114 Ops/s 1.2515 Ops/s $\color{#d91a1a}-3.20\%$
test_transformed 1.1446s 1.0726s 0.9323 Ops/s 0.9295 Ops/s $\color{#35bf28}+0.30\%$
test_serial 2.4627s 2.3833s 0.4196 Ops/s 0.4051 Ops/s $\color{#35bf28}+3.57\%$
test_parallel 2.0857s 2.0252s 0.4938 Ops/s 0.4826 Ops/s $\color{#35bf28}+2.33\%$
test_step_mdp_speed[True-True-True-True-True] 0.3320ms 33.2169μs 30.1052 KOps/s 30.7728 KOps/s $\color{#d91a1a}-2.17\%$
test_step_mdp_speed[True-True-True-True-False] 43.9500μs 19.4261μs 51.4772 KOps/s 51.4699 KOps/s $\color{#35bf28}+0.01\%$
test_step_mdp_speed[True-True-True-False-True] 0.2120ms 18.3269μs 54.5646 KOps/s 55.0946 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-True-True-False-False] 0.2033ms 10.9114μs 91.6470 KOps/s 91.0314 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[True-True-False-True-True] 61.9510μs 34.1902μs 29.2482 KOps/s 29.3291 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-True-False-True-False] 0.2238ms 21.3484μs 46.8419 KOps/s 47.4139 KOps/s $\color{#d91a1a}-1.21\%$
test_step_mdp_speed[True-True-False-False-True] 0.2418ms 20.1887μs 49.5326 KOps/s 50.0434 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-True-False-False-False] 0.2403ms 12.8050μs 78.0945 KOps/s 78.2503 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[True-False-True-True-True] 69.2710μs 35.9780μs 27.7947 KOps/s 27.7981 KOps/s $\color{#d91a1a}-0.01\%$
test_step_mdp_speed[True-False-True-True-False] 49.0300μs 23.1717μs 43.1560 KOps/s 43.6337 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-False-True-False-True] 0.2264ms 19.9971μs 50.0072 KOps/s 50.1101 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[True-False-True-False-False] 35.6800μs 12.7597μs 78.3718 KOps/s 78.3721 KOps/s $-0.00\%$
test_step_mdp_speed[True-False-False-True-True] 0.2494ms 37.4830μs 26.6787 KOps/s 26.7770 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-False-False-True-False] 0.2146ms 24.8581μs 40.2283 KOps/s 40.7053 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[True-False-False-False-True] 0.2202ms 21.8505μs 45.7654 KOps/s 46.6169 KOps/s $\color{#d91a1a}-1.83\%$
test_step_mdp_speed[True-False-False-False-False] 39.8710μs 14.5877μs 68.5510 KOps/s 68.6986 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-True-True-True-True] 0.2632ms 36.1744μs 27.6438 KOps/s 28.1325 KOps/s $\color{#d91a1a}-1.74\%$
test_step_mdp_speed[False-True-True-True-False] 0.2162ms 23.5434μs 42.4748 KOps/s 43.6769 KOps/s $\color{#d91a1a}-2.75\%$
test_step_mdp_speed[False-True-True-False-True] 0.2183ms 24.0994μs 41.4948 KOps/s 41.3589 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[False-True-True-False-False] 46.3910μs 14.6935μs 68.0572 KOps/s 69.6557 KOps/s $\color{#d91a1a}-2.29\%$
test_step_mdp_speed[False-True-False-True-True] 0.2795ms 38.2795μs 26.1237 KOps/s 26.4103 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[False-True-False-True-False] 49.4000μs 25.1946μs 39.6910 KOps/s 39.9859 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[False-True-False-False-True] 0.2560ms 25.8307μs 38.7136 KOps/s 39.9541 KOps/s $\color{#d91a1a}-3.10\%$
test_step_mdp_speed[False-True-False-False-False] 0.2236ms 16.5806μs 60.3116 KOps/s 61.4613 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[False-False-True-True-True] 0.2478ms 39.5345μs 25.2944 KOps/s 25.2159 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-True-True-False] 57.4100μs 27.0991μs 36.9016 KOps/s 37.8743 KOps/s $\color{#d91a1a}-2.57\%$
test_step_mdp_speed[False-False-True-False-True] 0.2162ms 25.8592μs 38.6710 KOps/s 38.7993 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-False-True-False-False] 0.2009ms 16.5629μs 60.3759 KOps/s 60.9647 KOps/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[False-False-False-True-True] 0.2293ms 41.0327μs 24.3708 KOps/s 24.3296 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[False-False-False-True-False] 53.5010μs 28.7641μs 34.7656 KOps/s 35.5788 KOps/s $\color{#d91a1a}-2.29\%$
test_step_mdp_speed[False-False-False-False-True] 59.2110μs 27.3283μs 36.5921 KOps/s 37.2597 KOps/s $\color{#d91a1a}-1.79\%$
test_step_mdp_speed[False-False-False-False-False] 0.2381ms 18.3566μs 54.4763 KOps/s 55.9891 KOps/s $\color{#d91a1a}-2.70\%$
test_values[generalized_advantage_estimate-True-True] 25.2155ms 24.8420ms 40.2544 Ops/s 41.6527 Ops/s $\color{#d91a1a}-3.36\%$
test_values[vec_generalized_advantage_estimate-True-True] 82.0570ms 3.2017ms 312.3372 Ops/s 293.3878 Ops/s $\textbf{\color{#35bf28}+6.46\%}$
test_values[td0_return_estimate-False-False] 0.1001ms 60.8962μs 16.4214 KOps/s 16.8912 KOps/s $\color{#d91a1a}-2.78\%$
test_values[td1_return_estimate-False-False] 55.7918ms 54.6389ms 18.3020 Ops/s 19.5282 Ops/s $\textbf{\color{#d91a1a}-6.28\%}$
test_values[vec_td1_return_estimate-False-False] 2.0476ms 1.7555ms 569.6368 Ops/s 575.7936 Ops/s $\color{#d91a1a}-1.07\%$
test_values[td_lambda_return_estimate-True-False] 89.0848ms 85.6033ms 11.6818 Ops/s 12.2543 Ops/s $\color{#d91a1a}-4.67\%$
test_values[vec_td_lambda_return_estimate-True-False] 4.2128ms 1.7928ms 557.7749 Ops/s 563.2928 Ops/s $\color{#d91a1a}-0.98\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.9250ms 23.8154ms 41.9897 Ops/s 43.7944 Ops/s $\color{#d91a1a}-4.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8784ms 0.6993ms 1.4299 KOps/s 1.4760 KOps/s $\color{#d91a1a}-3.12\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7099ms 0.6449ms 1.5505 KOps/s 1.5915 KOps/s $\color{#d91a1a}-2.57\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5193ms 1.4513ms 689.0225 Ops/s 698.5582 Ops/s $\color{#d91a1a}-1.37\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9341ms 0.6635ms 1.5071 KOps/s 1.5431 KOps/s $\color{#d91a1a}-2.33\%$
test_dqn_speed 9.4429ms 1.4448ms 692.1359 Ops/s 705.8264 Ops/s $\color{#d91a1a}-1.94\%$
test_ddpg_speed 3.1352ms 2.7489ms 363.7848 Ops/s 364.2430 Ops/s $\color{#d91a1a}-0.13\%$
test_sac_speed 8.6762ms 8.3674ms 119.5113 Ops/s 117.9255 Ops/s $\color{#35bf28}+1.34\%$
test_redq_speed 12.2745ms 10.8588ms 92.0914 Ops/s 92.3302 Ops/s $\color{#d91a1a}-0.26\%$
test_redq_deprec_speed 12.7637ms 11.8387ms 84.4690 Ops/s 85.7129 Ops/s $\color{#d91a1a}-1.45\%$
test_td3_speed 18.3585ms 8.6247ms 115.9455 Ops/s 114.9410 Ops/s $\color{#35bf28}+0.87\%$
test_cql_speed 27.3442ms 26.0764ms 38.3489 Ops/s 38.8608 Ops/s $\color{#d91a1a}-1.32\%$
test_a2c_speed 5.9801ms 5.3844ms 185.7202 Ops/s 186.7213 Ops/s $\color{#d91a1a}-0.54\%$
test_ppo_speed 5.9083ms 5.6685ms 176.4142 Ops/s 175.7513 Ops/s $\color{#35bf28}+0.38\%$
test_reinforce_speed 5.2610ms 4.5884ms 217.9426 Ops/s 226.1285 Ops/s $\color{#d91a1a}-3.62\%$
test_iql_speed 21.1281ms 20.2920ms 49.2804 Ops/s 49.8672 Ops/s $\color{#d91a1a}-1.18\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.8161ms 3.5543ms 281.3490 Ops/s 282.1524 Ops/s $\color{#d91a1a}-0.28\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8285ms 0.5599ms 1.7859 KOps/s 1.7838 KOps/s $\color{#35bf28}+0.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7399ms 0.5309ms 1.8836 KOps/s 1.8608 KOps/s $\color{#35bf28}+1.22\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9337ms 3.6092ms 277.0719 Ops/s 281.6886 Ops/s $\color{#d91a1a}-1.64\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8342ms 0.5550ms 1.8018 KOps/s 1.8090 KOps/s $\color{#d91a1a}-0.39\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7313ms 0.5289ms 1.8908 KOps/s 1.8958 KOps/s $\color{#d91a1a}-0.26\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.1120ms 3.7042ms 269.9615 Ops/s 272.6377 Ops/s $\color{#d91a1a}-0.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9105ms 0.6866ms 1.4565 KOps/s 1.4554 KOps/s $\color{#35bf28}+0.07\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8707ms 0.6590ms 1.5174 KOps/s 1.5198 KOps/s $\color{#d91a1a}-0.16\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.8356ms 3.5719ms 279.9633 Ops/s 281.1445 Ops/s $\color{#d91a1a}-0.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7116ms 0.5641ms 1.7728 KOps/s 1.7824 KOps/s $\color{#d91a1a}-0.54\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6632ms 0.5377ms 1.8597 KOps/s 1.8677 KOps/s $\color{#d91a1a}-0.43\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.0150ms 3.6079ms 277.1702 Ops/s 277.4196 Ops/s $\color{#d91a1a}-0.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9610ms 0.5565ms 1.7968 KOps/s 1.7933 KOps/s $\color{#35bf28}+0.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7222ms 0.5322ms 1.8790 KOps/s 1.8861 KOps/s $\color{#d91a1a}-0.37\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.9566ms 3.7077ms 269.7114 Ops/s 273.9093 Ops/s $\color{#d91a1a}-1.53\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8343ms 0.6839ms 1.4621 KOps/s 1.4561 KOps/s $\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9268ms 0.6607ms 1.5135 KOps/s 1.5121 KOps/s $\color{#35bf28}+0.09\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1359s 10.7208ms 93.2763 Ops/s 92.6689 Ops/s $\color{#35bf28}+0.66\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1267s 18.4937ms 54.0725 Ops/s 62.8612 Ops/s $\textbf{\color{#d91a1a}-13.98\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.5402ms 3.0898ms 323.6418 Ops/s 321.8477 Ops/s $\color{#35bf28}+0.56\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1261s 10.3001ms 97.0864 Ops/s 96.4656 Ops/s $\color{#35bf28}+0.64\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.5352ms 16.2415ms 61.5708 Ops/s 62.8733 Ops/s $\color{#d91a1a}-2.07\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.1504ms 3.0922ms 323.3971 Ops/s 321.5521 Ops/s $\color{#35bf28}+0.57\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1268s 10.6238ms 94.1280 Ops/s 93.4469 Ops/s $\color{#35bf28}+0.73\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.7943ms 16.4791ms 60.6829 Ops/s 54.1324 Ops/s $\textbf{\color{#35bf28}+12.10\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.3832ms 3.4158ms 292.7613 Ops/s 289.2913 Ops/s $\color{#35bf28}+1.20\%$

@svnv-svsv-jm
Copy link

Thanks!

vmoens added a commit that referenced this pull request Feb 28, 2024
@vmoens
[BugFix] Use setdefault in _cache_values (#1910)
52161f1
@vmoens vmoens mentioned this pull request Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Suitable for minor Suitable to be integrated in minor release (no new feature)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] I had to patch these 2 methods in order to run my script
3 participants
@vmoens @svnv-svsv-jm @facebook-github-bot