[NOMERG] Add @overload to forward in losses

[vmoens]

Loss functions accept non-tensordict data thanks to the tensordict.dispatch decorator.

However, we do not provide an overloaded forward, which could be useful to let users know about the typical signature using default settings.

In this PR, I propose that feature for A2C. The list input keys can be found in A2CLoss.in_keys property and also extrapolated from test/test_cost.py:TestA2C:test_a2c_notensordict.

All losses that are tested with test_<smth>_notensordict can benefit from this overload.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1893

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (13 Unrelated Failures)

As of commit 738d971 with merge base 4d52d5f ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 51ef1822a8e85db1d4f8962e3c7b465803eca551d40a166b564f6a026f219a32 /exec failed with exit code 139
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_rb.py::TestStorages::test_storage_dumps_loads[True-pytree-LazyTensorStorage-device_data0]
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  test/test_cost.py::TestA2C::test_a2c_notensordict[terminated2-done2-reward2-observation2-action2]
• Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh)
  test/test_cost.py::TestA2C::test_a2c_notensordict[terminated2-done2-reward2-observation2-action2]
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  test/test_cost.py::TestA2C::test_a2c_notensordict[terminated2-done2-reward2-observation2-action2]
• Unit-tests on Linux / tests-gpu (3.8, 12.1) / linux-job (gh)
  test/test_rb.py::TestStorages::test_storage_dumps_loads[True-pytree-LazyTensorStorage-device_data0]
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  test/test_cost.py::TestA2C::test_a2c_notensordict[terminated2-done2-reward2-observation2-action2]
• Unit-tests on Linux / tests-optdeps (3.9, 12.1) / linux-job (gh)
  test/test_cost.py::TestA2C::test_a2c_notensordict[terminated2-done2-reward2-observation2-action2]
• Unit-tests on Linux / tests-stable-gpu (3.8, 11.8) / linux-job (gh)
  test/test_cost.py::TestA2C::test_a2c_notensordict[terminated2-done2-reward2-observation2-action2]
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
  test/test_cost.py::TestA2C::test_a2c_notensordict[terminated2-done2-reward2-observation2-action2]
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)
  test/test_cost.py::TestA2C::test_a2c_notensordict[terminated2-done2-reward2-observation2-action2]
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  test/test_cost.py::TestA2C::test_a2c_notensordict[terminated2-done2-reward2-observation2-action2]
• Unit-tests on Windows / unittests-gpu / windows-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1303s	66.3885ms	15.0628 Ops/s	15.8959 Ops/s	$\textbf{\color{#d91a1a}-5.24\%}$
test_sync	39.6712ms	33.6040ms	29.7584 Ops/s	29.2267 Ops/s	$\color{#35bf28}+1.82\%$
test_async	71.7394ms	33.9623ms	29.4444 Ops/s	29.9630 Ops/s	$\color{#d91a1a}-1.73\%$
test_simple	0.4846s	0.4286s	2.3329 Ops/s	2.2666 Ops/s	$\color{#35bf28}+2.93\%$
test_transformed	0.6412s	0.5919s	1.6895 Ops/s	1.6860 Ops/s	$\color{#35bf28}+0.21\%$
test_serial	1.4869s	1.4373s	0.6958 Ops/s	0.7091 Ops/s	$\color{#d91a1a}-1.88\%$
test_parallel	1.4512s	1.3922s	0.7183 Ops/s	0.7215 Ops/s	$\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-True-True-True-True]	0.1414ms	21.4208μs	46.6837 KOps/s	46.8440 KOps/s	$\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-True-True-True-False]	54.9430μs	12.9805μs	77.0387 KOps/s	75.4129 KOps/s	$\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-True-True-False-True]	38.3320μs	12.5063μs	79.9599 KOps/s	76.2871 KOps/s	$\color{#35bf28}+4.81\%$
test_step_mdp_speed[True-True-True-False-False]	49.8730μs	7.4882μs	133.5442 KOps/s	129.4157 KOps/s	$\color{#35bf28}+3.19\%$
test_step_mdp_speed[True-True-False-True-True]	50.6050μs	22.7372μs	43.9809 KOps/s	44.0930 KOps/s	$\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-True-False-True-False]	58.3790μs	14.1629μs	70.6072 KOps/s	68.8173 KOps/s	$\color{#35bf28}+2.60\%$
test_step_mdp_speed[True-True-False-False-True]	38.1420μs	13.8280μs	72.3169 KOps/s	72.8700 KOps/s	$\color{#d91a1a}-0.76\%$
test_step_mdp_speed[True-True-False-False-False]	33.2420μs	8.7771μs	113.9329 KOps/s	111.1092 KOps/s	$\color{#35bf28}+2.54\%$
test_step_mdp_speed[True-False-True-True-True]	47.6590μs	24.1745μs	41.3660 KOps/s	41.3405 KOps/s	$\color{#35bf28}+0.06\%$
test_step_mdp_speed[True-False-True-True-False]	64.6010μs	15.7488μs	63.4968 KOps/s	63.2408 KOps/s	$\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-False-True-False-True]	46.2660μs	13.8574μs	72.1637 KOps/s	72.4645 KOps/s	$\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-False-True-False-False]	41.1770μs	8.7989μs	113.6504 KOps/s	110.8157 KOps/s	$\color{#35bf28}+2.56\%$
test_step_mdp_speed[True-False-False-True-True]	58.9500μs	25.3890μs	39.3872 KOps/s	39.7925 KOps/s	$\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-False-False-True-False]	38.7620μs	16.9666μs	58.9392 KOps/s	59.5740 KOps/s	$\color{#d91a1a}-1.07\%$
test_step_mdp_speed[True-False-False-False-True]	61.8750μs	15.0093μs	66.6255 KOps/s	66.4721 KOps/s	$\color{#35bf28}+0.23\%$
test_step_mdp_speed[True-False-False-False-False]	36.1980μs	10.0089μs	99.9112 KOps/s	98.4996 KOps/s	$\color{#35bf28}+1.43\%$
test_step_mdp_speed[False-True-True-True-True]	0.1003ms	23.8370μs	41.9515 KOps/s	41.4268 KOps/s	$\color{#35bf28}+1.27\%$
test_step_mdp_speed[False-True-True-True-False]	60.9240μs	15.6321μs	63.9711 KOps/s	63.6936 KOps/s	$\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-True-True-False-True]	43.4010μs	15.9229μs	62.8026 KOps/s	61.8748 KOps/s	$\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-True-True-False-False]	37.6900μs	10.0627μs	99.3765 KOps/s	99.3774 KOps/s	$-0.00\%$
test_step_mdp_speed[False-True-False-True-True]	34.7760μs	25.5874μs	39.0817 KOps/s	38.7370 KOps/s	$\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-True-False-True-False]	44.1520μs	16.9583μs	58.9681 KOps/s	58.4829 KOps/s	$\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-True-False-False-True]	62.1160μs	17.0905μs	58.5122 KOps/s	57.4664 KOps/s	$\color{#35bf28}+1.82\%$
test_step_mdp_speed[False-True-False-False-False]	38.2210μs	11.2944μs	88.5392 KOps/s	88.6733 KOps/s	$\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-False-True-True-True]	57.9980μs	26.5316μs	37.6909 KOps/s	37.3737 KOps/s	$\color{#35bf28}+0.85\%$
test_step_mdp_speed[False-False-True-True-False]	62.6770μs	18.2474μs	54.8025 KOps/s	54.8632 KOps/s	$\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-False-True-False-True]	40.1750μs	17.1780μs	58.2139 KOps/s	57.9089 KOps/s	$\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-False-True-False-False]	66.2840μs	11.2855μs	88.6096 KOps/s	88.8253 KOps/s	$\color{#d91a1a}-0.24\%$
test_step_mdp_speed[False-False-False-True-True]	57.2170μs	27.5107μs	36.3494 KOps/s	36.2075 KOps/s	$\color{#35bf28}+0.39\%$
test_step_mdp_speed[False-False-False-True-False]	67.0730μs	19.4044μs	51.5346 KOps/s	51.8225 KOps/s	$\color{#d91a1a}-0.56\%$
test_step_mdp_speed[False-False-False-False-True]	49.0210μs	18.2609μs	54.7617 KOps/s	54.6922 KOps/s	$\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-False-False-False-False]	36.5780μs	12.5190μs	79.8788 KOps/s	80.5668 KOps/s	$\color{#d91a1a}-0.85\%$
test_values[generalized_advantage_estimate-True-True]	9.7599ms	9.3421ms	107.0418 Ops/s	106.4952 Ops/s	$\color{#35bf28}+0.51\%$
test_values[vec_generalized_advantage_estimate-True-True]	36.7538ms	33.3543ms	29.9812 Ops/s	28.6007 Ops/s	$\color{#35bf28}+4.83\%$
test_values[td0_return_estimate-False-False]	0.2339ms	0.1733ms	5.7717 KOps/s	5.4414 KOps/s	$\textbf{\color{#35bf28}+6.07\%}$
test_values[td1_return_estimate-False-False]	27.2092ms	23.6814ms	42.2272 Ops/s	42.2110 Ops/s	$\color{#35bf28}+0.04\%$
test_values[vec_td1_return_estimate-False-False]	48.7936ms	34.1361ms	29.2945 Ops/s	28.3849 Ops/s	$\color{#35bf28}+3.20\%$
test_values[td_lambda_return_estimate-True-False]	36.9309ms	33.7608ms	29.6202 Ops/s	29.7198 Ops/s	$\color{#d91a1a}-0.34\%$
test_values[vec_td_lambda_return_estimate-True-False]	34.4871ms	33.4906ms	29.8591 Ops/s	28.3773 Ops/s	$\textbf{\color{#35bf28}+5.22\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.4548ms	8.2140ms	121.7427 Ops/s	120.0647 Ops/s	$\color{#35bf28}+1.40\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.0405ms	1.8902ms	529.0500 Ops/s	502.9398 Ops/s	$\textbf{\color{#35bf28}+5.19\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5485ms	0.3507ms	2.8516 KOps/s	2.8184 KOps/s	$\color{#35bf28}+1.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	58.6037ms	42.6974ms	23.4206 Ops/s	21.3543 Ops/s	$\textbf{\color{#35bf28}+9.68\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.5158ms	3.0329ms	329.7148 Ops/s	331.0230 Ops/s	$\color{#d91a1a}-0.40\%$
test_dqn_speed	1.8210ms	1.3844ms	722.3180 Ops/s	672.3417 Ops/s	$\textbf{\color{#35bf28}+7.43\%}$
test_ddpg_speed	3.2985ms	2.7537ms	363.1486 Ops/s	361.7231 Ops/s	$\color{#35bf28}+0.39\%$
test_sac_speed	9.8950ms	8.6338ms	115.8234 Ops/s	117.1354 Ops/s	$\color{#d91a1a}-1.12\%$
test_redq_speed	14.5317ms	13.3239ms	75.0531 Ops/s	74.1663 Ops/s	$\color{#35bf28}+1.20\%$
test_redq_deprec_speed	13.9748ms	13.4093ms	74.5750 Ops/s	71.2035 Ops/s	$\color{#35bf28}+4.73\%$
test_td3_speed	9.2443ms	8.7331ms	114.5071 Ops/s	115.5177 Ops/s	$\color{#d91a1a}-0.87\%$
test_cql_speed	38.3614ms	36.5713ms	27.3439 Ops/s	27.5464 Ops/s	$\color{#d91a1a}-0.74\%$
test_a2c_speed	9.4082ms	7.4174ms	134.8175 Ops/s	137.5849 Ops/s	$\color{#d91a1a}-2.01\%$
test_ppo_speed	9.1976ms	7.5151ms	133.0659 Ops/s	130.6164 Ops/s	$\color{#35bf28}+1.88\%$
test_reinforce_speed	7.6339ms	6.5486ms	152.7053 Ops/s	152.9957 Ops/s	$\color{#d91a1a}-0.19\%$
test_iql_speed	0.1145s	35.3986ms	28.2497 Ops/s	28.3106 Ops/s	$\color{#d91a1a}-0.21\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.9952ms	2.7101ms	368.9894 Ops/s	376.7213 Ops/s	$\color{#d91a1a}-2.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7184ms	0.5133ms	1.9483 KOps/s	1.9391 KOps/s	$\color{#35bf28}+0.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6487ms	0.4873ms	2.0521 KOps/s	2.0553 KOps/s	$\color{#d91a1a}-0.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.9379ms	2.6949ms	371.0766 Ops/s	376.3810 Ops/s	$\color{#d91a1a}-1.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7433ms	0.5054ms	1.9787 KOps/s	1.9773 KOps/s	$\color{#35bf28}+0.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7546ms	0.4811ms	2.0787 KOps/s	2.0849 KOps/s	$\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.9471ms	2.8092ms	355.9749 Ops/s	363.3679 Ops/s	$\color{#d91a1a}-2.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.7962ms	0.6289ms	1.5902 KOps/s	1.5896 KOps/s	$\color{#35bf28}+0.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	93.8742ms	0.6692ms	1.4943 KOps/s	1.4827 KOps/s	$\color{#35bf28}+0.78\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.0932ms	2.6581ms	376.2113 Ops/s	379.5852 Ops/s	$\color{#d91a1a}-0.89\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6653ms	0.5108ms	1.9578 KOps/s	1.9514 KOps/s	$\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7063ms	0.4891ms	2.0444 KOps/s	2.0439 KOps/s	$\color{#35bf28}+0.03\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.1374ms	2.8413ms	351.9552 Ops/s	374.7949 Ops/s	$\textbf{\color{#d91a1a}-6.09\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7734ms	0.5145ms	1.9437 KOps/s	1.9677 KOps/s	$\color{#d91a1a}-1.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6583ms	0.4843ms	2.0650 KOps/s	2.0583 KOps/s	$\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.2639ms	2.8170ms	354.9880 Ops/s	360.8408 Ops/s	$\color{#d91a1a}-1.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.7359ms	0.6286ms	1.5909 KOps/s	1.5926 KOps/s	$\color{#d91a1a}-0.11\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8025ms	0.6049ms	1.6533 KOps/s	1.6748 KOps/s	$\color{#d91a1a}-1.28\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1014s	7.9380ms	125.9756 Ops/s	100.9139 Ops/s	$\textbf{\color{#35bf28}+24.83\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1114s	14.9890ms	66.7155 Ops/s	75.4702 Ops/s	$\textbf{\color{#d91a1a}-11.60\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	3.6402ms	2.5037ms	399.4106 Ops/s	397.1479 Ops/s	$\color{#35bf28}+0.57\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	97.7157ms	7.8784ms	126.9296 Ops/s	126.0975 Ops/s	$\color{#35bf28}+0.66\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	15.3663ms	13.0652ms	76.5394 Ops/s	75.0328 Ops/s	$\color{#35bf28}+2.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.2462ms	2.5388ms	393.8920 Ops/s	395.6189 Ops/s	$\color{#d91a1a}-0.44\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1010s	8.1551ms	122.6223 Ops/s	99.1567 Ops/s	$\textbf{\color{#35bf28}+23.67\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.5738ms	13.3918ms	74.6724 Ops/s	73.7236 Ops/s	$\color{#35bf28}+1.29\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	96.0599ms	4.6369ms	215.6608 Ops/s	355.5665 Ops/s	$\textbf{\color{#d91a1a}-39.35\%}$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1167s	0.1154s	8.6673 Ops/s	8.6901 Ops/s	$\color{#d91a1a}-0.26\%$
test_sync	0.1753s	0.1031s	9.7015 Ops/s	9.6204 Ops/s	$\color{#35bf28}+0.84\%$
test_async	0.2602s	92.2162ms	10.8441 Ops/s	10.8466 Ops/s	$\color{#d91a1a}-0.02\%$
test_single_pixels	0.1306s	0.1277s	7.8304 Ops/s	7.8131 Ops/s	$\color{#35bf28}+0.22\%$
test_sync_pixels	82.7875ms	80.0681ms	12.4894 Ops/s	12.7804 Ops/s	$\color{#d91a1a}-2.28\%$
test_async_pixels	0.2067s	75.9462ms	13.1672 Ops/s	12.7277 Ops/s	$\color{#35bf28}+3.45\%$
test_simple	0.8932s	0.8196s	1.2202 Ops/s	1.2479 Ops/s	$\color{#d91a1a}-2.22\%$
test_transformed	1.1362s	1.0721s	0.9328 Ops/s	0.9382 Ops/s	$\color{#d91a1a}-0.58\%$
test_serial	2.4674s	2.4012s	0.4165 Ops/s	0.4197 Ops/s	$\color{#d91a1a}-0.76\%$
test_parallel	2.1133s	2.0476s	0.4884 Ops/s	0.4918 Ops/s	$\color{#d91a1a}-0.70\%$
test_step_mdp_speed[True-True-True-True-True]	0.3192ms	32.7781μs	30.5082 KOps/s	29.9541 KOps/s	$\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-True-True-True-False]	0.2068ms	19.6102μs	50.9939 KOps/s	50.5160 KOps/s	$\color{#35bf28}+0.95\%$
test_step_mdp_speed[True-True-True-False-True]	48.4510μs	18.3739μs	54.4252 KOps/s	54.0197 KOps/s	$\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-True-True-False-False]	0.2167ms	11.1904μs	89.3623 KOps/s	88.9091 KOps/s	$\color{#35bf28}+0.51\%$
test_step_mdp_speed[True-True-False-True-True]	0.2237ms	34.4366μs	29.0389 KOps/s	28.4482 KOps/s	$\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-True-False-True-False]	37.1810μs	21.3439μs	46.8517 KOps/s	46.5735 KOps/s	$\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-True-False-False-True]	0.2067ms	20.5342μs	48.6992 KOps/s	48.6086 KOps/s	$\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-True-False-False-False]	0.2299ms	12.9835μs	77.0207 KOps/s	76.7395 KOps/s	$\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-False-True-True-True]	67.3100μs	36.6327μs	27.2980 KOps/s	26.9038 KOps/s	$\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-False-True-True-False]	44.0810μs	23.5789μs	42.4108 KOps/s	42.2569 KOps/s	$\color{#35bf28}+0.36\%$
test_step_mdp_speed[True-False-True-False-True]	0.2203ms	20.3194μs	49.2141 KOps/s	48.6601 KOps/s	$\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-False-True-False-False]	30.7100μs	12.9263μs	77.3619 KOps/s	77.1211 KOps/s	$\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-False-True-True]	56.9910μs	38.0128μs	26.3069 KOps/s	26.0373 KOps/s	$\color{#35bf28}+1.04\%$
test_step_mdp_speed[True-False-False-True-False]	56.7210μs	25.0061μs	39.9902 KOps/s	39.5085 KOps/s	$\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-False-False-False-True]	0.2086ms	22.2437μs	44.9565 KOps/s	45.3633 KOps/s	$\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-False-False-False-False]	0.2102ms	14.8104μs	67.5202 KOps/s	67.5310 KOps/s	$\color{#d91a1a}-0.02\%$
test_step_mdp_speed[False-True-True-True-True]	55.9800μs	36.2945μs	27.5524 KOps/s	26.8069 KOps/s	$\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-True-True-True-False]	0.2206ms	23.5319μs	42.4955 KOps/s	42.3483 KOps/s	$\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-True-True-False-True]	0.2323ms	24.3595μs	41.0518 KOps/s	40.5500 KOps/s	$\color{#35bf28}+1.24\%$
test_step_mdp_speed[False-True-True-False-False]	43.9400μs	14.9445μs	66.9143 KOps/s	67.6034 KOps/s	$\color{#d91a1a}-1.02\%$
test_step_mdp_speed[False-True-False-True-True]	0.2429ms	38.3809μs	26.0546 KOps/s	25.8198 KOps/s	$\color{#35bf28}+0.91\%$
test_step_mdp_speed[False-True-False-True-False]	47.5400μs	25.1352μs	39.7849 KOps/s	39.3164 KOps/s	$\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-True-False-False-True]	0.2204ms	25.3190μs	39.4960 KOps/s	38.4324 KOps/s	$\color{#35bf28}+2.77\%$
test_step_mdp_speed[False-True-False-False-False]	0.2174ms	16.5479μs	60.4308 KOps/s	60.0724 KOps/s	$\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-False-True-True-True]	0.2351ms	40.3334μs	24.7933 KOps/s	24.8202 KOps/s	$\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-False-True-True-False]	77.6100μs	27.3167μs	36.6077 KOps/s	37.0217 KOps/s	$\color{#d91a1a}-1.12\%$
test_step_mdp_speed[False-False-True-False-True]	46.9600μs	25.8418μs	38.6970 KOps/s	38.0110 KOps/s	$\color{#35bf28}+1.80\%$
test_step_mdp_speed[False-False-True-False-False]	0.2108ms	16.5733μs	60.3380 KOps/s	60.8871 KOps/s	$\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-False-False-True-True]	0.2453ms	41.3702μs	24.1720 KOps/s	23.6987 KOps/s	$\color{#35bf28}+2.00\%$
test_step_mdp_speed[False-False-False-True-False]	46.3100μs	28.8254μs	34.6916 KOps/s	35.0233 KOps/s	$\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-False-False-False-True]	0.2283ms	27.2376μs	36.7140 KOps/s	35.6421 KOps/s	$\color{#35bf28}+3.01\%$
test_step_mdp_speed[False-False-False-False-False]	35.7610μs	18.2166μs	54.8950 KOps/s	55.0140 KOps/s	$\color{#d91a1a}-0.22\%$
test_values[generalized_advantage_estimate-True-True]	23.9781ms	23.6291ms	42.3208 Ops/s	41.5408 Ops/s	$\color{#35bf28}+1.88\%$
test_values[vec_generalized_advantage_estimate-True-True]	83.0216ms	3.2136ms	311.1734 Ops/s	307.7660 Ops/s	$\color{#35bf28}+1.11\%$
test_values[td0_return_estimate-False-False]	96.7200μs	59.6120μs	16.7751 KOps/s	17.0290 KOps/s	$\color{#d91a1a}-1.49\%$
test_values[td1_return_estimate-False-False]	51.2835ms	50.6817ms	19.7310 Ops/s	19.3762 Ops/s	$\color{#35bf28}+1.83\%$
test_values[vec_td1_return_estimate-False-False]	2.0426ms	1.7484ms	571.9576 Ops/s	573.6847 Ops/s	$\color{#d91a1a}-0.30\%$
test_values[td_lambda_return_estimate-True-False]	81.6269ms	80.4830ms	12.4250 Ops/s	12.1046 Ops/s	$\color{#35bf28}+2.65\%$
test_values[vec_td_lambda_return_estimate-True-False]	3.9223ms	1.7767ms	562.8370 Ops/s	556.2869 Ops/s	$\color{#35bf28}+1.18\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	22.4399ms	22.1319ms	45.1836 Ops/s	44.4375 Ops/s	$\color{#35bf28}+1.68\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.8700ms	0.6876ms	1.4542 KOps/s	1.4431 KOps/s	$\color{#35bf28}+0.77\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7062ms	0.6362ms	1.5718 KOps/s	1.5748 KOps/s	$\color{#d91a1a}-0.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.4686ms	1.4401ms	694.3831 Ops/s	694.5938 Ops/s	$\color{#d91a1a}-0.03\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9160ms	0.6555ms	1.5256 KOps/s	1.5108 KOps/s	$\color{#35bf28}+0.98\%$
test_dqn_speed	8.2734ms	1.4501ms	689.6217 Ops/s	715.6738 Ops/s	$\color{#d91a1a}-3.64\%$
test_ddpg_speed	3.1872ms	2.7655ms	361.5976 Ops/s	364.8486 Ops/s	$\color{#d91a1a}-0.89\%$
test_sac_speed	9.0553ms	8.5094ms	117.5167 Ops/s	117.8311 Ops/s	$\color{#d91a1a}-0.27\%$
test_redq_speed	11.5202ms	10.7428ms	93.0856 Ops/s	93.5116 Ops/s	$\color{#d91a1a}-0.46\%$
test_redq_deprec_speed	12.3600ms	11.7019ms	85.4559 Ops/s	83.3673 Ops/s	$\color{#35bf28}+2.51\%$
test_td3_speed	17.9667ms	8.7836ms	113.8486 Ops/s	113.9407 Ops/s	$\color{#d91a1a}-0.08\%$
test_cql_speed	27.3916ms	26.3288ms	37.9813 Ops/s	38.3736 Ops/s	$\color{#d91a1a}-1.02\%$
test_a2c_speed	6.3754ms	5.5288ms	180.8707 Ops/s	183.1007 Ops/s	$\color{#d91a1a}-1.22\%$
test_ppo_speed	6.8169ms	5.7620ms	173.5515 Ops/s	174.9248 Ops/s	$\color{#d91a1a}-0.79\%$
test_reinforce_speed	5.1210ms	4.5521ms	219.6809 Ops/s	222.9327 Ops/s	$\color{#d91a1a}-1.46\%$
test_iql_speed	20.5071ms	19.7811ms	50.5533 Ops/s	49.6857 Ops/s	$\color{#35bf28}+1.75\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.7297ms	3.5699ms	280.1181 Ops/s	280.2185 Ops/s	$\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8728ms	0.5586ms	1.7901 KOps/s	1.7914 KOps/s	$\color{#d91a1a}-0.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7401ms	0.5322ms	1.8791 KOps/s	1.8753 KOps/s	$\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.9179ms	3.5918ms	278.4085 Ops/s	279.0021 Ops/s	$\color{#d91a1a}-0.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7801ms	0.5507ms	1.8159 KOps/s	1.8075 KOps/s	$\color{#35bf28}+0.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6663ms	0.5287ms	1.8916 KOps/s	1.8826 KOps/s	$\color{#35bf28}+0.48\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.1497ms	3.7462ms	266.9391 Ops/s	272.2238 Ops/s	$\color{#d91a1a}-1.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0272ms	0.6818ms	1.4667 KOps/s	1.4511 KOps/s	$\color{#35bf28}+1.07\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8949ms	0.6561ms	1.5242 KOps/s	1.5189 KOps/s	$\color{#35bf28}+0.35\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.7655ms	3.5733ms	279.8540 Ops/s	279.8819 Ops/s	$-0.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8293ms	0.5605ms	1.7841 KOps/s	1.7946 KOps/s	$\color{#d91a1a}-0.59\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7943ms	0.5324ms	1.8783 KOps/s	1.8660 KOps/s	$\color{#35bf28}+0.66\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.8931ms	3.5937ms	278.2674 Ops/s	275.2306 Ops/s	$\color{#35bf28}+1.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6916ms	0.5589ms	1.7891 KOps/s	1.8012 KOps/s	$\color{#d91a1a}-0.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7985ms	0.5407ms	1.8496 KOps/s	1.8705 KOps/s	$\color{#d91a1a}-1.12\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.9730ms	3.7190ms	268.8886 Ops/s	266.1486 Ops/s	$\color{#35bf28}+1.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9916ms	0.6913ms	1.4465 KOps/s	1.4509 KOps/s	$\color{#d91a1a}-0.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8913ms	0.6604ms	1.5142 KOps/s	1.5045 KOps/s	$\color{#35bf28}+0.65\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1294s	10.3725ms	96.4092 Ops/s	94.7016 Ops/s	$\color{#35bf28}+1.80\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	18.1887ms	15.7249ms	63.5933 Ops/s	64.1607 Ops/s	$\color{#d91a1a}-0.88\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	6.7690ms	3.0538ms	327.4586 Ops/s	320.7710 Ops/s	$\color{#35bf28}+2.08\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1213s	10.0617ms	99.3864 Ops/s	98.9314 Ops/s	$\color{#35bf28}+0.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1326s	17.9593ms	55.6816 Ops/s	63.8321 Ops/s	$\textbf{\color{#d91a1a}-12.77\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	7.2345ms	3.0703ms	325.6960 Ops/s	329.8069 Ops/s	$\color{#d91a1a}-1.25\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1204s	10.3593ms	96.5314 Ops/s	78.2184 Ops/s	$\textbf{\color{#35bf28}+23.41\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.1797ms	15.9012ms	62.8885 Ops/s	62.9341 Ops/s	$\color{#d91a1a}-0.07\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	8.0055ms	3.3513ms	298.3884 Ops/s	299.0980 Ops/s	$\color{#d91a1a}-0.24\%$