[Refactor] Use filter_empty=False in apply for params

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1882

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (4 Unrelated Failures)

As of commit 2bb68e7 with merge base e53eb73 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 8e2375c6f7fcd023eab531d569d7b495d757b60bebbb2bba36b75b42b6b4f913 /exec failed with exit code 139
• Unit-tests on Linux / tests-optdeps (3.9, 12.1) / linux-job (gh)
  test/test_rb.py::TestStorages::test_storage_dumps_loads[True-pytree-LazyTensorStorage-device_data0]
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128
• Unit-tests on Windows / unittests-gpu / windows-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1281s	66.5641ms	15.0231 Ops/s	16.1774 Ops/s	$\textbf{\color{#d91a1a}-7.14\%}$
test_sync	53.2975ms	37.3006ms	26.8092 Ops/s	29.4406 Ops/s	$\textbf{\color{#d91a1a}-8.94\%}$
test_async	50.4739ms	32.3245ms	30.9362 Ops/s	30.8720 Ops/s	$\color{#35bf28}+0.21\%$
test_simple	0.4888s	0.4301s	2.3249 Ops/s	2.3163 Ops/s	$\color{#35bf28}+0.37\%$
test_transformed	0.6522s	0.5953s	1.6799 Ops/s	1.6838 Ops/s	$\color{#d91a1a}-0.23\%$
test_serial	1.4700s	1.4140s	0.7072 Ops/s	0.7069 Ops/s	$\color{#35bf28}+0.04\%$
test_parallel	1.4573s	1.4033s	0.7126 Ops/s	0.7102 Ops/s	$\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-True-True-True-True]	0.1257ms	21.2596μs	47.0375 KOps/s	45.9885 KOps/s	$\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-True-True-True-False]	42.6590μs	13.0747μs	76.4838 KOps/s	75.7368 KOps/s	$\color{#35bf28}+0.99\%$
test_step_mdp_speed[True-True-True-False-True]	38.8620μs	12.3375μs	81.0539 KOps/s	77.9450 KOps/s	$\color{#35bf28}+3.99\%$
test_step_mdp_speed[True-True-True-False-False]	47.2980μs	7.6825μs	130.1667 KOps/s	129.3939 KOps/s	$\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-True-False-True-True]	63.5400μs	22.4430μs	44.5574 KOps/s	43.1206 KOps/s	$\color{#35bf28}+3.33\%$
test_step_mdp_speed[True-True-False-True-False]	41.5780μs	14.3057μs	69.9023 KOps/s	68.7715 KOps/s	$\color{#35bf28}+1.64\%$
test_step_mdp_speed[True-True-False-False-True]	47.5290μs	13.6403μs	73.3121 KOps/s	71.1930 KOps/s	$\color{#35bf28}+2.98\%$
test_step_mdp_speed[True-True-False-False-False]	48.4810μs	8.8011μs	113.6217 KOps/s	111.6673 KOps/s	$\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-False-True-True-True]	67.4560μs	23.7328μs	42.1358 KOps/s	40.8809 KOps/s	$\color{#35bf28}+3.07\%$
test_step_mdp_speed[True-False-True-True-False]	50.0230μs	15.6007μs	64.0998 KOps/s	63.0876 KOps/s	$\color{#35bf28}+1.60\%$
test_step_mdp_speed[True-False-True-False-True]	45.4150μs	13.6264μs	73.3871 KOps/s	71.3593 KOps/s	$\color{#35bf28}+2.84\%$
test_step_mdp_speed[True-False-True-False-False]	48.3110μs	8.8444μs	113.0656 KOps/s	111.1893 KOps/s	$\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-False-False-True-True]	59.5010μs	24.9952μs	40.0077 KOps/s	38.8310 KOps/s	$\color{#35bf28}+3.03\%$
test_step_mdp_speed[True-False-False-True-False]	38.9230μs	16.8975μs	59.1802 KOps/s	58.3723 KOps/s	$\color{#35bf28}+1.38\%$
test_step_mdp_speed[True-False-False-False-True]	36.4080μs	14.8334μs	67.4153 KOps/s	65.4629 KOps/s	$\color{#35bf28}+2.98\%$
test_step_mdp_speed[True-False-False-False-False]	35.2850μs	10.1264μs	98.7519 KOps/s	97.9477 KOps/s	$\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-True-True-True-True]	50.0240μs	23.8188μs	41.9836 KOps/s	40.3412 KOps/s	$\color{#35bf28}+4.07\%$
test_step_mdp_speed[False-True-True-True-False]	41.9990μs	15.7390μs	63.5366 KOps/s	61.8852 KOps/s	$\color{#35bf28}+2.67\%$
test_step_mdp_speed[False-True-True-False-True]	48.4700μs	15.9614μs	62.6511 KOps/s	60.1061 KOps/s	$\color{#35bf28}+4.23\%$
test_step_mdp_speed[False-True-True-False-False]	48.4400μs	10.0364μs	99.6373 KOps/s	97.6432 KOps/s	$\color{#35bf28}+2.04\%$
test_step_mdp_speed[False-True-False-True-True]	49.4320μs	25.2134μs	39.6614 KOps/s	38.1591 KOps/s	$\color{#35bf28}+3.94\%$
test_step_mdp_speed[False-True-False-True-False]	44.7740μs	16.7918μs	59.5528 KOps/s	58.4847 KOps/s	$\color{#35bf28}+1.83\%$
test_step_mdp_speed[False-True-False-False-True]	42.8890μs	16.9491μs	59.0003 KOps/s	57.0727 KOps/s	$\color{#35bf28}+3.38\%$
test_step_mdp_speed[False-True-False-False-False]	35.2760μs	11.2719μs	88.7161 KOps/s	87.0115 KOps/s	$\color{#35bf28}+1.96\%$
test_step_mdp_speed[False-False-True-True-True]	90.2380μs	26.0656μs	38.3647 KOps/s	36.8704 KOps/s	$\color{#35bf28}+4.05\%$
test_step_mdp_speed[False-False-True-True-False]	41.7480μs	18.1900μs	54.9753 KOps/s	54.6064 KOps/s	$\color{#35bf28}+0.68\%$
test_step_mdp_speed[False-False-True-False-True]	42.2590μs	17.0041μs	58.8093 KOps/s	57.2376 KOps/s	$\color{#35bf28}+2.75\%$
test_step_mdp_speed[False-False-True-False-False]	89.5840μs	11.4532μs	87.3115 KOps/s	87.7151 KOps/s	$\color{#d91a1a}-0.46\%$
test_step_mdp_speed[False-False-False-True-True]	0.1021ms	27.1509μs	36.8312 KOps/s	35.6340 KOps/s	$\color{#35bf28}+3.36\%$
test_step_mdp_speed[False-False-False-True-False]	59.7410μs	19.1541μs	52.2081 KOps/s	51.3102 KOps/s	$\color{#35bf28}+1.75\%$
test_step_mdp_speed[False-False-False-False-True]	62.4500μs	18.0698μs	55.3410 KOps/s	54.1430 KOps/s	$\color{#35bf28}+2.21\%$
test_step_mdp_speed[False-False-False-False-False]	43.6680μs	12.3911μs	80.7033 KOps/s	79.7897 KOps/s	$\color{#35bf28}+1.15\%$
test_values[generalized_advantage_estimate-True-True]	12.0560ms	9.3479ms	106.9757 Ops/s	107.1324 Ops/s	$\color{#d91a1a}-0.15\%$
test_values[vec_generalized_advantage_estimate-True-True]	36.5309ms	35.0873ms	28.5003 Ops/s	28.4236 Ops/s	$\color{#35bf28}+0.27\%$
test_values[td0_return_estimate-False-False]	0.2344ms	0.1628ms	6.1435 KOps/s	6.0700 KOps/s	$\color{#35bf28}+1.21\%$
test_values[td1_return_estimate-False-False]	23.5617ms	23.2996ms	42.9191 Ops/s	43.4374 Ops/s	$\color{#d91a1a}-1.19\%$
test_values[vec_td1_return_estimate-False-False]	36.7567ms	35.4336ms	28.2218 Ops/s	28.1136 Ops/s	$\color{#35bf28}+0.38\%$
test_values[td_lambda_return_estimate-True-False]	36.8415ms	33.9668ms	29.4405 Ops/s	30.0811 Ops/s	$\color{#d91a1a}-2.13\%$
test_values[vec_td_lambda_return_estimate-True-False]	36.3667ms	35.1921ms	28.4155 Ops/s	28.1961 Ops/s	$\color{#35bf28}+0.78\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.1533ms	8.0492ms	124.2355 Ops/s	121.7241 Ops/s	$\color{#35bf28}+2.06\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.4277ms	1.9666ms	508.4930 Ops/s	509.5299 Ops/s	$\color{#d91a1a}-0.20\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5460ms	0.3502ms	2.8552 KOps/s	2.8943 KOps/s	$\color{#d91a1a}-1.35\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	49.3851ms	46.2573ms	21.6182 Ops/s	21.9137 Ops/s	$\color{#d91a1a}-1.35\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.7277ms	3.0112ms	332.0901 Ops/s	332.5866 Ops/s	$\color{#d91a1a}-0.15\%$
test_dqn_speed	12.7880ms	5.9781ms	167.2779 Ops/s	157.9498 Ops/s	$\textbf{\color{#35bf28}+5.91\%}$
test_ddpg_speed	13.6796ms	11.7335ms	85.2259 Ops/s	87.0525 Ops/s	$\color{#d91a1a}-2.10\%$
test_sac_speed	26.3342ms	23.8277ms	41.9680 Ops/s	43.0570 Ops/s	$\color{#d91a1a}-2.53\%$
test_redq_speed	15.7895ms	13.9576ms	71.6456 Ops/s	70.5613 Ops/s	$\color{#35bf28}+1.54\%$
test_redq_deprec_speed	22.3345ms	21.6235ms	46.2461 Ops/s	46.5349 Ops/s	$\color{#d91a1a}-0.62\%$
test_td3_speed	25.2626ms	16.6831ms	59.9410 Ops/s	61.3623 Ops/s	$\color{#d91a1a}-2.32\%$
test_cql_speed	75.4951ms	74.0589ms	13.5028 Ops/s	13.5803 Ops/s	$\color{#d91a1a}-0.57\%$
test_a2c_speed	23.3502ms	22.0417ms	45.3685 Ops/s	45.6002 Ops/s	$\color{#d91a1a}-0.51\%$
test_ppo_speed	23.6527ms	22.2319ms	44.9805 Ops/s	45.3522 Ops/s	$\color{#d91a1a}-0.82\%$
test_reinforce_speed	23.2356ms	21.1902ms	47.1916 Ops/s	47.5387 Ops/s	$\color{#d91a1a}-0.73\%$
test_iql_speed	56.2818ms	55.6891ms	17.9568 Ops/s	18.1266 Ops/s	$\color{#d91a1a}-0.94\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.8560ms	2.5741ms	388.4844 Ops/s	381.7969 Ops/s	$\color{#35bf28}+1.75\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7925ms	0.4956ms	2.0178 KOps/s	1.9997 KOps/s	$\color{#35bf28}+0.91\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.8334ms	0.4990ms	2.0039 KOps/s	2.1325 KOps/s	$\textbf{\color{#d91a1a}-6.03\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.8453ms	2.5346ms	394.5447 Ops/s	389.2951 Ops/s	$\color{#35bf28}+1.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7780ms	0.4910ms	2.0367 KOps/s	2.0330 KOps/s	$\color{#35bf28}+0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7774ms	0.4656ms	2.1476 KOps/s	2.1418 KOps/s	$\color{#35bf28}+0.27\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.6394ms	2.3434ms	426.7250 Ops/s	421.3179 Ops/s	$\color{#35bf28}+1.28\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.7146ms	0.6061ms	1.6500 KOps/s	1.6287 KOps/s	$\color{#35bf28}+1.31\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7819ms	0.5812ms	1.7205 KOps/s	1.7197 KOps/s	$\color{#35bf28}+0.04\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.9340ms	2.5784ms	387.8413 Ops/s	373.4142 Ops/s	$\color{#35bf28}+3.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8195ms	0.4988ms	2.0048 KOps/s	1.9868 KOps/s	$\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5588ms	0.4695ms	2.1300 KOps/s	2.1281 KOps/s	$\color{#35bf28}+0.09\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.9355ms	2.6038ms	384.0491 Ops/s	384.8575 Ops/s	$\color{#d91a1a}-0.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8657ms	0.4935ms	2.0262 KOps/s	2.0168 KOps/s	$\color{#35bf28}+0.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7185ms	0.4666ms	2.1433 KOps/s	2.1411 KOps/s	$\color{#35bf28}+0.10\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.5698ms	2.3836ms	419.5298 Ops/s	408.1382 Ops/s	$\color{#35bf28}+2.79\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1221ms	0.6180ms	1.6181 KOps/s	1.6116 KOps/s	$\color{#35bf28}+0.40\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9714ms	0.5871ms	1.7033 KOps/s	1.6980 KOps/s	$\color{#35bf28}+0.32\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	98.9394ms	9.7717ms	102.3360 Ops/s	131.2778 Ops/s	$\textbf{\color{#d91a1a}-22.05\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	16.1373ms	13.2220ms	75.6314 Ops/s	75.1691 Ops/s	$\color{#35bf28}+0.61\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	4.2733ms	2.5107ms	398.2953 Ops/s	400.0337 Ops/s	$\color{#d91a1a}-0.43\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	99.3703ms	9.7391ms	102.6788 Ops/s	104.7513 Ops/s	$\color{#d91a1a}-1.98\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	16.2387ms	13.2659ms	75.3811 Ops/s	75.8913 Ops/s	$\color{#d91a1a}-0.67\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.9442ms	2.5484ms	392.4098 Ops/s	396.5255 Ops/s	$\color{#d91a1a}-1.04\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	97.1207ms	7.9989ms	125.0177 Ops/s	125.3772 Ops/s	$\color{#d91a1a}-0.29\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	15.6115ms	13.3766ms	74.7572 Ops/s	65.6636 Ops/s	$\textbf{\color{#35bf28}+13.85\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	4.9831ms	2.7110ms	368.8677 Ops/s	371.3813 Ops/s	$\color{#d91a1a}-0.68\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1175s	0.1166s	8.5727 Ops/s	8.4891 Ops/s	$\color{#35bf28}+0.98\%$
test_sync	95.6636ms	95.4589ms	10.4757 Ops/s	10.4090 Ops/s	$\color{#35bf28}+0.64\%$
test_async	0.2530s	91.7237ms	10.9023 Ops/s	10.8320 Ops/s	$\color{#35bf28}+0.65\%$
test_single_pixels	0.1389s	0.1386s	7.2139 Ops/s	7.1666 Ops/s	$\color{#35bf28}+0.66\%$
test_sync_pixels	83.9133ms	82.6409ms	12.1005 Ops/s	11.7185 Ops/s	$\color{#35bf28}+3.26\%$
test_async_pixels	0.1455s	74.1120ms	13.4931 Ops/s	13.2814 Ops/s	$\color{#35bf28}+1.59\%$
test_simple	0.9312s	0.8457s	1.1825 Ops/s	1.1886 Ops/s	$\color{#d91a1a}-0.52\%$
test_transformed	1.1584s	1.0965s	0.9120 Ops/s	0.9139 Ops/s	$\color{#d91a1a}-0.21\%$
test_serial	2.4376s	2.4357s	0.4106 Ops/s	0.4117 Ops/s	$\color{#d91a1a}-0.27\%$
test_parallel	2.1853s	2.1211s	0.4715 Ops/s	0.4689 Ops/s	$\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-True-True-True-True]	75.3120μs	32.7005μs	30.5806 KOps/s	30.6243 KOps/s	$\color{#d91a1a}-0.14\%$
test_step_mdp_speed[True-True-True-True-False]	44.0300μs	19.5061μs	51.2661 KOps/s	50.5532 KOps/s	$\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-True-True-False-True]	46.3810μs	18.4700μs	54.1418 KOps/s	54.4930 KOps/s	$\color{#d91a1a}-0.64\%$
test_step_mdp_speed[True-True-True-False-False]	29.8300μs	11.1814μs	89.4341 KOps/s	89.1044 KOps/s	$\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-True-False-True-True]	75.6420μs	34.3703μs	29.0949 KOps/s	29.2553 KOps/s	$\color{#d91a1a}-0.55\%$
test_step_mdp_speed[True-True-False-True-False]	44.4110μs	21.3588μs	46.8191 KOps/s	47.1917 KOps/s	$\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-True-False-False-True]	48.9000μs	20.3677μs	49.0973 KOps/s	49.5681 KOps/s	$\color{#d91a1a}-0.95\%$
test_step_mdp_speed[True-True-False-False-False]	36.2000μs	13.0724μs	76.4969 KOps/s	76.7234 KOps/s	$\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-False-True-True-True]	60.0510μs	36.6953μs	27.2515 KOps/s	27.3981 KOps/s	$\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-False-True-True-False]	52.1410μs	23.4076μs	42.7212 KOps/s	42.4389 KOps/s	$\color{#35bf28}+0.67\%$
test_step_mdp_speed[True-False-True-False-True]	58.7210μs	20.2839μs	49.3001 KOps/s	49.4151 KOps/s	$\color{#d91a1a}-0.23\%$
test_step_mdp_speed[True-False-True-False-False]	44.6910μs	13.1227μs	76.2041 KOps/s	76.1711 KOps/s	$\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-False-False-True-True]	71.6310μs	38.3533μs	26.0733 KOps/s	26.1536 KOps/s	$\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-False-False-True-False]	47.1210μs	25.3908μs	39.3844 KOps/s	39.9624 KOps/s	$\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-False-False-False-True]	45.6500μs	21.8376μs	45.7926 KOps/s	45.0201 KOps/s	$\color{#35bf28}+1.72\%$
test_step_mdp_speed[True-False-False-False-False]	37.3210μs	14.7278μs	67.8987 KOps/s	66.9510 KOps/s	$\color{#35bf28}+1.42\%$
test_step_mdp_speed[False-True-True-True-True]	59.1510μs	35.8035μs	27.9302 KOps/s	27.4235 KOps/s	$\color{#35bf28}+1.85\%$
test_step_mdp_speed[False-True-True-True-False]	42.5400μs	23.5963μs	42.3795 KOps/s	42.9423 KOps/s	$\color{#d91a1a}-1.31\%$
test_step_mdp_speed[False-True-True-False-True]	74.2510μs	24.2359μs	41.2611 KOps/s	40.7524 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-True-True-False-False]	38.8500μs	15.1150μs	66.1596 KOps/s	66.8712 KOps/s	$\color{#d91a1a}-1.06\%$
test_step_mdp_speed[False-True-False-True-True]	74.3510μs	38.2228μs	26.1624 KOps/s	26.1882 KOps/s	$\color{#d91a1a}-0.10\%$
test_step_mdp_speed[False-True-False-True-False]	50.6000μs	25.3936μs	39.3800 KOps/s	39.2189 KOps/s	$\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-True-False-False-True]	47.6510μs	25.9746μs	38.4991 KOps/s	38.3098 KOps/s	$\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-True-False-False-False]	45.8110μs	16.8944μs	59.1912 KOps/s	59.8451 KOps/s	$\color{#d91a1a}-1.09\%$
test_step_mdp_speed[False-False-True-True-True]	70.7810μs	40.2739μs	24.8300 KOps/s	25.1562 KOps/s	$\color{#d91a1a}-1.30\%$
test_step_mdp_speed[False-False-True-True-False]	69.9510μs	27.5701μs	36.2712 KOps/s	36.2646 KOps/s	$\color{#35bf28}+0.02\%$
test_step_mdp_speed[False-False-True-False-True]	58.5510μs	26.3056μs	38.0147 KOps/s	38.3590 KOps/s	$\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-False-True-False-False]	52.8910μs	16.7962μs	59.5374 KOps/s	59.2075 KOps/s	$\color{#35bf28}+0.56\%$
test_step_mdp_speed[False-False-False-True-True]	64.9310μs	41.8487μs	23.8956 KOps/s	24.0746 KOps/s	$\color{#d91a1a}-0.74\%$
test_step_mdp_speed[False-False-False-True-False]	67.3210μs	29.4240μs	33.9858 KOps/s	34.0803 KOps/s	$\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-False-False-False-True]	49.5900μs	27.7060μs	36.0933 KOps/s	35.4905 KOps/s	$\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-False-False-False-False]	47.8500μs	18.7192μs	53.4210 KOps/s	53.6813 KOps/s	$\color{#d91a1a}-0.48\%$
test_values[generalized_advantage_estimate-True-True]	25.2437ms	24.6170ms	40.6223 Ops/s	39.9393 Ops/s	$\color{#35bf28}+1.71\%$
test_values[vec_generalized_advantage_estimate-True-True]	83.6708ms	3.2386ms	308.7708 Ops/s	304.1874 Ops/s	$\color{#35bf28}+1.51\%$
test_values[td0_return_estimate-False-False]	95.7110μs	60.8320μs	16.4387 KOps/s	16.1699 KOps/s	$\color{#35bf28}+1.66\%$
test_values[td1_return_estimate-False-False]	53.3824ms	52.9637ms	18.8809 Ops/s	18.5077 Ops/s	$\color{#35bf28}+2.02\%$
test_values[vec_td1_return_estimate-False-False]	2.0169ms	1.7614ms	567.7208 Ops/s	564.7171 Ops/s	$\color{#35bf28}+0.53\%$
test_values[td_lambda_return_estimate-True-False]	84.8667ms	84.5392ms	11.8288 Ops/s	11.6149 Ops/s	$\color{#35bf28}+1.84\%$
test_values[vec_td_lambda_return_estimate-True-False]	3.9532ms	1.7995ms	555.7242 Ops/s	552.3298 Ops/s	$\color{#35bf28}+0.61\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	23.7773ms	23.5794ms	42.4099 Ops/s	41.7600 Ops/s	$\color{#35bf28}+1.56\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.8663ms	0.7018ms	1.4248 KOps/s	1.4078 KOps/s	$\color{#35bf28}+1.21\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7079ms	0.6578ms	1.5203 KOps/s	1.5131 KOps/s	$\color{#35bf28}+0.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.4931ms	1.4547ms	687.4330 Ops/s	684.6932 Ops/s	$\color{#35bf28}+0.40\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9631ms	0.6750ms	1.4814 KOps/s	1.4788 KOps/s	$\color{#35bf28}+0.17\%$
test_dqn_speed	14.0707ms	7.6338ms	130.9957 Ops/s	135.0021 Ops/s	$\color{#d91a1a}-2.97\%$
test_ddpg_speed	15.3057ms	14.3815ms	69.5338 Ops/s	69.8568 Ops/s	$\color{#d91a1a}-0.46\%$
test_sac_speed	29.3296ms	28.6923ms	34.8526 Ops/s	35.4937 Ops/s	$\color{#d91a1a}-1.81\%$
test_redq_speed	0.1002s	12.5263ms	79.8323 Ops/s	86.4987 Ops/s	$\textbf{\color{#d91a1a}-7.71\%}$
test_redq_deprec_speed	23.7877ms	23.0447ms	43.3940 Ops/s	44.2214 Ops/s	$\color{#d91a1a}-1.87\%$
test_td3_speed	29.0921ms	19.1909ms	52.1079 Ops/s	52.7818 Ops/s	$\color{#d91a1a}-1.28\%$
test_cql_speed	76.2928ms	75.5481ms	13.2366 Ops/s	13.4367 Ops/s	$\color{#d91a1a}-1.49\%$
test_a2c_speed	26.4624ms	25.2954ms	39.5328 Ops/s	40.0603 Ops/s	$\color{#d91a1a}-1.32\%$
test_ppo_speed	26.8591ms	25.5651ms	39.1159 Ops/s	39.7212 Ops/s	$\color{#d91a1a}-1.52\%$
test_reinforce_speed	25.1621ms	24.4008ms	40.9822 Ops/s	41.8051 Ops/s	$\color{#d91a1a}-1.97\%$
test_iql_speed	51.4390ms	50.4796ms	19.8100 Ops/s	19.9538 Ops/s	$\color{#d91a1a}-0.72\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.6481ms	3.5534ms	281.4222 Ops/s	278.5635 Ops/s	$\color{#35bf28}+1.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6761ms	0.5526ms	1.8097 KOps/s	1.8106 KOps/s	$\color{#d91a1a}-0.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6566ms	0.5243ms	1.9074 KOps/s	1.9076 KOps/s	$\color{#d91a1a}-0.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.7450ms	3.5979ms	277.9387 Ops/s	278.6272 Ops/s	$\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6851ms	0.5490ms	1.8216 KOps/s	1.8248 KOps/s	$\color{#d91a1a}-0.17\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6674ms	0.5248ms	1.9056 KOps/s	1.9179 KOps/s	$\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.3699ms	3.2956ms	303.4368 Ops/s	304.9777 Ops/s	$\color{#d91a1a}-0.51\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8140ms	0.6733ms	1.4853 KOps/s	1.4716 KOps/s	$\color{#35bf28}+0.93\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7964ms	0.6457ms	1.5488 KOps/s	1.5355 KOps/s	$\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.6930ms	3.6097ms	277.0328 Ops/s	279.9590 Ops/s	$\color{#d91a1a}-1.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6919ms	0.5521ms	1.8113 KOps/s	1.8074 KOps/s	$\color{#35bf28}+0.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6918ms	0.5241ms	1.9081 KOps/s	1.8922 KOps/s	$\color{#35bf28}+0.84\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.7734ms	3.6208ms	276.1842 Ops/s	277.1045 Ops/s	$\color{#d91a1a}-0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6933ms	0.5433ms	1.8406 KOps/s	1.8231 KOps/s	$\color{#35bf28}+0.96\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6640ms	0.5207ms	1.9204 KOps/s	1.9117 KOps/s	$\color{#35bf28}+0.45\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.3974ms	3.3023ms	302.8221 Ops/s	300.7353 Ops/s	$\color{#35bf28}+0.69\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8334ms	0.6745ms	1.4827 KOps/s	1.4784 KOps/s	$\color{#35bf28}+0.29\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8066ms	0.6524ms	1.5329 KOps/s	1.5396 KOps/s	$\color{#d91a1a}-0.44\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1250s	10.1407ms	98.6125 Ops/s	100.6210 Ops/s	$\color{#d91a1a}-2.00\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	18.4636ms	16.2370ms	61.5877 Ops/s	60.5229 Ops/s	$\color{#35bf28}+1.76\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	6.3857ms	3.0604ms	326.7561 Ops/s	326.1193 Ops/s	$\color{#35bf28}+0.20\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1191s	12.2318ms	81.7541 Ops/s	98.3987 Ops/s	$\textbf{\color{#d91a1a}-16.92\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.3169ms	16.2960ms	61.3646 Ops/s	61.0336 Ops/s	$\color{#35bf28}+0.54\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.6799ms	3.0620ms	326.5856 Ops/s	325.7935 Ops/s	$\color{#35bf28}+0.24\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1197s	10.3121ms	96.9730 Ops/s	97.1191 Ops/s	$\color{#d91a1a}-0.15\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.9546ms	16.5341ms	60.4812 Ops/s	53.1097 Ops/s	$\textbf{\color{#35bf28}+13.88\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.7596ms	3.2993ms	303.0931 Ops/s	305.5221 Ops/s	$\color{#d91a1a}-0.80\%$