[BugFix] Fix update_priority generic signature for Samplers

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2252

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 9 Unrelated Failures

As of commit 0ca0122 with merge base 849b3de ():

NEW FAILURES - The following jobs have failed:

• Generate documentation / build-docs (3.9, 12.1) / linux-job (gh)
  No files were found with the provided path: /home/ec2-user/actions-runner/_work/_temp/artifacts/. No artifacts will be uploaded.
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  RuntimeError: Command docker exec -t d9795510a7a192e498b220b2b8273589eb1d5adc73cca9e29c354cd0b11e5550 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 917c1f1f234cea36ce2c2815e398c1da096fb90ffbf14947f9db1c307bff4638 /exec failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1237s	60.7417ms	16.4631 Ops/s	17.5562 Ops/s	$\textbf{\color{#d91a1a}-6.23\%}$
test_sync	43.1179ms	32.1368ms	31.1170 Ops/s	31.5534 Ops/s	$\color{#d91a1a}-1.38\%$
test_async	69.5155ms	30.3695ms	32.9277 Ops/s	35.3232 Ops/s	$\textbf{\color{#d91a1a}-6.78\%}$
test_simple	0.3887s	0.3852s	2.5961 Ops/s	2.4720 Ops/s	$\textbf{\color{#35bf28}+5.02\%}$
test_transformed	0.5601s	0.5544s	1.8038 Ops/s	1.8252 Ops/s	$\color{#d91a1a}-1.17\%$
test_serial	1.3523s	1.2898s	0.7753 Ops/s	0.7776 Ops/s	$\color{#d91a1a}-0.29\%$
test_parallel	1.1829s	1.0962s	0.9123 Ops/s	0.9091 Ops/s	$\color{#35bf28}+0.35\%$
test_step_mdp_speed[True-True-True-True-True]	0.1963ms	22.7875μs	43.8837 KOps/s	44.4096 KOps/s	$\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-True-True-True-False]	48.9010μs	13.3875μs	74.6965 KOps/s	75.8267 KOps/s	$\color{#d91a1a}-1.49\%$
test_step_mdp_speed[True-True-True-False-True]	50.4750μs	13.2780μs	75.3125 KOps/s	75.7945 KOps/s	$\color{#d91a1a}-0.64\%$
test_step_mdp_speed[True-True-True-False-False]	39.1930μs	7.7730μs	128.6505 KOps/s	128.3853 KOps/s	$\color{#35bf28}+0.21\%$
test_step_mdp_speed[True-True-False-True-True]	77.7260μs	24.1723μs	41.3696 KOps/s	41.6546 KOps/s	$\color{#d91a1a}-0.68\%$
test_step_mdp_speed[True-True-False-True-False]	64.0800μs	14.7164μs	67.9515 KOps/s	68.9412 KOps/s	$\color{#d91a1a}-1.44\%$
test_step_mdp_speed[True-True-False-False-True]	46.8880μs	14.5448μs	68.7533 KOps/s	69.6581 KOps/s	$\color{#d91a1a}-1.30\%$
test_step_mdp_speed[True-True-False-False-False]	31.2590μs	9.0475μs	110.5277 KOps/s	112.1135 KOps/s	$\color{#d91a1a}-1.41\%$
test_step_mdp_speed[True-False-True-True-True]	64.4710μs	25.5515μs	39.1367 KOps/s	39.7032 KOps/s	$\color{#d91a1a}-1.43\%$
test_step_mdp_speed[True-False-True-True-False]	63.0290μs	16.2484μs	61.5447 KOps/s	64.0328 KOps/s	$\color{#d91a1a}-3.89\%$
test_step_mdp_speed[True-False-True-False-True]	44.9840μs	14.5671μs	68.6479 KOps/s	70.0779 KOps/s	$\color{#d91a1a}-2.04\%$
test_step_mdp_speed[True-False-True-False-False]	34.3040μs	9.1551μs	109.2285 KOps/s	111.6742 KOps/s	$\color{#d91a1a}-2.19\%$
test_step_mdp_speed[True-False-False-True-True]	62.8770μs	26.8537μs	37.2388 KOps/s	38.2449 KOps/s	$\color{#d91a1a}-2.63\%$
test_step_mdp_speed[True-False-False-True-False]	44.6030μs	17.3987μs	57.4756 KOps/s	59.1749 KOps/s	$\color{#d91a1a}-2.87\%$
test_step_mdp_speed[True-False-False-False-True]	58.2090μs	15.7158μs	63.6302 KOps/s	64.4396 KOps/s	$\color{#d91a1a}-1.26\%$
test_step_mdp_speed[True-False-False-False-False]	35.0360μs	10.3966μs	96.1852 KOps/s	98.7795 KOps/s	$\color{#d91a1a}-2.63\%$
test_step_mdp_speed[False-True-True-True-True]	77.9060μs	25.6611μs	38.9695 KOps/s	39.4475 KOps/s	$\color{#d91a1a}-1.21\%$
test_step_mdp_speed[False-True-True-True-False]	44.1830μs	16.3005μs	61.3477 KOps/s	63.7262 KOps/s	$\color{#d91a1a}-3.73\%$
test_step_mdp_speed[False-True-True-False-True]	44.2730μs	16.9366μs	59.0437 KOps/s	60.5217 KOps/s	$\color{#d91a1a}-2.44\%$
test_step_mdp_speed[False-True-True-False-False]	37.9210μs	10.4287μs	95.8893 KOps/s	98.8216 KOps/s	$\color{#d91a1a}-2.97\%$
test_step_mdp_speed[False-True-False-True-True]	60.7440μs	26.9793μs	37.0655 KOps/s	38.1183 KOps/s	$\color{#d91a1a}-2.76\%$
test_step_mdp_speed[False-True-False-True-False]	65.3720μs	17.2995μs	57.8051 KOps/s	59.1901 KOps/s	$\color{#d91a1a}-2.34\%$
test_step_mdp_speed[False-True-False-False-True]	47.8200μs	18.0614μs	55.3667 KOps/s	55.9997 KOps/s	$\color{#d91a1a}-1.13\%$
test_step_mdp_speed[False-True-False-False-False]	44.6430μs	11.5433μs	86.6305 KOps/s	88.2725 KOps/s	$\color{#d91a1a}-1.86\%$
test_step_mdp_speed[False-False-True-True-True]	63.5490μs	28.1245μs	35.5562 KOps/s	36.2472 KOps/s	$\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-False-True-True-False]	54.1710μs	18.8709μs	52.9917 KOps/s	55.0458 KOps/s	$\color{#d91a1a}-3.73\%$
test_step_mdp_speed[False-False-True-False-True]	54.8420μs	18.1446μs	55.1129 KOps/s	56.2997 KOps/s	$\color{#d91a1a}-2.11\%$
test_step_mdp_speed[False-False-True-False-False]	55.3240μs	11.6603μs	85.7610 KOps/s	88.2497 KOps/s	$\color{#d91a1a}-2.82\%$
test_step_mdp_speed[False-False-False-True-True]	64.0500μs	29.4518μs	33.9538 KOps/s	33.9658 KOps/s	$\color{#d91a1a}-0.04\%$
test_step_mdp_speed[False-False-False-True-False]	54.1210μs	19.9559μs	50.1104 KOps/s	51.8097 KOps/s	$\color{#d91a1a}-3.28\%$
test_step_mdp_speed[False-False-False-False-True]	44.9640μs	19.2209μs	52.0267 KOps/s	53.3934 KOps/s	$\color{#d91a1a}-2.56\%$
test_step_mdp_speed[False-False-False-False-False]	51.3260μs	12.7141μs	78.6527 KOps/s	81.1740 KOps/s	$\color{#d91a1a}-3.11\%$
test_values[generalized_advantage_estimate-True-True]	9.9256ms	9.6560ms	103.5628 Ops/s	104.7714 Ops/s	$\color{#d91a1a}-1.15\%$
test_values[vec_generalized_advantage_estimate-True-True]	37.9932ms	35.7103ms	28.0031 Ops/s	28.2388 Ops/s	$\color{#d91a1a}-0.83\%$
test_values[td0_return_estimate-False-False]	0.2228ms	0.1726ms	5.7932 KOps/s	5.5828 KOps/s	$\color{#35bf28}+3.77\%$
test_values[td1_return_estimate-False-False]	24.1514ms	23.7809ms	42.0506 Ops/s	42.4260 Ops/s	$\color{#d91a1a}-0.88\%$
test_values[vec_td1_return_estimate-False-False]	37.7799ms	35.5888ms	28.0988 Ops/s	28.1098 Ops/s	$\color{#d91a1a}-0.04\%$
test_values[td_lambda_return_estimate-True-False]	37.0226ms	33.9162ms	29.4845 Ops/s	29.3325 Ops/s	$\color{#35bf28}+0.52\%$
test_values[vec_td_lambda_return_estimate-True-False]	38.2478ms	35.5138ms	28.1580 Ops/s	28.1230 Ops/s	$\color{#35bf28}+0.12\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	11.6916ms	8.2561ms	121.1232 Ops/s	120.6874 Ops/s	$\color{#35bf28}+0.36\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.3315ms	1.9985ms	500.3700 Ops/s	530.6972 Ops/s	$\textbf{\color{#d91a1a}-5.71\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4418ms	0.3487ms	2.8678 KOps/s	2.7783 KOps/s	$\color{#35bf28}+3.22\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	46.7812ms	44.4707ms	22.4867 Ops/s	23.0937 Ops/s	$\color{#d91a1a}-2.63\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.6539ms	3.0655ms	326.2155 Ops/s	326.9458 Ops/s	$\color{#d91a1a}-0.22\%$
test_dqn_speed	6.8532ms	1.3855ms	721.7412 Ops/s	728.5066 Ops/s	$\color{#d91a1a}-0.93\%$
test_ddpg_speed	3.5778ms	2.8729ms	348.0774 Ops/s	349.9316 Ops/s	$\color{#d91a1a}-0.53\%$
test_sac_speed	9.5391ms	8.7134ms	114.7653 Ops/s	114.6423 Ops/s	$\color{#35bf28}+0.11\%$
test_redq_speed	15.8925ms	13.9927ms	71.4661 Ops/s	64.8659 Ops/s	$\textbf{\color{#35bf28}+10.18\%}$
test_redq_deprec_speed	15.2637ms	14.3660ms	69.6088 Ops/s	68.7091 Ops/s	$\color{#35bf28}+1.31\%$
test_td3_speed	18.7455ms	8.7602ms	114.1529 Ops/s	115.9326 Ops/s	$\color{#d91a1a}-1.54\%$
test_cql_speed	38.9351ms	37.4777ms	26.6826 Ops/s	26.6232 Ops/s	$\color{#35bf28}+0.22\%$
test_a2c_speed	8.7769ms	7.8123ms	128.0040 Ops/s	126.8770 Ops/s	$\color{#35bf28}+0.89\%$
test_ppo_speed	9.4681ms	8.2084ms	121.8262 Ops/s	120.9215 Ops/s	$\color{#35bf28}+0.75\%$
test_reinforce_speed	7.9882ms	7.0355ms	142.1357 Ops/s	141.0591 Ops/s	$\color{#35bf28}+0.76\%$
test_iql_speed	34.3990ms	33.6024ms	29.7598 Ops/s	29.6385 Ops/s	$\color{#35bf28}+0.41\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.2651ms	3.8053ms	262.7938 Ops/s	263.0808 Ops/s	$\color{#d91a1a}-0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7961ms	0.5189ms	1.9273 KOps/s	1.8932 KOps/s	$\color{#35bf28}+1.80\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7230ms	0.4941ms	2.0240 KOps/s	1.9929 KOps/s	$\color{#35bf28}+1.56\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.1416ms	3.7949ms	263.5101 Ops/s	266.9646 Ops/s	$\color{#d91a1a}-1.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8959ms	0.5116ms	1.9545 KOps/s	1.9430 KOps/s	$\color{#35bf28}+0.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7592ms	0.4884ms	2.0474 KOps/s	2.0325 KOps/s	$\color{#35bf28}+0.73\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.4616ms	1.7513ms	571.0065 Ops/s	570.3334 Ops/s	$\color{#35bf28}+0.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.3755ms	1.6614ms	601.8905 Ops/s	598.9783 Ops/s	$\color{#35bf28}+0.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.5156ms	3.8590ms	259.1337 Ops/s	257.3135 Ops/s	$\color{#35bf28}+0.71\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.3858ms	0.6518ms	1.5342 KOps/s	1.5417 KOps/s	$\color{#d91a1a}-0.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.1283s	0.7367ms	1.3574 KOps/s	1.5908 KOps/s	$\textbf{\color{#d91a1a}-14.67\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.1340ms	3.8171ms	261.9776 Ops/s	269.6157 Ops/s	$\color{#d91a1a}-2.83\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6926ms	0.5204ms	1.9217 KOps/s	1.9263 KOps/s	$\color{#d91a1a}-0.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	4.0381ms	0.4980ms	2.0080 KOps/s	2.0035 KOps/s	$\color{#35bf28}+0.22\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.7223ms	3.8447ms	260.0977 Ops/s	267.7208 Ops/s	$\color{#d91a1a}-2.85\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1208s	0.6790ms	1.4728 KOps/s	1.9569 KOps/s	$\textbf{\color{#d91a1a}-24.74\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5907ms	0.4921ms	2.0322 KOps/s	1.9832 KOps/s	$\color{#35bf28}+2.47\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.6020ms	3.9830ms	251.0702 Ops/s	258.9336 Ops/s	$\color{#d91a1a}-3.04\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.7950ms	0.6536ms	1.5300 KOps/s	1.5378 KOps/s	$\color{#d91a1a}-0.51\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.1151ms	0.6395ms	1.5637 KOps/s	1.5813 KOps/s	$\color{#d91a1a}-1.11\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1233s	6.1653ms	162.1991 Ops/s	155.0073 Ops/s	$\color{#35bf28}+4.64\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1330s	15.0320ms	66.5246 Ops/s	65.6204 Ops/s	$\color{#35bf28}+1.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.7675ms	1.0878ms	919.3262 Ops/s	938.3989 Ops/s	$\color{#d91a1a}-2.03\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1233s	6.2431ms	160.1766 Ops/s	161.8178 Ops/s	$\color{#d91a1a}-1.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	15.6953ms	12.5938ms	79.4042 Ops/s	77.6754 Ops/s	$\color{#35bf28}+2.23\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	1.9128ms	1.0991ms	909.7945 Ops/s	947.9545 Ops/s	$\color{#d91a1a}-4.03\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1255s	6.4833ms	154.2414 Ops/s	157.8626 Ops/s	$\color{#d91a1a}-2.29\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	15.9407ms	12.8236ms	77.9815 Ops/s	76.8949 Ops/s	$\color{#35bf28}+1.41\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.9169ms	1.2436ms	804.1134 Ops/s	817.7342 Ops/s	$\color{#d91a1a}-1.67\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1760s	0.1239s	8.0681 Ops/s	8.5983 Ops/s	$\textbf{\color{#d91a1a}-6.17\%}$
test_sync	0.1061s	0.1046s	9.5568 Ops/s	9.5807 Ops/s	$\color{#d91a1a}-0.25\%$
test_async	0.1936s	93.4000ms	10.7066 Ops/s	10.4031 Ops/s	$\color{#35bf28}+2.92\%$
test_single_pixels	0.1293s	0.1282s	7.7987 Ops/s	7.9164 Ops/s	$\color{#d91a1a}-1.49\%$
test_sync_pixels	85.2529ms	82.3451ms	12.1440 Ops/s	12.1084 Ops/s	$\color{#35bf28}+0.29\%$
test_async_pixels	0.1582s	68.2808ms	14.6454 Ops/s	14.5780 Ops/s	$\color{#35bf28}+0.46\%$
test_simple	0.8822s	0.8227s	1.2155 Ops/s	1.2521 Ops/s	$\color{#d91a1a}-2.92\%$
test_transformed	1.1375s	1.0866s	0.9203 Ops/s	0.9381 Ops/s	$\color{#d91a1a}-1.90\%$
test_serial	2.5579s	2.5041s	0.3993 Ops/s	0.4079 Ops/s	$\color{#d91a1a}-2.11\%$
test_parallel	2.4497s	2.3616s	0.4234 Ops/s	0.4227 Ops/s	$\color{#35bf28}+0.17\%$
test_step_mdp_speed[True-True-True-True-True]	0.1209ms	33.5255μs	29.8281 KOps/s	28.9080 KOps/s	$\color{#35bf28}+3.18\%$
test_step_mdp_speed[True-True-True-True-False]	45.5900μs	19.5744μs	51.0873 KOps/s	50.2642 KOps/s	$\color{#35bf28}+1.64\%$
test_step_mdp_speed[True-True-True-False-True]	50.1510μs	19.3038μs	51.8033 KOps/s	51.2183 KOps/s	$\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-True-True-False-False]	26.7800μs	11.3758μs	87.9059 KOps/s	88.4820 KOps/s	$\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-True-False-True-True]	56.7710μs	36.1688μs	27.6481 KOps/s	27.2933 KOps/s	$\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-True-False-True-False]	39.2520μs	21.4116μs	46.7037 KOps/s	45.9634 KOps/s	$\color{#35bf28}+1.61\%$
test_step_mdp_speed[True-True-False-False-True]	45.9010μs	21.0542μs	47.4964 KOps/s	46.3046 KOps/s	$\color{#35bf28}+2.57\%$
test_step_mdp_speed[True-True-False-False-False]	39.4020μs	13.1791μs	75.8779 KOps/s	74.9713 KOps/s	$\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-False-True-True-True]	58.6810μs	37.3171μs	26.7973 KOps/s	26.5487 KOps/s	$\color{#35bf28}+0.94\%$
test_step_mdp_speed[True-False-True-True-False]	45.2510μs	23.4696μs	42.6083 KOps/s	42.5279 KOps/s	$\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-False-True-False-True]	51.6400μs	21.1644μs	47.2492 KOps/s	46.7214 KOps/s	$\color{#35bf28}+1.13\%$
test_step_mdp_speed[True-False-True-False-False]	29.0300μs	13.1454μs	76.0724 KOps/s	74.7602 KOps/s	$\color{#35bf28}+1.76\%$
test_step_mdp_speed[True-False-False-True-True]	67.4510μs	39.4724μs	25.3341 KOps/s	25.2521 KOps/s	$\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-False-False-True-False]	0.1134ms	25.4113μs	39.3526 KOps/s	39.1523 KOps/s	$\color{#35bf28}+0.51\%$
test_step_mdp_speed[True-False-False-False-True]	40.8310μs	22.9790μs	43.5180 KOps/s	43.1192 KOps/s	$\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-False-False-False-False]	59.9610μs	15.0682μs	66.3651 KOps/s	65.7584 KOps/s	$\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-True-True-True-True]	56.6210μs	37.4635μs	26.6926 KOps/s	26.0570 KOps/s	$\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-True-True-True-False]	45.4710μs	23.4529μs	42.6386 KOps/s	42.1085 KOps/s	$\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-True-True-False-True]	52.5700μs	25.0146μs	39.9767 KOps/s	38.7374 KOps/s	$\color{#35bf28}+3.20\%$
test_step_mdp_speed[False-True-True-False-False]	47.5110μs	14.8433μs	67.3702 KOps/s	66.8275 KOps/s	$\color{#35bf28}+0.81\%$
test_step_mdp_speed[False-True-False-True-True]	61.9620μs	38.8495μs	25.7404 KOps/s	25.0670 KOps/s	$\color{#35bf28}+2.69\%$
test_step_mdp_speed[False-True-False-True-False]	46.3010μs	25.0123μs	39.9804 KOps/s	39.3198 KOps/s	$\color{#35bf28}+1.68\%$
test_step_mdp_speed[False-True-False-False-True]	62.7710μs	26.7536μs	37.3782 KOps/s	36.5556 KOps/s	$\color{#35bf28}+2.25\%$
test_step_mdp_speed[False-True-False-False-False]	44.6910μs	16.6491μs	60.0632 KOps/s	59.4113 KOps/s	$\color{#35bf28}+1.10\%$
test_step_mdp_speed[False-False-True-True-True]	59.9510μs	41.2694μs	24.2310 KOps/s	24.0511 KOps/s	$\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-False-True-True-False]	44.9310μs	27.2168μs	36.7420 KOps/s	36.0972 KOps/s	$\color{#35bf28}+1.79\%$
test_step_mdp_speed[False-False-True-False-True]	45.7810μs	26.7335μs	37.4063 KOps/s	36.6216 KOps/s	$\color{#35bf28}+2.14\%$
test_step_mdp_speed[False-False-True-False-False]	31.5410μs	16.7205μs	59.8068 KOps/s	59.0495 KOps/s	$\color{#35bf28}+1.28\%$
test_step_mdp_speed[False-False-False-True-True]	64.2910μs	43.3861μs	23.0488 KOps/s	22.8520 KOps/s	$\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-False-False-True-False]	82.0920μs	29.0054μs	34.4763 KOps/s	33.8860 KOps/s	$\color{#35bf28}+1.74\%$
test_step_mdp_speed[False-False-False-False-True]	57.4410μs	28.2206μs	35.4351 KOps/s	34.9102 KOps/s	$\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-False-False-False-False]	33.1510μs	18.4412μs	54.2264 KOps/s	53.1269 KOps/s	$\color{#35bf28}+2.07\%$
test_values[generalized_advantage_estimate-True-True]	25.3552ms	24.5573ms	40.7211 Ops/s	42.7715 Ops/s	$\color{#d91a1a}-4.79\%$
test_values[vec_generalized_advantage_estimate-True-True]	87.8027ms	2.6516ms	377.1322 Ops/s	372.4946 Ops/s	$\color{#35bf28}+1.25\%$
test_values[td0_return_estimate-False-False]	91.9320μs	65.5478μs	15.2561 KOps/s	15.1766 KOps/s	$\color{#35bf28}+0.52\%$
test_values[td1_return_estimate-False-False]	55.8546ms	55.0508ms	18.1650 Ops/s	18.7195 Ops/s	$\color{#d91a1a}-2.96\%$
test_values[vec_td1_return_estimate-False-False]	1.4096ms	1.0788ms	926.9743 Ops/s	932.3241 Ops/s	$\color{#d91a1a}-0.57\%$
test_values[td_lambda_return_estimate-True-False]	87.7969ms	87.1503ms	11.4744 Ops/s	11.4773 Ops/s	$\color{#d91a1a}-0.02\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.4029ms	1.0750ms	930.2179 Ops/s	929.0995 Ops/s	$\color{#35bf28}+0.12\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	24.8464ms	24.6100ms	40.6339 Ops/s	42.3989 Ops/s	$\color{#d91a1a}-4.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9270ms	0.7165ms	1.3958 KOps/s	1.3960 KOps/s	$\color{#d91a1a}-0.02\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7458ms	0.6683ms	1.4963 KOps/s	1.4868 KOps/s	$\color{#35bf28}+0.64\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5177ms	1.4685ms	680.9701 Ops/s	684.3351 Ops/s	$\color{#d91a1a}-0.49\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.7762ms	0.6837ms	1.4627 KOps/s	1.4820 KOps/s	$\color{#d91a1a}-1.30\%$
test_dqn_speed	1.5919ms	1.4442ms	692.4305 Ops/s	707.1376 Ops/s	$\color{#d91a1a}-2.08\%$
test_ddpg_speed	3.2898ms	2.9490ms	339.0992 Ops/s	346.9870 Ops/s	$\color{#d91a1a}-2.27\%$
test_sac_speed	8.5173ms	8.3084ms	120.3602 Ops/s	122.0025 Ops/s	$\color{#d91a1a}-1.35\%$
test_redq_speed	0.1028s	11.5524ms	86.5622 Ops/s	94.2244 Ops/s	$\textbf{\color{#d91a1a}-8.13\%}$
test_redq_deprec_speed	11.9826ms	11.5122ms	86.8644 Ops/s	80.8728 Ops/s	$\textbf{\color{#35bf28}+7.41\%}$
test_td3_speed	8.3079ms	8.2490ms	121.2263 Ops/s	123.0493 Ops/s	$\color{#d91a1a}-1.48\%$
test_cql_speed	26.1326ms	25.4614ms	39.2751 Ops/s	39.6523 Ops/s	$\color{#d91a1a}-0.95\%$
test_a2c_speed	6.8791ms	5.7367ms	174.3147 Ops/s	177.5059 Ops/s	$\color{#d91a1a}-1.80\%$
test_ppo_speed	6.1476ms	5.9887ms	166.9805 Ops/s	167.0816 Ops/s	$\color{#d91a1a}-0.06\%$
test_reinforce_speed	4.9342ms	4.7012ms	212.7107 Ops/s	215.8291 Ops/s	$\color{#d91a1a}-1.44\%$
test_iql_speed	20.0371ms	19.4478ms	51.4197 Ops/s	51.7369 Ops/s	$\color{#d91a1a}-0.61\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.7471ms	4.6518ms	214.9714 Ops/s	215.9165 Ops/s	$\color{#d91a1a}-0.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.1068s	0.6885ms	1.4524 KOps/s	1.6870 KOps/s	$\textbf{\color{#d91a1a}-13.90\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7624ms	0.5704ms	1.7531 KOps/s	1.7613 KOps/s	$\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.8012ms	4.6426ms	215.3962 Ops/s	216.6801 Ops/s	$\color{#d91a1a}-0.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.4541ms	0.5882ms	1.7002 KOps/s	1.7142 KOps/s	$\color{#d91a1a}-0.82\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7571ms	0.5662ms	1.7660 KOps/s	1.7772 KOps/s	$\color{#d91a1a}-0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	4.3768ms	2.1410ms	467.0802 Ops/s	492.8105 Ops/s	$\textbf{\color{#d91a1a}-5.22\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.3428ms	1.9951ms	501.2384 Ops/s	514.9460 Ops/s	$\color{#d91a1a}-2.66\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.9149ms	4.8077ms	207.9991 Ops/s	209.6256 Ops/s	$\color{#d91a1a}-0.78\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9042ms	0.7402ms	1.3509 KOps/s	1.2869 KOps/s	$\color{#35bf28}+4.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.5597ms	0.7220ms	1.3850 KOps/s	1.3430 KOps/s	$\color{#35bf28}+3.13\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.8456ms	4.6520ms	214.9635 Ops/s	215.2180 Ops/s	$\color{#d91a1a}-0.12\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7186ms	0.5974ms	1.6741 KOps/s	1.6862 KOps/s	$\color{#d91a1a}-0.72\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	4.3159ms	0.5788ms	1.7277 KOps/s	1.7454 KOps/s	$\color{#d91a1a}-1.01\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.8592ms	4.6630ms	214.4526 Ops/s	216.9658 Ops/s	$\color{#d91a1a}-1.16\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1359s	0.7890ms	1.2674 KOps/s	1.6988 KOps/s	$\textbf{\color{#d91a1a}-25.40\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7287ms	0.5719ms	1.7485 KOps/s	1.7468 KOps/s	$\color{#35bf28}+0.10\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.9032ms	4.8048ms	208.1254 Ops/s	209.4785 Ops/s	$\color{#d91a1a}-0.65\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8954ms	0.7455ms	1.3413 KOps/s	1.3532 KOps/s	$\color{#d91a1a}-0.88\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8744ms	0.7253ms	1.3787 KOps/s	1.3828 KOps/s	$\color{#d91a1a}-0.29\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1266s	7.4211ms	134.7513 Ops/s	134.6929 Ops/s	$\color{#35bf28}+0.04\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	21.1863ms	15.8949ms	62.9134 Ops/s	64.6058 Ops/s	$\color{#d91a1a}-2.62\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.3195ms	1.2913ms	774.3959 Ops/s	777.6205 Ops/s	$\color{#d91a1a}-0.41\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1267s	9.7692ms	102.3624 Ops/s	101.4445 Ops/s	$\color{#35bf28}+0.90\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.0328ms	15.9430ms	62.7234 Ops/s	63.9252 Ops/s	$\color{#d91a1a}-1.88\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.3361ms	1.2948ms	772.3251 Ops/s	764.2500 Ops/s	$\color{#35bf28}+1.06\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1250s	7.4804ms	133.6835 Ops/s	132.8149 Ops/s	$\color{#35bf28}+0.65\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.3521ms	15.8826ms	62.9620 Ops/s	63.9015 Ops/s	$\color{#d91a1a}-1.47\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.4519ms	1.5719ms	636.1929 Ops/s	637.2823 Ops/s	$\color{#d91a1a}-0.17\%$