[Feature] Trainer Infrastructure - Timing and Utilities

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] Documentation #3192
• [Feature] Add timing options #3191
• [Feature] PPO Trainer Updates #3190
• [Feature] Trainer Algorithms - Configuration System #3189
• -> [Feature] Trainer Infrastructure - Timing and Utilities #3188

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3188

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 654200e with merge base 01d2801 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 139.
• Generate documentation / build-docs (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t a6377d2b764b6ea7fa6613a92395688df854cf6838bb54a9384789cfb149aa49 /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 0313155bd704dc1de2fee5ce382466ce8beeeffe789bb0a91cf37719ec5c44ca /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 9b959b8a54e07c760127db5dc12a333e9c0ebc989581494a154bdec1c83862ad /exec failed with exit code 1
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 878ee8a590abab97dc0291a8a4a852eceb5d776e9442e0bb1aadff816b32cbde /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Linux / tests-gpu (3.11, 12.8) / linux-job (gh) (trunk failure)
  test/test_collector.py::TestPolicyFactory::test_weight_update
• Unit-tests on Linux / tests-olddeps (3.9, 11.6) / linux-job (gh) (trunk failure)
  test/test_collector.py::TestPolicyFactory::test_weight_update
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh) (trunk failure)
  test/test_collector.py::TestPolicyFactory::test_weight_update

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	82.7783μs	81.4433μs	12.2785 KOps/s	12.2185 KOps/s	$\color{#35bf28}+0.49\%$
test_tensor_to_bytestream_speed[torch.save]	0.1416ms	0.1409ms	7.0987 KOps/s	7.0113 KOps/s	$\color{#35bf28}+1.25\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1272s	0.1270s	7.8746 Ops/s	7.9665 Ops/s	$\color{#d91a1a}-1.15\%$
test_tensor_to_bytestream_speed[numpy]	2.9431μs	2.9364μs	340.5495 KOps/s	358.3561 KOps/s	$\color{#d91a1a}-4.97\%$
test_tensor_to_bytestream_speed[safetensors]	42.3199μs	41.6796μs	23.9926 KOps/s	23.7514 KOps/s	$\color{#35bf28}+1.02\%$
test_simple	0.5477s	0.5466s	1.8297 Ops/s	1.7351 Ops/s	$\textbf{\color{#35bf28}+5.45\%}$
test_transformed	1.2257s	1.1269s	0.8874 Ops/s	0.8828 Ops/s	$\color{#35bf28}+0.52\%$
test_serial	1.6594s	1.6551s	0.6042 Ops/s	0.5906 Ops/s	$\color{#35bf28}+2.30\%$
test_parallel	1.1752s	1.1103s	0.9007 Ops/s	0.9260 Ops/s	$\color{#d91a1a}-2.74\%$
test_step_mdp_speed[True-True-True-True-True]	0.1454ms	43.0648μs	23.2208 KOps/s	22.7363 KOps/s	$\color{#35bf28}+2.13\%$
test_step_mdp_speed[True-True-True-True-False]	56.4510μs	24.5326μs	40.7620 KOps/s	39.6676 KOps/s	$\color{#35bf28}+2.76\%$
test_step_mdp_speed[True-True-True-False-True]	79.6610μs	24.4797μs	40.8502 KOps/s	40.2054 KOps/s	$\color{#35bf28}+1.60\%$
test_step_mdp_speed[True-True-True-False-False]	39.6000μs	13.6057μs	73.4985 KOps/s	72.2504 KOps/s	$\color{#35bf28}+1.73\%$
test_step_mdp_speed[True-True-False-True-True]	78.5310μs	46.9378μs	21.3048 KOps/s	21.0281 KOps/s	$\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-True-False-True-False]	89.6710μs	27.6312μs	36.1910 KOps/s	36.1607 KOps/s	$\color{#35bf28}+0.08\%$
test_step_mdp_speed[True-True-False-False-True]	59.9210μs	27.3237μs	36.5982 KOps/s	35.9231 KOps/s	$\color{#35bf28}+1.88\%$
test_step_mdp_speed[True-True-False-False-False]	48.9010μs	16.4395μs	60.8290 KOps/s	58.6888 KOps/s	$\color{#35bf28}+3.65\%$
test_step_mdp_speed[True-False-True-True-True]	79.8810μs	49.6692μs	20.1332 KOps/s	19.5466 KOps/s	$\color{#35bf28}+3.00\%$
test_step_mdp_speed[True-False-True-True-False]	55.3700μs	30.2199μs	33.0908 KOps/s	32.3928 KOps/s	$\color{#35bf28}+2.15\%$
test_step_mdp_speed[True-False-True-False-True]	67.8510μs	27.6594μs	36.1541 KOps/s	36.3065 KOps/s	$\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-False-True-False-False]	48.7610μs	16.3617μs	61.1182 KOps/s	59.5854 KOps/s	$\color{#35bf28}+2.57\%$
test_step_mdp_speed[True-False-False-True-True]	96.9910μs	51.9042μs	19.2663 KOps/s	18.8728 KOps/s	$\color{#35bf28}+2.09\%$
test_step_mdp_speed[True-False-False-True-False]	61.5010μs	32.6674μs	30.6115 KOps/s	30.3401 KOps/s	$\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-False-False-False-True]	57.6010μs	29.7667μs	33.5946 KOps/s	32.6412 KOps/s	$\color{#35bf28}+2.92\%$
test_step_mdp_speed[True-False-False-False-False]	51.0410μs	18.9599μs	52.7428 KOps/s	51.9722 KOps/s	$\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-True-True-True-True]	79.7510μs	50.2301μs	19.9084 KOps/s	19.8425 KOps/s	$\color{#35bf28}+0.33\%$
test_step_mdp_speed[False-True-True-True-False]	61.7310μs	30.0715μs	33.2541 KOps/s	32.5768 KOps/s	$\color{#35bf28}+2.08\%$
test_step_mdp_speed[False-True-True-False-True]	2.6632ms	31.0002μs	32.2579 KOps/s	31.0737 KOps/s	$\color{#35bf28}+3.81\%$
test_step_mdp_speed[False-True-True-False-False]	46.3410μs	17.8710μs	55.9564 KOps/s	54.0625 KOps/s	$\color{#35bf28}+3.50\%$
test_step_mdp_speed[False-True-False-True-True]	90.4110μs	51.8873μs	19.2726 KOps/s	18.7833 KOps/s	$\color{#35bf28}+2.60\%$
test_step_mdp_speed[False-True-False-True-False]	64.7310μs	32.3583μs	30.9039 KOps/s	30.3646 KOps/s	$\color{#35bf28}+1.78\%$
test_step_mdp_speed[False-True-False-False-True]	62.6910μs	33.1543μs	30.1620 KOps/s	29.4658 KOps/s	$\color{#35bf28}+2.36\%$
test_step_mdp_speed[False-True-False-False-False]	64.1010μs	20.6067μs	48.5280 KOps/s	47.7952 KOps/s	$\color{#35bf28}+1.53\%$
test_step_mdp_speed[False-False-True-True-True]	93.4020μs	54.9450μs	18.2000 KOps/s	17.9422 KOps/s	$\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-False-True-True-False]	0.1364ms	34.6882μs	28.8283 KOps/s	27.8905 KOps/s	$\color{#35bf28}+3.36\%$
test_step_mdp_speed[False-False-True-False-True]	66.4010μs	33.4719μs	29.8758 KOps/s	29.1863 KOps/s	$\color{#35bf28}+2.36\%$
test_step_mdp_speed[False-False-True-False-False]	48.9000μs	20.5267μs	48.7170 KOps/s	48.1443 KOps/s	$\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-False-False-True-True]	86.5220μs	56.0298μs	17.8477 KOps/s	17.4609 KOps/s	$\color{#35bf28}+2.22\%$
test_step_mdp_speed[False-False-False-True-False]	65.8310μs	37.9925μs	26.3210 KOps/s	26.0800 KOps/s	$\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-False-False-False-True]	58.0010μs	35.5266μs	28.1479 KOps/s	27.5852 KOps/s	$\color{#35bf28}+2.04\%$
test_step_mdp_speed[False-False-False-False-False]	70.4110μs	23.0155μs	43.4489 KOps/s	43.1842 KOps/s	$\color{#35bf28}+0.61\%$
test_values[generalized_advantage_estimate-True-True]	10.4377ms	10.0986ms	99.0240 Ops/s	99.8359 Ops/s	$\color{#d91a1a}-0.81\%$
test_values[vec_generalized_advantage_estimate-True-True]	16.4606ms	11.3388ms	88.1928 Ops/s	55.6419 Ops/s	$\textbf{\color{#35bf28}+58.50\%}$
test_values[td0_return_estimate-False-False]	0.2427ms	0.1336ms	7.4840 KOps/s	7.8100 KOps/s	$\color{#d91a1a}-4.17\%$
test_values[td1_return_estimate-False-False]	27.6687ms	27.1723ms	36.8022 Ops/s	37.3446 Ops/s	$\color{#d91a1a}-1.45\%$
test_values[vec_td1_return_estimate-False-False]	12.1178ms	11.3714ms	87.9399 Ops/s	56.6090 Ops/s	$\textbf{\color{#35bf28}+55.35\%}$
test_values[td_lambda_return_estimate-True-False]	41.4571ms	40.6886ms	24.5769 Ops/s	24.7453 Ops/s	$\color{#d91a1a}-0.68\%$
test_values[vec_td_lambda_return_estimate-True-False]	12.5746ms	11.5001ms	86.9560 Ops/s	56.7270 Ops/s	$\textbf{\color{#35bf28}+53.29\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.8597ms	8.6478ms	115.6367 Ops/s	116.4468 Ops/s	$\color{#d91a1a}-0.70\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.8929ms	1.5139ms	660.5261 Ops/s	643.1033 Ops/s	$\color{#35bf28}+2.71\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4830ms	0.4176ms	2.3944 KOps/s	2.3941 KOps/s	$+0.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	30.5664ms	28.9748ms	34.5128 Ops/s	28.7454 Ops/s	$\textbf{\color{#35bf28}+20.06\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	2.1235ms	1.7313ms	577.5862 Ops/s	574.0877 Ops/s	$\color{#35bf28}+0.61\%$
test_dqn_speed[False-None]	1.5183ms	1.4235ms	702.5153 Ops/s	708.8555 Ops/s	$\color{#d91a1a}-0.89\%$
test_dqn_speed[False-backward]	2.0174ms	1.9342ms	516.9975 Ops/s	517.6991 Ops/s	$\color{#d91a1a}-0.14\%$
test_dqn_speed[True-None]	1.1943ms	0.5391ms	1.8548 KOps/s	1.8871 KOps/s	$\color{#d91a1a}-1.71\%$
test_dqn_speed[True-backward]	1.0321ms	0.9827ms	1.0176 KOps/s	996.4293 Ops/s	$\color{#35bf28}+2.12\%$
test_dqn_speed[reduce-overhead-None]	0.9302ms	0.5209ms	1.9198 KOps/s	1.9062 KOps/s	$\color{#35bf28}+0.71\%$
test_dqn_speed[reduce-overhead-backward]	1.0661ms	0.9767ms	1.0239 KOps/s	1.0169 KOps/s	$\color{#35bf28}+0.69\%$
test_ddpg_speed[False-None]	3.2954ms	2.9197ms	342.5027 Ops/s	344.9394 Ops/s	$\color{#d91a1a}-0.71\%$
test_ddpg_speed[False-backward]	4.9773ms	4.1482ms	241.0703 Ops/s	241.4894 Ops/s	$\color{#d91a1a}-0.17\%$
test_ddpg_speed[True-None]	5.9005ms	1.4026ms	712.9570 Ops/s	664.3493 Ops/s	$\textbf{\color{#35bf28}+7.32\%}$
test_ddpg_speed[True-backward]	2.5506ms	2.4363ms	410.4646 Ops/s	349.9893 Ops/s	$\textbf{\color{#35bf28}+17.28\%}$
test_ddpg_speed[reduce-overhead-None]	1.5195ms	1.3920ms	718.3986 Ops/s	720.4898 Ops/s	$\color{#d91a1a}-0.29\%$
test_ddpg_speed[reduce-overhead-backward]	2.4941ms	2.3798ms	420.2073 Ops/s	340.0793 Ops/s	$\textbf{\color{#35bf28}+23.56\%}$
test_sac_speed[False-None]	8.5243ms	7.9412ms	125.9250 Ops/s	123.7719 Ops/s	$\color{#35bf28}+1.74\%$
test_sac_speed[False-backward]	11.9156ms	11.4210ms	87.5579 Ops/s	89.2091 Ops/s	$\color{#d91a1a}-1.85\%$
test_sac_speed[True-None]	2.5166ms	2.1696ms	460.9176 Ops/s	459.1152 Ops/s	$\color{#35bf28}+0.39\%$
test_sac_speed[True-backward]	4.2926ms	4.1382ms	241.6507 Ops/s	239.9685 Ops/s	$\color{#35bf28}+0.70\%$
test_sac_speed[reduce-overhead-None]	2.7149ms	2.1756ms	459.6536 Ops/s	450.3840 Ops/s	$\color{#35bf28}+2.06\%$
test_sac_speed[reduce-overhead-backward]	4.3818ms	4.1710ms	239.7478 Ops/s	218.8814 Ops/s	$\textbf{\color{#35bf28}+9.53\%}$
test_redq_speed[False-None]	11.1424ms	10.5839ms	94.4832 Ops/s	91.6235 Ops/s	$\color{#35bf28}+3.12\%$
test_redq_speed[False-backward]	19.2648ms	18.6065ms	53.7446 Ops/s	54.2612 Ops/s	$\color{#d91a1a}-0.95\%$
test_redq_speed[True-None]	5.0842ms	4.7267ms	211.5641 Ops/s	213.2328 Ops/s	$\color{#d91a1a}-0.78\%$
test_redq_speed[True-backward]	12.2059ms	10.2271ms	97.7790 Ops/s	99.8467 Ops/s	$\color{#d91a1a}-2.07\%$
test_redq_speed[reduce-overhead-None]	5.0282ms	4.5471ms	219.9189 Ops/s	209.8366 Ops/s	$\color{#35bf28}+4.80\%$
test_redq_speed[reduce-overhead-backward]	10.7727ms	10.3920ms	96.2280 Ops/s	99.0903 Ops/s	$\color{#d91a1a}-2.89\%$
test_redq_deprec_speed[False-None]	11.5985ms	11.2244ms	89.0917 Ops/s	90.4176 Ops/s	$\color{#d91a1a}-1.47\%$
test_redq_deprec_speed[False-backward]	17.0264ms	16.3184ms	61.2805 Ops/s	62.4528 Ops/s	$\color{#d91a1a}-1.88\%$
test_redq_deprec_speed[True-None]	4.1054ms	3.7493ms	266.7180 Ops/s	256.9978 Ops/s	$\color{#35bf28}+3.78\%$
test_redq_deprec_speed[True-backward]	8.4299ms	7.8890ms	126.7586 Ops/s	121.7952 Ops/s	$\color{#35bf28}+4.08\%$
test_redq_deprec_speed[reduce-overhead-None]	4.0929ms	3.6897ms	271.0282 Ops/s	265.2000 Ops/s	$\color{#35bf28}+2.20\%$
test_redq_deprec_speed[reduce-overhead-backward]	8.1948ms	7.9780ms	125.3454 Ops/s	124.4971 Ops/s	$\color{#35bf28}+0.68\%$
test_td3_speed[False-None]	8.1636ms	8.0454ms	124.2946 Ops/s	119.1778 Ops/s	$\color{#35bf28}+4.29\%$
test_td3_speed[False-backward]	11.7151ms	11.1145ms	89.9722 Ops/s	91.9514 Ops/s	$\color{#d91a1a}-2.15\%$
test_td3_speed[True-None]	1.9741ms	1.8883ms	529.5693 Ops/s	534.6483 Ops/s	$\color{#d91a1a}-0.95\%$
test_td3_speed[True-backward]	3.8503ms	3.7211ms	268.7388 Ops/s	241.1937 Ops/s	$\textbf{\color{#35bf28}+11.42\%}$
test_td3_speed[reduce-overhead-None]	1.9078ms	1.8573ms	538.4139 Ops/s	542.3635 Ops/s	$\color{#d91a1a}-0.73\%$
test_td3_speed[reduce-overhead-backward]	4.1036ms	3.8047ms	262.8334 Ops/s	257.4123 Ops/s	$\color{#35bf28}+2.11\%$
test_cql_speed[False-None]	29.4005ms	26.7850ms	37.3343 Ops/s	37.2650 Ops/s	$\color{#35bf28}+0.19\%$
test_cql_speed[False-backward]	38.5257ms	36.3489ms	27.5112 Ops/s	27.2946 Ops/s	$\color{#35bf28}+0.79\%$
test_cql_speed[True-None]	13.3907ms	12.7753ms	78.2759 Ops/s	75.9565 Ops/s	$\color{#35bf28}+3.05\%$
test_cql_speed[True-backward]	19.5788ms	18.9521ms	52.7646 Ops/s	54.0722 Ops/s	$\color{#d91a1a}-2.42\%$
test_cql_speed[reduce-overhead-None]	13.2303ms	12.8104ms	78.0616 Ops/s	78.4607 Ops/s	$\color{#d91a1a}-0.51\%$
test_cql_speed[reduce-overhead-backward]	19.6486ms	19.0000ms	52.6316 Ops/s	54.4084 Ops/s	$\color{#d91a1a}-3.27\%$
test_a2c_speed[False-None]	6.0881ms	5.5487ms	180.2221 Ops/s	180.1731 Ops/s	$\color{#35bf28}+0.03\%$
test_a2c_speed[False-backward]	12.7470ms	12.1847ms	82.0700 Ops/s	83.2385 Ops/s	$\color{#d91a1a}-1.40\%$
test_a2c_speed[True-None]	4.2901ms	3.8031ms	262.9412 Ops/s	260.9535 Ops/s	$\color{#35bf28}+0.76\%$
test_a2c_speed[True-backward]	9.1042ms	8.8441ms	113.0693 Ops/s	105.8512 Ops/s	$\textbf{\color{#35bf28}+6.82\%}$
test_a2c_speed[reduce-overhead-None]	4.3006ms	3.8374ms	260.5908 Ops/s	267.0480 Ops/s	$\color{#d91a1a}-2.42\%$
test_a2c_speed[reduce-overhead-backward]	9.4048ms	9.1379ms	109.4341 Ops/s	108.1724 Ops/s	$\color{#35bf28}+1.17\%$
test_ppo_speed[False-None]	6.4474ms	6.0612ms	164.9835 Ops/s	164.0276 Ops/s	$\color{#35bf28}+0.58\%$
test_ppo_speed[False-backward]	13.6858ms	13.0841ms	76.4286 Ops/s	76.9211 Ops/s	$\color{#d91a1a}-0.64\%$
test_ppo_speed[True-None]	3.9191ms	3.7272ms	268.2952 Ops/s	269.8237 Ops/s	$\color{#d91a1a}-0.57\%$
test_ppo_speed[True-backward]	9.2121ms	8.7656ms	114.0829 Ops/s	115.1713 Ops/s	$\color{#d91a1a}-0.95\%$
test_ppo_speed[reduce-overhead-None]	3.8984ms	3.7433ms	267.1415 Ops/s	272.0186 Ops/s	$\color{#d91a1a}-1.79\%$
test_ppo_speed[reduce-overhead-backward]	9.3986ms	8.9940ms	111.1847 Ops/s	111.0232 Ops/s	$\color{#35bf28}+0.15\%$
test_reinforce_speed[False-None]	5.0654ms	4.7232ms	211.7203 Ops/s	212.4045 Ops/s	$\color{#d91a1a}-0.32\%$
test_reinforce_speed[False-backward]	7.8843ms	7.6451ms	130.8019 Ops/s	131.7829 Ops/s	$\color{#d91a1a}-0.74\%$
test_reinforce_speed[True-None]	3.3925ms	2.9783ms	335.7613 Ops/s	339.8325 Ops/s	$\color{#d91a1a}-1.20\%$
test_reinforce_speed[True-backward]	8.4696ms	7.9478ms	125.8215 Ops/s	128.2364 Ops/s	$\color{#d91a1a}-1.88\%$
test_reinforce_speed[reduce-overhead-None]	3.5044ms	2.9511ms	338.8603 Ops/s	328.2882 Ops/s	$\color{#35bf28}+3.22\%$
test_reinforce_speed[reduce-overhead-backward]	8.5457ms	8.1731ms	122.3527 Ops/s	120.1855 Ops/s	$\color{#35bf28}+1.80\%$
test_iql_speed[False-None]	25.5006ms	20.9281ms	47.7826 Ops/s	47.3262 Ops/s	$\color{#35bf28}+0.96\%$
test_iql_speed[False-backward]	37.0261ms	31.7070ms	31.5387 Ops/s	31.3913 Ops/s	$\color{#35bf28}+0.47\%$
test_iql_speed[True-None]	9.1901ms	8.7702ms	114.0221 Ops/s	112.6333 Ops/s	$\color{#35bf28}+1.23\%$
test_iql_speed[True-backward]	17.8956ms	17.2840ms	57.8571 Ops/s	57.7736 Ops/s	$\color{#35bf28}+0.14\%$
test_iql_speed[reduce-overhead-None]	9.2250ms	8.8576ms	112.8971 Ops/s	101.1576 Ops/s	$\textbf{\color{#35bf28}+11.61\%}$
test_iql_speed[reduce-overhead-backward]	18.2394ms	17.7791ms	56.2457 Ops/s	56.8737 Ops/s	$\color{#d91a1a}-1.10\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.5082ms	6.0508ms	165.2660 Ops/s	167.8006 Ops/s	$\color{#d91a1a}-1.51\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5903ms	0.3129ms	3.1958 KOps/s	3.4974 KOps/s	$\textbf{\color{#d91a1a}-8.62\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7019ms	0.2966ms	3.3710 KOps/s	3.7862 KOps/s	$\textbf{\color{#d91a1a}-10.96\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9525ms	5.7244ms	174.6920 Ops/s	176.3880 Ops/s	$\color{#d91a1a}-0.96\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.0528ms	0.3269ms	3.0588 KOps/s	3.3206 KOps/s	$\textbf{\color{#d91a1a}-7.88\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6690ms	0.3251ms	3.0755 KOps/s	3.8050 KOps/s	$\textbf{\color{#d91a1a}-19.17\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.6936ms	1.4522ms	688.6236 Ops/s	790.4668 Ops/s	$\textbf{\color{#d91a1a}-12.88\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.7932ms	1.3623ms	734.0330 Ops/s	841.8463 Ops/s	$\textbf{\color{#d91a1a}-12.81\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	9.9419ms	6.1842ms	161.7017 Ops/s	170.9303 Ops/s	$\textbf{\color{#d91a1a}-5.40\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.9335ms	0.5329ms	1.8765 KOps/s	1.9972 KOps/s	$\textbf{\color{#d91a1a}-6.04\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7652ms	0.5143ms	1.9442 KOps/s	2.0665 KOps/s	$\textbf{\color{#d91a1a}-5.92\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.9139ms	5.7395ms	174.2299 Ops/s	176.8036 Ops/s	$\color{#d91a1a}-1.46\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	2.0955ms	0.3495ms	2.8609 KOps/s	765.9590 Ops/s	$\textbf{\color{#35bf28}+273.50\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5558ms	0.3633ms	2.7523 KOps/s	3.7668 KOps/s	$\textbf{\color{#d91a1a}-26.93\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.0466ms	5.7170ms	174.9154 Ops/s	174.4321 Ops/s	$\color{#35bf28}+0.28\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6775ms	0.3437ms	2.9099 KOps/s	3.5673 KOps/s	$\textbf{\color{#d91a1a}-18.43\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5779ms	0.3181ms	3.1436 KOps/s	3.8307 KOps/s	$\textbf{\color{#d91a1a}-17.94\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.1204ms	5.9120ms	169.1488 Ops/s	167.9258 Ops/s	$\color{#35bf28}+0.73\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0209ms	0.4803ms	2.0822 KOps/s	2.2688 KOps/s	$\textbf{\color{#d91a1a}-8.22\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7043ms	0.4630ms	2.1598 KOps/s	2.4071 KOps/s	$\textbf{\color{#d91a1a}-10.27\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	6.5568ms	5.0397ms	198.4264 Ops/s	194.8688 Ops/s	$\color{#35bf28}+1.83\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	9.1816ms	2.3235ms	430.3776 Ops/s	430.5643 Ops/s	$\color{#d91a1a}-0.04\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	7.1891ms	1.2634ms	791.5359 Ops/s	806.9852 Ops/s	$\color{#d91a1a}-1.91\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.5136s	15.2635ms	65.5158 Ops/s	56.2376 Ops/s	$\textbf{\color{#35bf28}+16.50\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	11.6708ms	1.9478ms	513.3969 Ops/s	482.0358 Ops/s	$\textbf{\color{#35bf28}+6.51\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.4469ms	1.1861ms	843.1119 Ops/s	852.0853 Ops/s	$\color{#d91a1a}-1.05\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	6.7282ms	5.2411ms	190.7989 Ops/s	189.0141 Ops/s	$\color{#35bf28}+0.94\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	10.8600ms	2.2423ms	445.9627 Ops/s	466.1618 Ops/s	$\color{#d91a1a}-4.33\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	3.7245ms	1.2582ms	794.8147 Ops/s	710.4110 Ops/s	$\textbf{\color{#35bf28}+11.88\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	34.8094ms	32.6412ms	30.6361 Ops/s	30.3201 Ops/s	$\color{#35bf28}+1.04\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.4515ms	17.8073ms	56.1568 Ops/s	57.4918 Ops/s	$\color{#d91a1a}-2.32\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	36.7578ms	34.1710ms	29.2646 Ops/s	29.4051 Ops/s	$\color{#d91a1a}-0.48\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.0116ms	17.6463ms	56.6692 Ops/s	56.2139 Ops/s	$\color{#35bf28}+0.81\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	37.8901ms	36.1686ms	27.6483 Ops/s	28.1393 Ops/s	$\color{#d91a1a}-1.74\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.5262ms	19.3098ms	51.7872 Ops/s	52.5179 Ops/s	$\color{#d91a1a}-1.39\%$