[Feature] Documentation

[vmoens]

Stack from ghstack (oldest at bottom):

• [Quality] Fix flaky test #3211
• -> [Feature] Documentation #3192
• [Feature] SAC Trainer #3191
• [Feature] PPO Trainer Updates #3190
• [Feature] Trainer Algorithms - Configuration System #3189
• [Feature] Trainer Infrastructure - Timing and Utilities #3188

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3192

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 7 New Failures, 9 Unrelated Failures

As of commit e31da55 with merge base 92c20cd ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 139.
• Generate documentation / build-docs (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t a39e4b8a1536239f82c0f8af62a8bbfd9511f2601a06ef1bf95456fdb7d02579 /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t c9d521a5861793b87aee19fcb7c443af3ca23d7a361bd9e192adfc163541ce69 /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t b88d7ef2c7f6e07779603a5db2a45dfc3f5247ae6eafab4473dac0e0e1668da6 /exec failed with exit code 1
• Libs Tests on Linux / unittests-sklearn (3.10, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t fc79a4b2198bfd477ec53d8e4257a69293f4fa43ff3931ab45388234ad1209c0 /exec failed with exit code 2
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 5f43cd965ae552adfb002657edb292759367a4c85c48c6a0a4d0d2c8f3778122 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  test/test_weightsync.py::TestMultiModelUpdates::test_multi_model_tensordict_updates

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh) (similar failure)
  test/test_weightsync.py::TestMultiModelUpdates::test_multi_model_tensordict_updates

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• LLM Tests on Linux / unittests (3.9, 12.8) / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh) (trunk failure)
  test/test_weightsync.py::TestMultiModelUpdates::test_multi_model_tensordict_updates
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh) (trunk failure)
  test/test_weightsync.py::TestMultiModelUpdates::test_multi_model_tensordict_updates
• Unit-tests on Linux / tests-gpu (3.11, 12.8) / linux-job (gh) (trunk failure)
  test/test_weightsync.py::TestMultiModelUpdates::test_multi_model_tensordict_updates
• Unit-tests on Linux / tests-olddeps (3.9, 11.6) / linux-job (gh) (trunk failure)
  test/test_weightsync.py::TestMultiModelUpdates::test_multi_model_tensordict_updates
• Unit-tests on Linux / tests-optdeps (3.11, 12.8) / linux-job (gh) (trunk failure)
  test/test_weightsync.py::TestMultiModelUpdates::test_multi_model_tensordict_updates
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh) (trunk failure)
  test/test_weightsync.py::TestMultiModelUpdates::test_multi_model_tensordict_updates
• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh) (trunk failure)
  Process completed with exit code 2.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}19$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	85.4525μs	82.7023μs	12.0916 KOps/s	11.7922 KOps/s	$\color{#35bf28}+2.54\%$
test_tensor_to_bytestream_speed[torch.save]	0.1438ms	0.1421ms	7.0389 KOps/s	7.0240 KOps/s	$\color{#35bf28}+0.21\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1202s	0.1197s	8.3536 Ops/s	8.1909 Ops/s	$\color{#35bf28}+1.99\%$
test_tensor_to_bytestream_speed[numpy]	2.8363μs	2.8300μs	353.3598 KOps/s	355.8075 KOps/s	$\color{#d91a1a}-0.69\%$
test_tensor_to_bytestream_speed[safetensors]	46.2353μs	44.8364μs	22.3033 KOps/s	22.6387 KOps/s	$\color{#d91a1a}-1.48\%$
test_simple	0.5603s	0.5560s	1.7985 Ops/s	1.7253 Ops/s	$\color{#35bf28}+4.25\%$
test_transformed	1.1201s	1.1173s	0.8950 Ops/s	0.8750 Ops/s	$\color{#35bf28}+2.28\%$
test_serial	1.6871s	1.6815s	0.5947 Ops/s	0.5836 Ops/s	$\color{#35bf28}+1.91\%$
test_parallel	1.1721s	1.0904s	0.9171 Ops/s	0.8830 Ops/s	$\color{#35bf28}+3.86\%$
test_step_mdp_speed[True-True-True-True-True]	0.2606ms	45.9847μs	21.7464 KOps/s	21.4300 KOps/s	$\color{#35bf28}+1.48\%$
test_step_mdp_speed[True-True-True-True-False]	70.5420μs	25.4937μs	39.2254 KOps/s	37.8080 KOps/s	$\color{#35bf28}+3.75\%$
test_step_mdp_speed[True-True-True-False-True]	53.4910μs	25.1929μs	39.6938 KOps/s	38.1673 KOps/s	$\color{#35bf28}+4.00\%$
test_step_mdp_speed[True-True-True-False-False]	41.4510μs	14.0703μs	71.0719 KOps/s	68.4907 KOps/s	$\color{#35bf28}+3.77\%$
test_step_mdp_speed[True-True-False-True-True]	76.9820μs	49.3840μs	20.2495 KOps/s	19.8898 KOps/s	$\color{#35bf28}+1.81\%$
test_step_mdp_speed[True-True-False-True-False]	54.1010μs	28.5312μs	35.0494 KOps/s	34.4478 KOps/s	$\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-True-False-False-True]	60.1310μs	28.4312μs	35.1726 KOps/s	34.1734 KOps/s	$\color{#35bf28}+2.92\%$
test_step_mdp_speed[True-True-False-False-False]	61.4310μs	16.9205μs	59.0997 KOps/s	56.8313 KOps/s	$\color{#35bf28}+3.99\%$
test_step_mdp_speed[True-False-True-True-True]	0.1287ms	51.3264μs	19.4832 KOps/s	18.5882 KOps/s	$\color{#35bf28}+4.81\%$
test_step_mdp_speed[True-False-True-True-False]	63.3510μs	31.0932μs	32.1614 KOps/s	31.1827 KOps/s	$\color{#35bf28}+3.14\%$
test_step_mdp_speed[True-False-True-False-True]	60.6510μs	28.3724μs	35.2455 KOps/s	33.6026 KOps/s	$\color{#35bf28}+4.89\%$
test_step_mdp_speed[True-False-True-False-False]	44.2600μs	16.9272μs	59.0766 KOps/s	56.7123 KOps/s	$\color{#35bf28}+4.17\%$
test_step_mdp_speed[True-False-False-True-True]	82.4120μs	53.9636μs	18.5310 KOps/s	17.6179 KOps/s	$\textbf{\color{#35bf28}+5.18\%}$
test_step_mdp_speed[True-False-False-True-False]	0.1064ms	33.6787μs	29.6924 KOps/s	28.7430 KOps/s	$\color{#35bf28}+3.30\%$
test_step_mdp_speed[True-False-False-False-True]	59.4110μs	31.1197μs	32.1340 KOps/s	30.9194 KOps/s	$\color{#35bf28}+3.93\%$
test_step_mdp_speed[True-False-False-False-False]	56.0810μs	19.5838μs	51.0626 KOps/s	49.2084 KOps/s	$\color{#35bf28}+3.77\%$
test_step_mdp_speed[False-True-True-True-True]	83.8520μs	51.4723μs	19.4279 KOps/s	18.6878 KOps/s	$\color{#35bf28}+3.96\%$
test_step_mdp_speed[False-True-True-True-False]	65.3710μs	31.1993μs	32.0520 KOps/s	30.8055 KOps/s	$\color{#35bf28}+4.05\%$
test_step_mdp_speed[False-True-True-False-True]	2.3847ms	32.5140μs	30.7560 KOps/s	28.8557 KOps/s	$\textbf{\color{#35bf28}+6.59\%}$
test_step_mdp_speed[False-True-True-False-False]	63.9610μs	18.9684μs	52.7194 KOps/s	51.6750 KOps/s	$\color{#35bf28}+2.02\%$
test_step_mdp_speed[False-True-False-True-True]	83.9910μs	53.9156μs	18.5475 KOps/s	17.6958 KOps/s	$\color{#35bf28}+4.81\%$
test_step_mdp_speed[False-True-False-True-False]	78.1420μs	34.0975μs	29.3277 KOps/s	28.5460 KOps/s	$\color{#35bf28}+2.74\%$
test_step_mdp_speed[False-True-False-False-True]	69.6110μs	34.7669μs	28.7630 KOps/s	27.4169 KOps/s	$\color{#35bf28}+4.91\%$
test_step_mdp_speed[False-True-False-False-False]	51.7510μs	21.3913μs	46.7480 KOps/s	45.8296 KOps/s	$\color{#35bf28}+2.00\%$
test_step_mdp_speed[False-False-True-True-True]	86.4720μs	56.8158μs	17.6008 KOps/s	17.1412 KOps/s	$\color{#35bf28}+2.68\%$
test_step_mdp_speed[False-False-True-True-False]	65.6920μs	36.6216μs	27.3063 KOps/s	26.7075 KOps/s	$\color{#35bf28}+2.24\%$
test_step_mdp_speed[False-False-True-False-True]	64.9910μs	34.7202μs	28.8017 KOps/s	27.1774 KOps/s	$\textbf{\color{#35bf28}+5.98\%}$
test_step_mdp_speed[False-False-True-False-False]	63.5710μs	21.4608μs	46.5965 KOps/s	45.3119 KOps/s	$\color{#35bf28}+2.83\%$
test_step_mdp_speed[False-False-False-True-True]	0.1059ms	58.5245μs	17.0869 KOps/s	16.3643 KOps/s	$\color{#35bf28}+4.42\%$
test_step_mdp_speed[False-False-False-True-False]	69.2120μs	38.7827μs	25.7847 KOps/s	24.8508 KOps/s	$\color{#35bf28}+3.76\%$
test_step_mdp_speed[False-False-False-False-True]	67.2910μs	37.1932μs	26.8867 KOps/s	25.8729 KOps/s	$\color{#35bf28}+3.92\%$
test_step_mdp_speed[False-False-False-False-False]	53.0210μs	23.8817μs	41.8731 KOps/s	41.1524 KOps/s	$\color{#35bf28}+1.75\%$
test_values[generalized_advantage_estimate-True-True]	10.4688ms	10.1725ms	98.3042 Ops/s	101.7241 Ops/s	$\color{#d91a1a}-3.36\%$
test_values[vec_generalized_advantage_estimate-True-True]	19.5526ms	17.5245ms	57.0629 Ops/s	57.4228 Ops/s	$\color{#d91a1a}-0.63\%$
test_values[td0_return_estimate-False-False]	0.2430ms	0.1320ms	7.5741 KOps/s	7.9975 KOps/s	$\textbf{\color{#d91a1a}-5.29\%}$
test_values[td1_return_estimate-False-False]	28.5897ms	28.1857ms	35.4790 Ops/s	36.3333 Ops/s	$\color{#d91a1a}-2.35\%$
test_values[vec_td1_return_estimate-False-False]	17.8686ms	17.5954ms	56.8330 Ops/s	57.3832 Ops/s	$\color{#d91a1a}-0.96\%$
test_values[td_lambda_return_estimate-True-False]	42.7794ms	42.1170ms	23.7434 Ops/s	24.2883 Ops/s	$\color{#d91a1a}-2.24\%$
test_values[vec_td_lambda_return_estimate-True-False]	18.0507ms	17.5966ms	56.8291 Ops/s	57.4081 Ops/s	$\color{#d91a1a}-1.01\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.1867ms	9.0368ms	110.6583 Ops/s	116.7521 Ops/s	$\textbf{\color{#d91a1a}-5.22\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.6722ms	1.5121ms	661.3413 Ops/s	650.9929 Ops/s	$\color{#35bf28}+1.59\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4520ms	0.4158ms	2.4052 KOps/s	2.4117 KOps/s	$\color{#d91a1a}-0.27\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	34.8689ms	34.4821ms	29.0006 Ops/s	28.7676 Ops/s	$\color{#35bf28}+0.81\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.8565ms	1.7180ms	582.0815 Ops/s	586.3248 Ops/s	$\color{#d91a1a}-0.72\%$
test_dqn_speed[False-None]	6.4562ms	1.4267ms	700.9066 Ops/s	690.8889 Ops/s	$\color{#35bf28}+1.45\%$
test_dqn_speed[False-backward]	1.9831ms	1.9245ms	519.6208 Ops/s	510.8677 Ops/s	$\color{#35bf28}+1.71\%$
test_dqn_speed[True-None]	0.7183ms	0.5363ms	1.8646 KOps/s	1.8152 KOps/s	$\color{#35bf28}+2.72\%$
test_dqn_speed[True-backward]	1.0432ms	0.9935ms	1.0065 KOps/s	1.0124 KOps/s	$\color{#d91a1a}-0.58\%$
test_dqn_speed[reduce-overhead-None]	0.6336ms	0.5237ms	1.9095 KOps/s	1.9096 KOps/s	$-0.01\%$
test_dqn_speed[reduce-overhead-backward]	1.0403ms	0.9722ms	1.0286 KOps/s	1.0345 KOps/s	$\color{#d91a1a}-0.56\%$
test_ddpg_speed[False-None]	3.1947ms	2.9054ms	344.1923 Ops/s	341.3424 Ops/s	$\color{#35bf28}+0.83\%$
test_ddpg_speed[False-backward]	4.5143ms	4.1696ms	239.8289 Ops/s	240.3755 Ops/s	$\color{#d91a1a}-0.23\%$
test_ddpg_speed[True-None]	1.5364ms	1.4195ms	704.4541 Ops/s	694.5507 Ops/s	$\color{#35bf28}+1.43\%$
test_ddpg_speed[True-backward]	2.4044ms	2.3638ms	423.0417 Ops/s	412.2486 Ops/s	$\color{#35bf28}+2.62\%$
test_ddpg_speed[reduce-overhead-None]	1.6138ms	1.3979ms	715.3524 Ops/s	711.3877 Ops/s	$\color{#35bf28}+0.56\%$
test_ddpg_speed[reduce-overhead-backward]	2.4115ms	2.3620ms	423.3667 Ops/s	417.3208 Ops/s	$\color{#35bf28}+1.45\%$
test_sac_speed[False-None]	8.3770ms	7.9337ms	126.0451 Ops/s	124.4253 Ops/s	$\color{#35bf28}+1.30\%$
test_sac_speed[False-backward]	11.4948ms	11.2003ms	89.2837 Ops/s	88.3555 Ops/s	$\color{#35bf28}+1.05\%$
test_sac_speed[True-None]	2.2466ms	2.1185ms	472.0287 Ops/s	463.5276 Ops/s	$\color{#35bf28}+1.83\%$
test_sac_speed[True-backward]	4.4663ms	4.0476ms	247.0597 Ops/s	245.8993 Ops/s	$\color{#35bf28}+0.47\%$
test_sac_speed[reduce-overhead-None]	2.3964ms	2.1140ms	473.0387 Ops/s	460.3387 Ops/s	$\color{#35bf28}+2.76\%$
test_sac_speed[reduce-overhead-backward]	4.1366ms	4.0002ms	249.9866 Ops/s	222.4415 Ops/s	$\textbf{\color{#35bf28}+12.38\%}$
test_redq_speed[False-None]	15.2476ms	10.3251ms	96.8513 Ops/s	96.3225 Ops/s	$\color{#35bf28}+0.55\%$
test_redq_speed[False-backward]	22.9603ms	18.2579ms	54.7707 Ops/s	54.8727 Ops/s	$\color{#d91a1a}-0.19\%$
test_redq_speed[True-None]	4.5244ms	4.2968ms	232.7314 Ops/s	224.5368 Ops/s	$\color{#35bf28}+3.65\%$
test_redq_speed[True-backward]	10.0066ms	9.6812ms	103.2925 Ops/s	101.5813 Ops/s	$\color{#35bf28}+1.68\%$
test_redq_speed[reduce-overhead-None]	4.4700ms	4.2549ms	235.0215 Ops/s	221.2294 Ops/s	$\textbf{\color{#35bf28}+6.23\%}$
test_redq_speed[reduce-overhead-backward]	10.0441ms	9.8706ms	101.3105 Ops/s	96.0947 Ops/s	$\textbf{\color{#35bf28}+5.43\%}$
test_redq_deprec_speed[False-None]	11.2498ms	10.8754ms	91.9510 Ops/s	88.0267 Ops/s	$\color{#35bf28}+4.46\%$
test_redq_deprec_speed[False-backward]	16.2769ms	15.6494ms	63.9003 Ops/s	61.0826 Ops/s	$\color{#35bf28}+4.61\%$
test_redq_deprec_speed[True-None]	3.7359ms	3.5600ms	280.9019 Ops/s	257.2055 Ops/s	$\textbf{\color{#35bf28}+9.21\%}$
test_redq_deprec_speed[True-backward]	7.5174ms	7.2676ms	137.5979 Ops/s	119.2701 Ops/s	$\textbf{\color{#35bf28}+15.37\%}$
test_redq_deprec_speed[reduce-overhead-None]	4.0803ms	3.5204ms	284.0567 Ops/s	271.4032 Ops/s	$\color{#35bf28}+4.66\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.5513ms	7.2899ms	137.1756 Ops/s	128.8697 Ops/s	$\textbf{\color{#35bf28}+6.45\%}$
test_td3_speed[False-None]	8.1696ms	7.9843ms	125.2459 Ops/s	124.5048 Ops/s	$\color{#35bf28}+0.60\%$
test_td3_speed[False-backward]	11.5349ms	10.8634ms	92.0520 Ops/s	91.2205 Ops/s	$\color{#35bf28}+0.91\%$
test_td3_speed[True-None]	1.8350ms	1.7940ms	557.4132 Ops/s	537.0180 Ops/s	$\color{#35bf28}+3.80\%$
test_td3_speed[True-backward]	3.6931ms	3.5944ms	278.2111 Ops/s	273.8372 Ops/s	$\color{#35bf28}+1.60\%$
test_td3_speed[reduce-overhead-None]	1.8514ms	1.7726ms	564.1355 Ops/s	545.9850 Ops/s	$\color{#35bf28}+3.32\%$
test_td3_speed[reduce-overhead-backward]	3.7963ms	3.6254ms	275.8321 Ops/s	274.8415 Ops/s	$\color{#35bf28}+0.36\%$
test_cql_speed[False-None]	28.9942ms	26.1462ms	38.2465 Ops/s	38.0764 Ops/s	$\color{#35bf28}+0.45\%$
test_cql_speed[False-backward]	36.8491ms	35.2889ms	28.3375 Ops/s	28.1973 Ops/s	$\color{#35bf28}+0.50\%$
test_cql_speed[True-None]	12.7815ms	12.3168ms	81.1898 Ops/s	79.6587 Ops/s	$\color{#35bf28}+1.92\%$
test_cql_speed[True-backward]	19.2252ms	18.2797ms	54.7056 Ops/s	54.6154 Ops/s	$\color{#35bf28}+0.17\%$
test_cql_speed[reduce-overhead-None]	12.8490ms	12.4506ms	80.3177 Ops/s	78.5248 Ops/s	$\color{#35bf28}+2.28\%$
test_cql_speed[reduce-overhead-backward]	18.6633ms	18.3554ms	54.4799 Ops/s	55.8866 Ops/s	$\color{#d91a1a}-2.52\%$
test_a2c_speed[False-None]	5.8419ms	5.3204ms	187.9550 Ops/s	182.5379 Ops/s	$\color{#35bf28}+2.97\%$
test_a2c_speed[False-backward]	12.2238ms	11.8492ms	84.3939 Ops/s	84.8853 Ops/s	$\color{#d91a1a}-0.58\%$
test_a2c_speed[True-None]	4.0147ms	3.6759ms	272.0394 Ops/s	258.1745 Ops/s	$\textbf{\color{#35bf28}+5.37\%}$
test_a2c_speed[True-backward]	9.0620ms	8.7119ms	114.7860 Ops/s	113.0837 Ops/s	$\color{#35bf28}+1.51\%$
test_a2c_speed[reduce-overhead-None]	3.8297ms	3.7151ms	269.1717 Ops/s	267.7005 Ops/s	$\color{#35bf28}+0.55\%$
test_a2c_speed[reduce-overhead-backward]	8.9811ms	8.8203ms	113.3751 Ops/s	107.0449 Ops/s	$\textbf{\color{#35bf28}+5.91\%}$
test_ppo_speed[False-None]	6.0802ms	5.8429ms	171.1473 Ops/s	168.1303 Ops/s	$\color{#35bf28}+1.79\%$
test_ppo_speed[False-backward]	12.9058ms	12.6060ms	79.3272 Ops/s	79.6478 Ops/s	$\color{#d91a1a}-0.40\%$
test_ppo_speed[True-None]	3.7866ms	3.6555ms	273.5595 Ops/s	263.0721 Ops/s	$\color{#35bf28}+3.99\%$
test_ppo_speed[True-backward]	8.6289ms	8.5010ms	117.6333 Ops/s	113.3863 Ops/s	$\color{#35bf28}+3.75\%$
test_ppo_speed[reduce-overhead-None]	3.8099ms	3.6267ms	275.7306 Ops/s	271.5360 Ops/s	$\color{#35bf28}+1.54\%$
test_ppo_speed[reduce-overhead-backward]	9.2499ms	8.8549ms	112.9316 Ops/s	102.4802 Ops/s	$\textbf{\color{#35bf28}+10.20\%}$
test_reinforce_speed[False-None]	4.9320ms	4.5997ms	217.4038 Ops/s	212.0409 Ops/s	$\color{#35bf28}+2.53\%$
test_reinforce_speed[False-backward]	7.6913ms	7.4683ms	133.8987 Ops/s	133.0178 Ops/s	$\color{#35bf28}+0.66\%$
test_reinforce_speed[True-None]	2.9990ms	2.8483ms	351.0818 Ops/s	338.7016 Ops/s	$\color{#35bf28}+3.66\%$
test_reinforce_speed[True-backward]	8.0209ms	7.7725ms	128.6590 Ops/s	128.1642 Ops/s	$\color{#35bf28}+0.39\%$
test_reinforce_speed[reduce-overhead-None]	2.9787ms	2.8416ms	351.9136 Ops/s	344.0797 Ops/s	$\color{#35bf28}+2.28\%$
test_reinforce_speed[reduce-overhead-backward]	8.3850ms	7.9556ms	125.6973 Ops/s	111.1451 Ops/s	$\textbf{\color{#35bf28}+13.09\%}$
test_iql_speed[False-None]	25.8544ms	20.4250ms	48.9597 Ops/s	48.5924 Ops/s	$\color{#35bf28}+0.76\%$
test_iql_speed[False-backward]	36.9054ms	31.2327ms	32.0177 Ops/s	32.2124 Ops/s	$\color{#d91a1a}-0.60\%$
test_iql_speed[True-None]	8.8467ms	8.5087ms	117.5265 Ops/s	113.2269 Ops/s	$\color{#35bf28}+3.80\%$
test_iql_speed[True-backward]	17.1641ms	16.7965ms	59.5363 Ops/s	59.4458 Ops/s	$\color{#35bf28}+0.15\%$
test_iql_speed[reduce-overhead-None]	8.8682ms	8.5925ms	116.3810 Ops/s	116.3591 Ops/s	$\color{#35bf28}+0.02\%$
test_iql_speed[reduce-overhead-backward]	18.3580ms	17.2093ms	58.1082 Ops/s	58.2566 Ops/s	$\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.6679ms	6.0204ms	166.1012 Ops/s	165.8412 Ops/s	$\color{#35bf28}+0.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5799ms	0.3235ms	3.0910 KOps/s	3.1265 KOps/s	$\color{#d91a1a}-1.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5684ms	0.3048ms	3.2804 KOps/s	3.2703 KOps/s	$\color{#35bf28}+0.31\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9874ms	5.7307ms	174.4999 Ops/s	174.2406 Ops/s	$\color{#35bf28}+0.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.0381ms	0.2739ms	3.6504 KOps/s	3.5869 KOps/s	$\color{#35bf28}+1.77\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.4736ms	0.2541ms	3.9347 KOps/s	3.2287 KOps/s	$\textbf{\color{#35bf28}+21.87\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.4597ms	1.2555ms	796.4689 Ops/s	800.3913 Ops/s	$\color{#d91a1a}-0.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.3604ms	1.1687ms	855.6436 Ops/s	858.6835 Ops/s	$\color{#d91a1a}-0.35\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.0189ms	5.8645ms	170.5164 Ops/s	169.5522 Ops/s	$\color{#35bf28}+0.57\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1314ms	0.4247ms	2.3544 KOps/s	2.2992 KOps/s	$\color{#35bf28}+2.40\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.5689ms	0.4025ms	2.4844 KOps/s	2.2341 KOps/s	$\textbf{\color{#35bf28}+11.20\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.8872ms	5.7660ms	173.4296 Ops/s	173.0899 Ops/s	$\color{#35bf28}+0.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0784ms	0.3300ms	3.0302 KOps/s	836.9448 Ops/s	$\textbf{\color{#35bf28}+262.06\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5830ms	0.3037ms	3.2932 KOps/s	3.0935 KOps/s	$\textbf{\color{#35bf28}+6.45\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9175ms	5.6980ms	175.4992 Ops/s	171.4491 Ops/s	$\color{#35bf28}+2.36\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.1847ms	0.3660ms	2.7320 KOps/s	3.3429 KOps/s	$\textbf{\color{#d91a1a}-18.28\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7681ms	0.3417ms	2.9266 KOps/s	3.2801 KOps/s	$\textbf{\color{#d91a1a}-10.78\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.9371ms	5.8380ms	171.2920 Ops/s	165.8074 Ops/s	$\color{#35bf28}+3.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	2.2481ms	0.5069ms	1.9729 KOps/s	2.3109 KOps/s	$\textbf{\color{#d91a1a}-14.63\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7165ms	0.4747ms	2.1064 KOps/s	2.3782 KOps/s	$\textbf{\color{#d91a1a}-11.43\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.5339s	15.6513ms	63.8923 Ops/s	193.9112 Ops/s	$\textbf{\color{#d91a1a}-67.05\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	10.5463ms	2.0769ms	481.4845 Ops/s	436.5917 Ops/s	$\textbf{\color{#35bf28}+10.28\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	5.7391ms	1.1694ms	855.1719 Ops/s	861.2118 Ops/s	$\color{#d91a1a}-0.70\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	6.6565ms	5.0169ms	199.3257 Ops/s	55.4351 Ops/s	$\textbf{\color{#35bf28}+259.57\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	5.5850ms	1.9713ms	507.2690 Ops/s	499.9463 Ops/s	$\color{#35bf28}+1.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	10.0649ms	1.2111ms	825.6684 Ops/s	799.3587 Ops/s	$\color{#35bf28}+3.29\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	8.5397ms	5.2831ms	189.2839 Ops/s	184.8745 Ops/s	$\color{#35bf28}+2.39\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	8.4604ms	2.1835ms	457.9799 Ops/s	452.7744 Ops/s	$\color{#35bf28}+1.15\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.9446ms	1.0766ms	928.8381 Ops/s	978.9612 Ops/s	$\textbf{\color{#d91a1a}-5.12\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	34.8860ms	32.9635ms	30.3366 Ops/s	29.8586 Ops/s	$\color{#35bf28}+1.60\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.1963ms	17.4859ms	57.1888 Ops/s	57.1964 Ops/s	$\color{#d91a1a}-0.01\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	35.8750ms	33.8097ms	29.5774 Ops/s	29.0937 Ops/s	$\color{#35bf28}+1.66\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.4303ms	17.5675ms	56.9232 Ops/s	56.4702 Ops/s	$\color{#35bf28}+0.80\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	37.9040ms	35.6127ms	28.0798 Ops/s	27.8428 Ops/s	$\color{#35bf28}+0.85\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.1127ms	18.8808ms	52.9638 Ops/s	53.2971 Ops/s	$\color{#d91a1a}-0.63\%$