[CI,NOMRERG] Test @v4

[vmoens]

Stack from ghstack (oldest at bottom):

• -> [CI,NOMRERG] Test @v4 #2686

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2686

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Failures because of deprecated version of actions/download-artifact: v3

❌ 12 New Failures

As of commit 051d999 with merge base f672c70 ():

NEW FAILURES - The following jobs have failed:

• .github/workflows/build-wheels-aarch64-linux.yml (gh)
• .github/workflows/build-wheels-linux.yml (gh)
• .github/workflows/build-wheels-m1.yml (gh)
• .github/workflows/build-wheels-windows.yml (gh)
• .github/workflows/docs.yml (gh)
• .github/workflows/lint.yml (gh)
• .github/workflows/test-linux-habitat.yml (gh)
• .github/workflows/test-linux-libs.yml (gh)
• .github/workflows/test-linux-rlhf.yml (gh)
• .github/workflows/test-linux-sota.yml (gh)
• .github/workflows/test-linux.yml (gh)
• .github/workflows/test-windows-optdepts.yml (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}17$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_simple	0.5490s	0.4535s	2.2051 Ops/s	2.1773 Ops/s	$\color{#35bf28}+1.28\%$
test_transformed	0.7334s	0.6368s	1.5704 Ops/s	1.5706 Ops/s	$\color{#d91a1a}-0.01\%$
test_serial	1.4954s	1.4044s	0.7121 Ops/s	0.7221 Ops/s	$\color{#d91a1a}-1.39\%$
test_parallel	1.3193s	1.2221s	0.8183 Ops/s	0.8152 Ops/s	$\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-True-True-True-True]	0.2818ms	30.2755μs	33.0300 KOps/s	32.4943 KOps/s	$\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-True-True-True-False]	56.0040μs	17.7757μs	56.2567 KOps/s	55.3786 KOps/s	$\color{#35bf28}+1.59\%$
test_step_mdp_speed[True-True-True-False-True]	71.7460μs	16.9102μs	59.1358 KOps/s	58.4648 KOps/s	$\color{#35bf28}+1.15\%$
test_step_mdp_speed[True-True-True-False-False]	48.8220μs	9.8977μs	101.0331 KOps/s	98.7910 KOps/s	$\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-True-False-True-True]	75.3510μs	31.9882μs	31.2615 KOps/s	30.7006 KOps/s	$\color{#35bf28}+1.83\%$
test_step_mdp_speed[True-True-False-True-False]	70.7530μs	19.6334μs	50.9335 KOps/s	50.0151 KOps/s	$\color{#35bf28}+1.84\%$
test_step_mdp_speed[True-True-False-False-True]	59.7420μs	19.0607μs	52.4640 KOps/s	52.2442 KOps/s	$\color{#35bf28}+0.42\%$
test_step_mdp_speed[True-True-False-False-False]	58.7400μs	11.8110μs	84.6667 KOps/s	82.8891 KOps/s	$\color{#35bf28}+2.14\%$
test_step_mdp_speed[True-False-True-True-True]	87.1330μs	33.8627μs	29.5310 KOps/s	28.7091 KOps/s	$\color{#35bf28}+2.86\%$
test_step_mdp_speed[True-False-True-True-False]	72.1450μs	21.4371μs	46.6482 KOps/s	45.8250 KOps/s	$\color{#35bf28}+1.80\%$
test_step_mdp_speed[True-False-True-False-True]	60.8740μs	18.8183μs	53.1398 KOps/s	52.4657 KOps/s	$\color{#35bf28}+1.28\%$
test_step_mdp_speed[True-False-True-False-False]	49.4820μs	11.7675μs	84.9800 KOps/s	83.1668 KOps/s	$\color{#35bf28}+2.18\%$
test_step_mdp_speed[True-False-False-True-True]	86.3620μs	35.8254μs	27.9132 KOps/s	27.8204 KOps/s	$\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-False-False-True-False]	66.5250μs	23.1079μs	43.2753 KOps/s	42.1159 KOps/s	$\color{#35bf28}+2.75\%$
test_step_mdp_speed[True-False-False-False-True]	0.5887ms	20.6104μs	48.5192 KOps/s	48.2150 KOps/s	$\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-False-False-False-False]	53.7010μs	13.5703μs	73.6903 KOps/s	71.7539 KOps/s	$\color{#35bf28}+2.70\%$
test_step_mdp_speed[False-True-True-True-True]	84.5990μs	34.1378μs	29.2930 KOps/s	29.0566 KOps/s	$\color{#35bf28}+0.81\%$
test_step_mdp_speed[False-True-True-True-False]	58.5400μs	21.4501μs	46.6199 KOps/s	45.7299 KOps/s	$\color{#35bf28}+1.95\%$
test_step_mdp_speed[False-True-True-False-True]	67.6970μs	21.8476μs	45.7717 KOps/s	45.3458 KOps/s	$\color{#35bf28}+0.94\%$
test_step_mdp_speed[False-True-True-False-False]	51.3070μs	13.3935μs	74.6630 KOps/s	72.9147 KOps/s	$\color{#35bf28}+2.40\%$
test_step_mdp_speed[False-True-False-True-True]	76.4230μs	35.8289μs	27.9105 KOps/s	27.6827 KOps/s	$\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-True-False-True-False]	63.6800μs	23.1654μs	43.1678 KOps/s	41.9187 KOps/s	$\color{#35bf28}+2.98\%$
test_step_mdp_speed[False-True-False-False-True]	2.6582ms	23.6033μs	42.3669 KOps/s	41.5813 KOps/s	$\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-True-False-False-False]	59.4210μs	15.0654μs	66.3771 KOps/s	60.4978 KOps/s	$\textbf{\color{#35bf28}+9.72\%}$
test_step_mdp_speed[False-False-True-True-True]	0.1215ms	37.6500μs	26.5605 KOps/s	26.3789 KOps/s	$\color{#35bf28}+0.69\%$
test_step_mdp_speed[False-False-True-True-False]	72.0150μs	25.1472μs	39.7658 KOps/s	39.0692 KOps/s	$\color{#35bf28}+1.78\%$
test_step_mdp_speed[False-False-True-False-True]	69.0000μs	23.4779μs	42.5932 KOps/s	42.0192 KOps/s	$\color{#35bf28}+1.37\%$
test_step_mdp_speed[False-False-True-False-False]	53.3100μs	15.2203μs	65.7017 KOps/s	65.2796 KOps/s	$\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-False-False-True-True]	79.5800μs	39.5696μs	25.2719 KOps/s	25.2199 KOps/s	$\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-False-False-True-False]	72.0750μs	27.1104μs	36.8862 KOps/s	36.6092 KOps/s	$\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-False-False-False-True]	63.2990μs	24.8802μs	40.1927 KOps/s	39.3370 KOps/s	$\color{#35bf28}+2.18\%$
test_step_mdp_speed[False-False-False-False-False]	66.2140μs	16.8883μs	59.2126 KOps/s	58.5160 KOps/s	$\color{#35bf28}+1.19\%$
test_values[generalized_advantage_estimate-True-True]	11.0899ms	9.9832ms	100.1684 Ops/s	98.0913 Ops/s	$\color{#35bf28}+2.12\%$
test_values[vec_generalized_advantage_estimate-True-True]	43.6524ms	34.9646ms	28.6003 Ops/s	29.4267 Ops/s	$\color{#d91a1a}-2.81\%$
test_values[td0_return_estimate-False-False]	0.2364ms	0.2026ms	4.9354 KOps/s	4.5498 KOps/s	$\textbf{\color{#35bf28}+8.47\%}$
test_values[td1_return_estimate-False-False]	27.8839ms	24.6376ms	40.5884 Ops/s	39.6248 Ops/s	$\color{#35bf28}+2.43\%$
test_values[vec_td1_return_estimate-False-False]	49.7526ms	35.9517ms	27.8151 Ops/s	28.3404 Ops/s	$\color{#d91a1a}-1.85\%$
test_values[td_lambda_return_estimate-True-False]	38.1128ms	35.0602ms	28.5223 Ops/s	27.5366 Ops/s	$\color{#35bf28}+3.58\%$
test_values[vec_td_lambda_return_estimate-True-False]	37.3336ms	35.0710ms	28.5136 Ops/s	28.0834 Ops/s	$\color{#35bf28}+1.53\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	15.7087ms	8.6908ms	115.0640 Ops/s	115.9546 Ops/s	$\color{#d91a1a}-0.77\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.5537ms	1.8901ms	529.0799 Ops/s	554.8177 Ops/s	$\color{#d91a1a}-4.64\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.6048ms	0.3613ms	2.7680 KOps/s	2.7542 KOps/s	$\color{#35bf28}+0.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	43.3912ms	41.9568ms	23.8341 Ops/s	23.0648 Ops/s	$\color{#35bf28}+3.34\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.3545ms	3.0977ms	322.8175 Ops/s	327.4707 Ops/s	$\color{#d91a1a}-1.42\%$
test_dqn_speed[False-None]	1.9074ms	1.4150ms	706.7171 Ops/s	698.4293 Ops/s	$\color{#35bf28}+1.19\%$
test_dqn_speed[False-backward]	1.9688ms	1.9012ms	525.9923 Ops/s	501.5255 Ops/s	$\color{#35bf28}+4.88\%$
test_dqn_speed[True-None]	0.8695ms	0.4801ms	2.0830 KOps/s	2.0238 KOps/s	$\color{#35bf28}+2.92\%$
test_dqn_speed[True-backward]	1.0279ms	0.9171ms	1.0904 KOps/s	987.1184 Ops/s	$\textbf{\color{#35bf28}+10.46\%}$
test_dqn_speed[reduce-overhead-None]	0.8283ms	0.4853ms	2.0604 KOps/s	2.0365 KOps/s	$\color{#35bf28}+1.18\%$
test_dqn_speed[reduce-overhead-backward]	0.9823ms	0.9099ms	1.0990 KOps/s	1.0694 KOps/s	$\color{#35bf28}+2.77\%$
test_ddpg_speed[False-None]	6.4566ms	3.1326ms	319.2274 Ops/s	342.1874 Ops/s	$\textbf{\color{#d91a1a}-6.71\%}$
test_ddpg_speed[False-backward]	4.6420ms	4.2626ms	234.5985 Ops/s	242.8553 Ops/s	$\color{#d91a1a}-3.40\%$
test_ddpg_speed[True-None]	2.0737ms	1.0326ms	968.4658 Ops/s	964.5746 Ops/s	$\color{#35bf28}+0.40\%$
test_ddpg_speed[True-backward]	2.1755ms	1.9898ms	502.5536 Ops/s	514.1178 Ops/s	$\color{#d91a1a}-2.25\%$
test_ddpg_speed[reduce-overhead-None]	1.2293ms	1.0265ms	974.1745 Ops/s	950.8589 Ops/s	$\color{#35bf28}+2.45\%$
test_ddpg_speed[reduce-overhead-backward]	2.1406ms	1.9999ms	500.0290 Ops/s	451.7015 Ops/s	$\textbf{\color{#35bf28}+10.70\%}$
test_sac_speed[False-None]	9.2877ms	8.5955ms	116.3403 Ops/s	122.1025 Ops/s	$\color{#d91a1a}-4.72\%$
test_sac_speed[False-backward]	14.0937ms	11.9673ms	83.5608 Ops/s	90.4358 Ops/s	$\textbf{\color{#d91a1a}-7.60\%}$
test_sac_speed[True-None]	2.6876ms	1.9156ms	522.0395 Ops/s	524.0972 Ops/s	$\color{#d91a1a}-0.39\%$
test_sac_speed[True-backward]	4.2273ms	4.0329ms	247.9627 Ops/s	247.9517 Ops/s	$+0.00\%$
test_sac_speed[reduce-overhead-None]	2.5199ms	1.9532ms	511.9776 Ops/s	523.0599 Ops/s	$\color{#d91a1a}-2.12\%$
test_sac_speed[reduce-overhead-backward]	4.2746ms	3.9452ms	253.4697 Ops/s	237.4127 Ops/s	$\textbf{\color{#35bf28}+6.76\%}$
test_redq_speed[False-None]	15.3766ms	13.5998ms	73.5308 Ops/s	70.8903 Ops/s	$\color{#35bf28}+3.72\%$
test_redq_speed[False-backward]	24.8117ms	23.1154ms	43.2612 Ops/s	41.7640 Ops/s	$\color{#35bf28}+3.59\%$
test_redq_speed[True-None]	6.4586ms	5.6529ms	176.8994 Ops/s	189.4360 Ops/s	$\textbf{\color{#d91a1a}-6.62\%}$
test_redq_speed[True-backward]	14.8245ms	13.1138ms	76.2557 Ops/s	75.9543 Ops/s	$\color{#35bf28}+0.40\%$
test_redq_speed[reduce-overhead-None]	6.6402ms	5.7283ms	174.5729 Ops/s	181.3542 Ops/s	$\color{#d91a1a}-3.74\%$
test_redq_speed[reduce-overhead-backward]	14.0875ms	12.8590ms	77.7665 Ops/s	77.2110 Ops/s	$\color{#35bf28}+0.72\%$
test_redq_deprec_speed[False-None]	16.0845ms	14.3207ms	69.8288 Ops/s	71.0369 Ops/s	$\color{#d91a1a}-1.70\%$
test_redq_deprec_speed[False-backward]	21.7510ms	19.9677ms	50.0808 Ops/s	49.3658 Ops/s	$\color{#35bf28}+1.45\%$
test_redq_deprec_speed[True-None]	4.8205ms	4.3988ms	227.3357 Ops/s	245.1433 Ops/s	$\textbf{\color{#d91a1a}-7.26\%}$
test_redq_deprec_speed[True-backward]	10.1822ms	9.2087ms	108.5924 Ops/s	109.2792 Ops/s	$\color{#d91a1a}-0.63\%$
test_redq_deprec_speed[reduce-overhead-None]	4.8244ms	4.3113ms	231.9491 Ops/s	224.1221 Ops/s	$\color{#35bf28}+3.49\%$
test_redq_deprec_speed[reduce-overhead-backward]	10.3906ms	9.2091ms	108.5881 Ops/s	108.5924 Ops/s	$-0.00\%$
test_td3_speed[False-None]	9.2694ms	8.7003ms	114.9381 Ops/s	112.6463 Ops/s	$\color{#35bf28}+2.03\%$
test_td3_speed[False-backward]	13.5260ms	11.5168ms	86.8297 Ops/s	89.1247 Ops/s	$\color{#d91a1a}-2.57\%$
test_td3_speed[True-None]	1.9619ms	1.8019ms	554.9615 Ops/s	544.5507 Ops/s	$\color{#35bf28}+1.91\%$
test_td3_speed[True-backward]	4.2014ms	3.7932ms	263.6308 Ops/s	275.0503 Ops/s	$\color{#d91a1a}-4.15\%$
test_td3_speed[reduce-overhead-None]	3.1750ms	1.8896ms	529.2166 Ops/s	535.2247 Ops/s	$\color{#d91a1a}-1.12\%$
test_td3_speed[reduce-overhead-backward]	4.1261ms	3.8644ms	258.7693 Ops/s	259.1396 Ops/s	$\color{#d91a1a}-0.14\%$
test_cql_speed[False-None]	41.6411ms	37.9845ms	26.3265 Ops/s	25.7802 Ops/s	$\color{#35bf28}+2.12\%$
test_cql_speed[False-backward]	52.2464ms	49.4996ms	20.2022 Ops/s	20.2816 Ops/s	$\color{#d91a1a}-0.39\%$
test_cql_speed[True-None]	17.6191ms	16.3127ms	61.3021 Ops/s	59.2941 Ops/s	$\color{#35bf28}+3.39\%$
test_cql_speed[True-backward]	43.6233ms	24.5758ms	40.6905 Ops/s	40.7095 Ops/s	$\color{#d91a1a}-0.05\%$
test_cql_speed[reduce-overhead-None]	17.4457ms	16.5488ms	60.4275 Ops/s	60.0971 Ops/s	$\color{#35bf28}+0.55\%$
test_cql_speed[reduce-overhead-backward]	24.3967ms	23.5139ms	42.5280 Ops/s	41.8724 Ops/s	$\color{#35bf28}+1.57\%$
test_a2c_speed[False-None]	9.2851ms	8.0267ms	124.5848 Ops/s	120.4526 Ops/s	$\color{#35bf28}+3.43\%$
test_a2c_speed[False-backward]	15.7563ms	15.4322ms	64.7997 Ops/s	62.8876 Ops/s	$\color{#35bf28}+3.04\%$
test_a2c_speed[True-None]	5.2942ms	4.6878ms	213.3196 Ops/s	217.9106 Ops/s	$\color{#d91a1a}-2.11\%$
test_a2c_speed[True-backward]	12.1603ms	11.8289ms	84.5385 Ops/s	87.5145 Ops/s	$\color{#d91a1a}-3.40\%$
test_a2c_speed[reduce-overhead-None]	5.5730ms	4.7758ms	209.3881 Ops/s	225.5802 Ops/s	$\textbf{\color{#d91a1a}-7.18\%}$
test_a2c_speed[reduce-overhead-backward]	12.8409ms	11.6354ms	85.9444 Ops/s	87.3917 Ops/s	$\color{#d91a1a}-1.66\%$
test_ppo_speed[False-None]	9.1899ms	8.3442ms	119.8430 Ops/s	119.8235 Ops/s	$\color{#35bf28}+0.02\%$
test_ppo_speed[False-backward]	17.1272ms	15.8326ms	63.1609 Ops/s	62.9044 Ops/s	$\color{#35bf28}+0.41\%$
test_ppo_speed[True-None]	4.3902ms	4.0178ms	248.8953 Ops/s	244.9309 Ops/s	$\color{#35bf28}+1.62\%$
test_ppo_speed[True-backward]	10.6895ms	10.4604ms	95.5990 Ops/s	95.0579 Ops/s	$\color{#35bf28}+0.57\%$
test_ppo_speed[reduce-overhead-None]	9.7383ms	4.4017ms	227.1857 Ops/s	252.1707 Ops/s	$\textbf{\color{#d91a1a}-9.91\%}$
test_ppo_speed[reduce-overhead-backward]	10.6635ms	10.3702ms	96.4303 Ops/s	97.2644 Ops/s	$\color{#d91a1a}-0.86\%$
test_reinforce_speed[False-None]	8.2652ms	6.9085ms	144.7499 Ops/s	144.2518 Ops/s	$\color{#35bf28}+0.35\%$
test_reinforce_speed[False-backward]	20.2456ms	10.7172ms	93.3083 Ops/s	97.5407 Ops/s	$\color{#d91a1a}-4.34\%$
test_reinforce_speed[True-None]	4.3577ms	3.0012ms	333.1968 Ops/s	356.1008 Ops/s	$\textbf{\color{#d91a1a}-6.43\%}$
test_reinforce_speed[True-backward]	11.6805ms	9.5577ms	104.6279 Ops/s	109.5984 Ops/s	$\color{#d91a1a}-4.54\%$
test_reinforce_speed[reduce-overhead-None]	4.6076ms	3.1244ms	320.0598 Ops/s	360.0070 Ops/s	$\textbf{\color{#d91a1a}-11.10\%}$
test_reinforce_speed[reduce-overhead-backward]	9.8255ms	9.4046ms	106.3309 Ops/s	110.8334 Ops/s	$\color{#d91a1a}-4.06\%$
test_iql_speed[False-None]	40.9623ms	34.5874ms	28.9122 Ops/s	29.8156 Ops/s	$\color{#d91a1a}-3.03\%$
test_iql_speed[False-backward]	57.6924ms	48.2375ms	20.7308 Ops/s	21.2997 Ops/s	$\color{#d91a1a}-2.67\%$
test_iql_speed[True-None]	12.3561ms	11.3984ms	87.7319 Ops/s	87.6224 Ops/s	$\color{#35bf28}+0.12\%$
test_iql_speed[True-backward]	24.3367ms	22.9437ms	43.5850 Ops/s	43.7596 Ops/s	$\color{#d91a1a}-0.40\%$
test_iql_speed[reduce-overhead-None]	12.0485ms	11.3323ms	88.2431 Ops/s	88.1465 Ops/s	$\color{#35bf28}+0.11\%$
test_iql_speed[reduce-overhead-backward]	24.0254ms	23.0341ms	43.4139 Ops/s	44.7014 Ops/s	$\color{#d91a1a}-2.88\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.0134ms	5.5968ms	178.6720 Ops/s	188.3143 Ops/s	$\textbf{\color{#d91a1a}-5.12\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8652ms	0.5469ms	1.8284 KOps/s	666.5816 Ops/s	$\textbf{\color{#35bf28}+174.29\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.9512ms	0.5237ms	1.9095 KOps/s	1.9583 KOps/s	$\color{#d91a1a}-2.49\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	9.6177ms	5.2998ms	188.6881 Ops/s	200.4495 Ops/s	$\textbf{\color{#d91a1a}-5.87\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.6754ms	0.5354ms	1.8679 KOps/s	1.9512 KOps/s	$\color{#d91a1a}-4.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.8198ms	0.5119ms	1.9534 KOps/s	1.9343 KOps/s	$\color{#35bf28}+0.99\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.5021ms	1.6804ms	595.1021 Ops/s	579.8592 Ops/s	$\color{#35bf28}+2.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.3404ms	1.6005ms	624.7863 Ops/s	628.4617 Ops/s	$\color{#d91a1a}-0.58\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.4718ms	5.5303ms	180.8214 Ops/s	194.3884 Ops/s	$\textbf{\color{#d91a1a}-6.98\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	4.3771ms	0.6797ms	1.4713 KOps/s	1.4974 KOps/s	$\color{#d91a1a}-1.74\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8971ms	0.6474ms	1.5446 KOps/s	1.5717 KOps/s	$\color{#d91a1a}-1.72\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.8200ms	5.2130ms	191.8296 Ops/s	196.3908 Ops/s	$\color{#d91a1a}-2.32\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.4861ms	0.5494ms	1.8201 KOps/s	1.8398 KOps/s	$\color{#d91a1a}-1.07\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7621ms	0.5203ms	1.9220 KOps/s	1.9616 KOps/s	$\color{#d91a1a}-2.02\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9608ms	5.1812ms	193.0039 Ops/s	202.4352 Ops/s	$\color{#d91a1a}-4.66\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.3022ms	0.5322ms	1.8789 KOps/s	1.9444 KOps/s	$\color{#d91a1a}-3.37\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.8651ms	0.5119ms	1.9536 KOps/s	1.9992 KOps/s	$\color{#d91a1a}-2.28\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.8648ms	5.4491ms	183.5177 Ops/s	195.0614 Ops/s	$\textbf{\color{#d91a1a}-5.92\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.2311ms	0.6772ms	1.4767 KOps/s	1.5251 KOps/s	$\color{#d91a1a}-3.17\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9621ms	0.6550ms	1.5267 KOps/s	1.5686 KOps/s	$\color{#d91a1a}-2.67\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.5185s	14.9342ms	66.9604 Ops/s	245.3974 Ops/s	$\textbf{\color{#d91a1a}-72.71\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	8.9279ms	2.4126ms	414.4865 Ops/s	427.3130 Ops/s	$\color{#d91a1a}-3.00\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	4.7715ms	1.3449ms	743.5678 Ops/s	733.1082 Ops/s	$\color{#35bf28}+1.43\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	7.0446ms	4.7212ms	211.8115 Ops/s	251.6087 Ops/s	$\textbf{\color{#d91a1a}-15.82\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	8.7689ms	2.3780ms	420.5262 Ops/s	447.5188 Ops/s	$\textbf{\color{#d91a1a}-6.03\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.7542ms	1.3417ms	745.3088 Ops/s	628.9406 Ops/s	$\textbf{\color{#35bf28}+18.50\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.4565s	13.8688ms	72.1044 Ops/s	241.5006 Ops/s	$\textbf{\color{#d91a1a}-70.14\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	7.5777ms	2.5116ms	398.1474 Ops/s	420.9915 Ops/s	$\textbf{\color{#d91a1a}-5.43\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.0203ms	1.5145ms	660.2991 Ops/s	684.8632 Ops/s	$\color{#d91a1a}-3.59\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	14.8732ms	13.5042ms	74.0511 Ops/s	74.2735 Ops/s	$\color{#d91a1a}-0.30\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	16.6887ms	15.2167ms	65.7172 Ops/s	66.9717 Ops/s	$\color{#d91a1a}-1.87\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	23.4755ms	22.3621ms	44.7185 Ops/s	45.1925 Ops/s	$\color{#d91a1a}-1.05\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	17.1602ms	15.4400ms	64.7667 Ops/s	63.3741 Ops/s	$\color{#35bf28}+2.20\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	22.7216ms	22.1621ms	45.1220 Ops/s	44.8151 Ops/s	$\color{#35bf28}+0.68\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	18.2170ms	16.7480ms	59.7085 Ops/s	60.8228 Ops/s	$\color{#d91a1a}-1.83\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}22$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_simple	0.6992s	0.6985s	1.4316 Ops/s	1.3908 Ops/s	$\color{#35bf28}+2.94\%$
test_transformed	0.9464s	0.9447s	1.0585 Ops/s	1.0373 Ops/s	$\color{#35bf28}+2.04\%$
test_serial	2.1667s	2.0815s	0.4804 Ops/s	0.4814 Ops/s	$\color{#d91a1a}-0.20\%$
test_parallel	1.9280s	1.8475s	0.5413 Ops/s	0.5447 Ops/s	$\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-True-True-True-True]	0.2310ms	38.1422μs	26.2177 KOps/s	26.8026 KOps/s	$\color{#d91a1a}-2.18\%$
test_step_mdp_speed[True-True-True-True-False]	54.3410μs	22.1274μs	45.1928 KOps/s	45.6453 KOps/s	$\color{#d91a1a}-0.99\%$
test_step_mdp_speed[True-True-True-False-True]	51.6110μs	21.3795μs	46.7737 KOps/s	47.6159 KOps/s	$\color{#d91a1a}-1.77\%$
test_step_mdp_speed[True-True-True-False-False]	53.0110μs	12.3274μs	81.1201 KOps/s	80.9040 KOps/s	$\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-True-False-True-True]	69.5410μs	40.8214μs	24.4969 KOps/s	24.3967 KOps/s	$\color{#35bf28}+0.41\%$
test_step_mdp_speed[True-True-False-True-False]	55.1210μs	24.5964μs	40.6564 KOps/s	40.9934 KOps/s	$\color{#d91a1a}-0.82\%$
test_step_mdp_speed[True-True-False-False-True]	51.2710μs	23.3928μs	42.7481 KOps/s	42.3103 KOps/s	$\color{#35bf28}+1.03\%$
test_step_mdp_speed[True-True-False-False-False]	44.7310μs	14.7346μs	67.8675 KOps/s	69.3794 KOps/s	$\color{#d91a1a}-2.18\%$
test_step_mdp_speed[True-False-True-True-True]	74.1910μs	43.0579μs	23.2245 KOps/s	23.3368 KOps/s	$\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-False-True-True-False]	59.7310μs	26.5889μs	37.6096 KOps/s	36.9896 KOps/s	$\color{#35bf28}+1.68\%$
test_step_mdp_speed[True-False-True-False-True]	54.3910μs	23.7235μs	42.1524 KOps/s	43.8539 KOps/s	$\color{#d91a1a}-3.88\%$
test_step_mdp_speed[True-False-True-False-False]	45.1700μs	14.5763μs	68.6044 KOps/s	67.9901 KOps/s	$\color{#35bf28}+0.90\%$
test_step_mdp_speed[True-False-False-True-True]	80.8310μs	45.0708μs	22.1873 KOps/s	22.3755 KOps/s	$\color{#d91a1a}-0.84\%$
test_step_mdp_speed[True-False-False-True-False]	64.4700μs	28.5586μs	35.0157 KOps/s	34.7332 KOps/s	$\color{#35bf28}+0.81\%$
test_step_mdp_speed[True-False-False-False-True]	54.0910μs	25.4508μs	39.2915 KOps/s	39.7999 KOps/s	$\color{#d91a1a}-1.28\%$
test_step_mdp_speed[True-False-False-False-False]	49.3700μs	16.4031μs	60.9641 KOps/s	59.2108 KOps/s	$\color{#35bf28}+2.96\%$
test_step_mdp_speed[False-True-True-True-True]	84.3520μs	42.5962μs	23.4763 KOps/s	24.1126 KOps/s	$\color{#d91a1a}-2.64\%$
test_step_mdp_speed[False-True-True-True-False]	57.9110μs	26.1216μs	38.2825 KOps/s	38.6317 KOps/s	$\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-True-True-False-True]	60.1910μs	26.9372μs	37.1234 KOps/s	37.7024 KOps/s	$\color{#d91a1a}-1.54\%$
test_step_mdp_speed[False-True-True-False-False]	44.4210μs	16.1885μs	61.7724 KOps/s	64.0310 KOps/s	$\color{#d91a1a}-3.53\%$
test_step_mdp_speed[False-True-False-True-True]	80.2610μs	45.3734μs	22.0394 KOps/s	22.9982 KOps/s	$\color{#d91a1a}-4.17\%$
test_step_mdp_speed[False-True-False-True-False]	57.8110μs	29.2052μs	34.2405 KOps/s	34.9079 KOps/s	$\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-True-False-False-True]	3.2659ms	29.9766μs	33.3594 KOps/s	33.5513 KOps/s	$\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-True-False-False-False]	51.6210μs	18.4853μs	54.0972 KOps/s	53.6131 KOps/s	$\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-False-True-True-True]	74.0010μs	47.0855μs	21.2380 KOps/s	21.2055 KOps/s	$\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-False-True-True-False]	67.9410μs	30.8268μs	32.4393 KOps/s	32.0354 KOps/s	$\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-False-True-False-True]	60.6710μs	29.6339μs	33.7452 KOps/s	34.7432 KOps/s	$\color{#d91a1a}-2.87\%$
test_step_mdp_speed[False-False-True-False-False]	47.8710μs	18.7563μs	53.3155 KOps/s	56.9297 KOps/s	$\textbf{\color{#d91a1a}-6.35\%}$
test_step_mdp_speed[False-False-False-True-True]	80.3510μs	49.6748μs	20.1309 KOps/s	20.4303 KOps/s	$\color{#d91a1a}-1.47\%$
test_step_mdp_speed[False-False-False-True-False]	66.6910μs	33.1590μs	30.1577 KOps/s	30.0482 KOps/s	$\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-False-True]	59.2310μs	30.9026μs	32.3598 KOps/s	32.3796 KOps/s	$\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-False-False-False-False]	51.6600μs	20.3733μs	49.0840 KOps/s	48.4982 KOps/s	$\color{#35bf28}+1.21\%$
test_values[generalized_advantage_estimate-True-True]	25.1819ms	24.5715ms	40.6975 Ops/s	40.1740 Ops/s	$\color{#35bf28}+1.30\%$
test_values[vec_generalized_advantage_estimate-True-True]	0.1012s	2.9176ms	342.7454 Ops/s	354.7170 Ops/s	$\color{#d91a1a}-3.37\%$
test_values[td0_return_estimate-False-False]	0.1039ms	78.9108μs	12.6725 KOps/s	12.6776 KOps/s	$\color{#d91a1a}-0.04\%$
test_values[td1_return_estimate-False-False]	55.9353ms	54.7062ms	18.2795 Ops/s	17.7737 Ops/s	$\color{#35bf28}+2.85\%$
test_values[vec_td1_return_estimate-False-False]	1.3642ms	1.0765ms	928.9411 Ops/s	927.9410 Ops/s	$\color{#35bf28}+0.11\%$
test_values[td_lambda_return_estimate-True-False]	91.2675ms	87.4780ms	11.4315 Ops/s	11.0831 Ops/s	$\color{#35bf28}+3.14\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.3826ms	1.0712ms	933.5326 Ops/s	928.9010 Ops/s	$\color{#35bf28}+0.50\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	25.5991ms	24.3727ms	41.0295 Ops/s	41.0063 Ops/s	$\color{#35bf28}+0.06\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.0211ms	0.7426ms	1.3467 KOps/s	1.3440 KOps/s	$\color{#35bf28}+0.20\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7663ms	0.6640ms	1.5060 KOps/s	1.5019 KOps/s	$\color{#35bf28}+0.27\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5171ms	1.4698ms	680.3490 Ops/s	677.3657 Ops/s	$\color{#35bf28}+0.44\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.7235ms	0.6792ms	1.4724 KOps/s	1.4692 KOps/s	$\color{#35bf28}+0.21\%$
test_dqn_speed[False-None]	1.6975ms	1.4851ms	673.3466 Ops/s	682.0454 Ops/s	$\color{#d91a1a}-1.28\%$
test_dqn_speed[False-backward]	2.1350ms	2.0819ms	480.3263 Ops/s	484.9782 Ops/s	$\color{#d91a1a}-0.96\%$
test_dqn_speed[True-None]	0.6008ms	0.5344ms	1.8711 KOps/s	1.8322 KOps/s	$\color{#35bf28}+2.13\%$
test_dqn_speed[True-backward]	1.1572ms	1.0825ms	923.7671 Ops/s	829.7423 Ops/s	$\textbf{\color{#35bf28}+11.33\%}$
test_dqn_speed[reduce-overhead-None]	0.6124ms	0.5534ms	1.8070 KOps/s	1.7796 KOps/s	$\color{#35bf28}+1.54\%$
test_dqn_speed[reduce-overhead-backward]	0.9702ms	0.9331ms	1.0717 KOps/s	938.7859 Ops/s	$\textbf{\color{#35bf28}+14.16\%}$
test_ddpg_speed[False-None]	3.1167ms	2.8159ms	355.1219 Ops/s	358.0406 Ops/s	$\color{#d91a1a}-0.82\%$
test_ddpg_speed[False-backward]	4.5272ms	4.0825ms	244.9495 Ops/s	240.4532 Ops/s	$\color{#35bf28}+1.87\%$
test_ddpg_speed[True-None]	1.1600ms	1.0577ms	945.4436 Ops/s	900.8113 Ops/s	$\color{#35bf28}+4.95\%$
test_ddpg_speed[True-backward]	2.1822ms	2.0912ms	478.1880 Ops/s	435.2892 Ops/s	$\textbf{\color{#35bf28}+9.86\%}$
test_ddpg_speed[reduce-overhead-None]	1.1572ms	1.0681ms	936.2624 Ops/s	916.6801 Ops/s	$\color{#35bf28}+2.14\%$
test_ddpg_speed[reduce-overhead-backward]	1.6357ms	1.5912ms	628.4391 Ops/s	561.4132 Ops/s	$\textbf{\color{#35bf28}+11.94\%}$
test_sac_speed[False-None]	8.3586ms	7.9346ms	126.0300 Ops/s	126.3663 Ops/s	$\color{#d91a1a}-0.27\%$
test_sac_speed[False-backward]	11.3192ms	10.8092ms	92.5137 Ops/s	90.7442 Ops/s	$\color{#35bf28}+1.95\%$
test_sac_speed[True-None]	1.5428ms	1.4930ms	669.8082 Ops/s	627.4888 Ops/s	$\textbf{\color{#35bf28}+6.74\%}$
test_sac_speed[True-backward]	3.2537ms	3.1493ms	317.5275 Ops/s	296.5754 Ops/s	$\textbf{\color{#35bf28}+7.06\%}$
test_sac_speed[reduce-overhead-None]	22.2010ms	12.3854ms	80.7403 Ops/s	79.6866 Ops/s	$\color{#35bf28}+1.32\%$
test_sac_speed[reduce-overhead-backward]	1.3797ms	1.3140ms	761.0588 Ops/s	736.7991 Ops/s	$\color{#35bf28}+3.29\%$
test_redq_speed[False-None]	8.1222ms	7.4044ms	135.0546 Ops/s	134.3991 Ops/s	$\color{#35bf28}+0.49\%$
test_redq_speed[False-backward]	12.0974ms	11.1485ms	89.6982 Ops/s	89.0591 Ops/s	$\color{#35bf28}+0.72\%$
test_redq_speed[True-None]	1.9823ms	1.9319ms	517.6261 Ops/s	509.5557 Ops/s	$\color{#35bf28}+1.58\%$
test_redq_speed[True-backward]	3.6175ms	3.5239ms	283.7744 Ops/s	264.0639 Ops/s	$\textbf{\color{#35bf28}+7.46\%}$
test_redq_speed[reduce-overhead-None]	2.0216ms	1.9452ms	514.0961 Ops/s	494.2823 Ops/s	$\color{#35bf28}+4.01\%$
test_redq_speed[reduce-overhead-backward]	3.6335ms	3.5590ms	280.9814 Ops/s	263.8519 Ops/s	$\textbf{\color{#35bf28}+6.49\%}$
test_redq_deprec_speed[False-None]	9.3466ms	8.9221ms	112.0809 Ops/s	111.2050 Ops/s	$\color{#35bf28}+0.79\%$
test_redq_deprec_speed[False-backward]	12.3292ms	11.8705ms	84.2424 Ops/s	81.7790 Ops/s	$\color{#35bf28}+3.01\%$
test_redq_deprec_speed[True-None]	2.3591ms	2.3035ms	434.1193 Ops/s	428.1359 Ops/s	$\color{#35bf28}+1.40\%$
test_redq_deprec_speed[True-backward]	3.9715ms	3.8891ms	257.1317 Ops/s	244.9820 Ops/s	$\color{#35bf28}+4.96\%$
test_redq_deprec_speed[reduce-overhead-None]	2.4035ms	2.2827ms	438.0738 Ops/s	432.1352 Ops/s	$\color{#35bf28}+1.37\%$
test_redq_deprec_speed[reduce-overhead-backward]	4.3016ms	3.9038ms	256.1599 Ops/s	243.5284 Ops/s	$\textbf{\color{#35bf28}+5.19\%}$
test_td3_speed[False-None]	8.1191ms	7.8340ms	127.6487 Ops/s	128.0732 Ops/s	$\color{#d91a1a}-0.33\%$
test_td3_speed[False-backward]	10.4973ms	10.0660ms	99.3445 Ops/s	96.8646 Ops/s	$\color{#35bf28}+2.56\%$
test_td3_speed[True-None]	1.5866ms	1.5616ms	640.3733 Ops/s	630.7042 Ops/s	$\color{#35bf28}+1.53\%$
test_td3_speed[True-backward]	3.1037ms	3.0453ms	328.3790 Ops/s	319.0149 Ops/s	$\color{#35bf28}+2.94\%$
test_td3_speed[reduce-overhead-None]	48.7896ms	24.9514ms	40.0780 Ops/s	39.8103 Ops/s	$\color{#35bf28}+0.67\%$
test_td3_speed[reduce-overhead-backward]	1.3457ms	1.2734ms	785.3256 Ops/s	778.4750 Ops/s	$\color{#35bf28}+0.88\%$
test_cql_speed[False-None]	17.0777ms	16.5516ms	60.4173 Ops/s	60.2561 Ops/s	$\color{#35bf28}+0.27\%$
test_cql_speed[False-backward]	22.3440ms	21.6322ms	46.2275 Ops/s	45.9170 Ops/s	$\color{#35bf28}+0.68\%$
test_cql_speed[True-None]	2.9032ms	2.8594ms	349.7275 Ops/s	343.9967 Ops/s	$\color{#35bf28}+1.67\%$
test_cql_speed[True-backward]	5.2664ms	5.0289ms	198.8501 Ops/s	193.7145 Ops/s	$\color{#35bf28}+2.65\%$
test_cql_speed[reduce-overhead-None]	0.3590s	14.6558ms	68.2325 Ops/s	76.1097 Ops/s	$\textbf{\color{#d91a1a}-10.35\%}$
test_cql_speed[reduce-overhead-backward]	1.5614ms	1.5014ms	666.0487 Ops/s	587.9584 Ops/s	$\textbf{\color{#35bf28}+13.28\%}$
test_a2c_speed[False-None]	3.2591ms	3.1690ms	315.5589 Ops/s	309.0357 Ops/s	$\color{#35bf28}+2.11\%$
test_a2c_speed[False-backward]	6.7101ms	6.0468ms	165.3776 Ops/s	156.9378 Ops/s	$\textbf{\color{#35bf28}+5.38\%}$
test_a2c_speed[True-None]	1.0791ms	0.9896ms	1.0105 KOps/s	988.9371 Ops/s	$\color{#35bf28}+2.18\%$
test_a2c_speed[True-backward]	2.5799ms	2.4994ms	400.0982 Ops/s	389.0158 Ops/s	$\color{#35bf28}+2.85\%$
test_a2c_speed[reduce-overhead-None]	20.5522ms	11.1455ms	89.7227 Ops/s	89.4842 Ops/s	$\color{#35bf28}+0.27\%$
test_a2c_speed[reduce-overhead-backward]	0.9916ms	0.9526ms	1.0497 KOps/s	876.6777 Ops/s	$\textbf{\color{#35bf28}+19.74\%}$
test_ppo_speed[False-None]	3.7399ms	3.6402ms	274.7139 Ops/s	274.3704 Ops/s	$\color{#35bf28}+0.13\%$
test_ppo_speed[False-backward]	7.1581ms	6.7227ms	148.7488 Ops/s	142.4868 Ops/s	$\color{#35bf28}+4.39\%$
test_ppo_speed[True-None]	1.0483ms	0.9376ms	1.0666 KOps/s	1.0416 KOps/s	$\color{#35bf28}+2.40\%$
test_ppo_speed[True-backward]	2.5625ms	2.4910ms	401.4510 Ops/s	400.4511 Ops/s	$\color{#35bf28}+0.25\%$
test_ppo_speed[reduce-overhead-None]	0.5665ms	0.5183ms	1.9293 KOps/s	69.9518 Ops/s	$\textbf{\color{#35bf28}+2658.06\%}$
test_ppo_speed[reduce-overhead-backward]	1.0032ms	0.9423ms	1.0612 KOps/s	996.3880 Ops/s	$\textbf{\color{#35bf28}+6.51\%}$
test_reinforce_speed[False-None]	2.3676ms	2.2395ms	446.5379 Ops/s	449.4679 Ops/s	$\color{#d91a1a}-0.65\%$
test_reinforce_speed[False-backward]	3.6791ms	3.2340ms	309.2145 Ops/s	310.4510 Ops/s	$\color{#d91a1a}-0.40\%$
test_reinforce_speed[True-None]	0.8896ms	0.8120ms	1.2315 KOps/s	1.1478 KOps/s	$\textbf{\color{#35bf28}+7.29\%}$
test_reinforce_speed[True-backward]	2.4392ms	2.3489ms	425.7310 Ops/s	415.0107 Ops/s	$\color{#35bf28}+2.58\%$
test_reinforce_speed[reduce-overhead-None]	0.2937s	11.7526ms	85.0877 Ops/s	93.0136 Ops/s	$\textbf{\color{#d91a1a}-8.52\%}$
test_reinforce_speed[reduce-overhead-backward]	1.0612ms	1.0138ms	986.3522 Ops/s	943.8642 Ops/s	$\color{#35bf28}+4.50\%$
test_iql_speed[False-None]	10.1593ms	9.2946ms	107.5895 Ops/s	109.3238 Ops/s	$\color{#d91a1a}-1.59\%$
test_iql_speed[False-backward]	13.5316ms	12.9220ms	77.3873 Ops/s	77.6809 Ops/s	$\color{#d91a1a}-0.38\%$
test_iql_speed[True-None]	1.7932ms	1.7304ms	577.8925 Ops/s	573.0355 Ops/s	$\color{#35bf28}+0.85\%$
test_iql_speed[True-backward]	4.4554ms	4.3113ms	231.9490 Ops/s	229.8969 Ops/s	$\color{#35bf28}+0.89\%$
test_iql_speed[reduce-overhead-None]	19.7536ms	11.1602ms	89.6043 Ops/s	70.5586 Ops/s	$\textbf{\color{#35bf28}+26.99\%}$
test_iql_speed[reduce-overhead-backward]	1.6397ms	1.5688ms	637.4375 Ops/s	622.2715 Ops/s	$\color{#35bf28}+2.44\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.7657ms	6.1954ms	161.4111 Ops/s	159.1675 Ops/s	$\color{#35bf28}+1.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5090ms	0.2728ms	3.6654 KOps/s	3.6292 KOps/s	$\color{#35bf28}+1.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.4318ms	0.2628ms	3.8045 KOps/s	3.9580 KOps/s	$\color{#d91a1a}-3.88\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.2045ms	5.9293ms	168.6553 Ops/s	166.8643 Ops/s	$\color{#35bf28}+1.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.0957ms	0.2912ms	3.4346 KOps/s	3.8220 KOps/s	$\textbf{\color{#d91a1a}-10.14\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5045ms	0.2875ms	3.4788 KOps/s	2.8193 KOps/s	$\textbf{\color{#35bf28}+23.39\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.6344ms	1.4465ms	691.3473 Ops/s	806.8459 Ops/s	$\textbf{\color{#d91a1a}-14.31\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.9653ms	1.3475ms	742.1206 Ops/s	880.1377 Ops/s	$\textbf{\color{#d91a1a}-15.68\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.2530ms	6.1160ms	163.5044 Ops/s	161.1503 Ops/s	$\color{#35bf28}+1.46\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8901ms	0.4253ms	2.3511 KOps/s	2.1767 KOps/s	$\textbf{\color{#35bf28}+8.01\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6127ms	0.3913ms	2.5555 KOps/s	2.2745 KOps/s	$\textbf{\color{#35bf28}+12.36\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.0173ms	5.9093ms	169.2239 Ops/s	166.8858 Ops/s	$\color{#35bf28}+1.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7679ms	0.3495ms	2.8609 KOps/s	2.9028 KOps/s	$\color{#d91a1a}-1.44\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5194ms	0.3233ms	3.0935 KOps/s	3.4811 KOps/s	$\textbf{\color{#d91a1a}-11.13\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.1216ms	5.8740ms	170.2406 Ops/s	168.2699 Ops/s	$\color{#35bf28}+1.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.5721ms	0.3216ms	3.1095 KOps/s	3.2859 KOps/s	$\textbf{\color{#d91a1a}-5.37\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5059ms	0.2757ms	3.6277 KOps/s	3.3334 KOps/s	$\textbf{\color{#35bf28}+8.83\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.1755ms	6.0008ms	166.6431 Ops/s	162.9003 Ops/s	$\color{#35bf28}+2.30\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8182ms	0.4584ms	2.1817 KOps/s	2.3980 KOps/s	$\textbf{\color{#d91a1a}-9.02\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6529ms	0.4372ms	2.2874 KOps/s	2.2355 KOps/s	$\color{#35bf28}+2.32\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	7.0851ms	5.4092ms	184.8719 Ops/s	188.9140 Ops/s	$\color{#d91a1a}-2.14\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	8.8143ms	1.9335ms	517.1842 Ops/s	452.1710 Ops/s	$\textbf{\color{#35bf28}+14.38\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	7.9263ms	1.2056ms	829.4930 Ops/s	809.4886 Ops/s	$\color{#35bf28}+2.47\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	7.3392ms	5.3171ms	188.0717 Ops/s	188.7334 Ops/s	$\color{#d91a1a}-0.35\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	8.3804ms	2.0091ms	497.7328 Ops/s	505.4047 Ops/s	$\color{#d91a1a}-1.52\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.9778ms	1.2132ms	824.2850 Ops/s	862.2730 Ops/s	$\color{#d91a1a}-4.41\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.4948s	15.4273ms	64.8200 Ops/s	33.0584 Ops/s	$\textbf{\color{#35bf28}+96.08\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	10.8573ms	2.2274ms	448.9512 Ops/s	449.1083 Ops/s	$\color{#d91a1a}-0.03\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	6.2806ms	1.3113ms	762.5833 Ops/s	738.6237 Ops/s	$\color{#35bf28}+3.24\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	17.7752ms	15.8691ms	63.0155 Ops/s	66.4099 Ops/s	$\textbf{\color{#d91a1a}-5.11\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.8000ms	17.4270ms	57.3824 Ops/s	57.7434 Ops/s	$\color{#d91a1a}-0.63\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	20.5297ms	19.8756ms	50.3129 Ops/s	50.6371 Ops/s	$\color{#d91a1a}-0.64\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	20.1519ms	17.5687ms	56.9194 Ops/s	57.5214 Ops/s	$\color{#d91a1a}-1.05\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	20.2792ms	19.7282ms	50.6887 Ops/s	51.3534 Ops/s	$\color{#d91a1a}-1.29\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	21.4172ms	19.4847ms	51.3223 Ops/s	52.4788 Ops/s	$\color{#d91a1a}-2.20\%$