[Quality] Fix flaky test

[vmoens]

Stack from ghstack (oldest at bottom):

• -> [Quality] Fix flaky test #3211
• [Feature] Documentation #3192
• [Feature] SAC Trainer #3191
• [Feature] PPO Trainer Updates #3190
• [Feature] Trainer Algorithms - Configuration System #3189
• [Feature] Trainer Infrastructure - Timing and Utilities #3188

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3211

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 32 Pending

As of commit 2e9e545 with merge base 92c20cd ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}20$. Worsened: $\large\color{#d91a1a}23$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	83.3429μs	80.0240μs	12.4963 KOps/s	12.1328 KOps/s	$\color{#35bf28}+3.00\%$
test_tensor_to_bytestream_speed[torch.save]	0.1365ms	0.1357ms	7.3702 KOps/s	7.2696 KOps/s	$\color{#35bf28}+1.38\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1126s	0.1123s	8.9044 Ops/s	9.1607 Ops/s	$\color{#d91a1a}-2.80\%$
test_tensor_to_bytestream_speed[numpy]	2.7311μs	2.7278μs	366.5999 KOps/s	370.2432 KOps/s	$\color{#d91a1a}-0.98\%$
test_tensor_to_bytestream_speed[safetensors]	42.0903μs	41.9070μs	23.8624 KOps/s	23.3329 KOps/s	$\color{#35bf28}+2.27\%$
test_simple	0.5326s	0.5306s	1.8847 Ops/s	1.7935 Ops/s	$\textbf{\color{#35bf28}+5.08\%}$
test_transformed	1.0792s	1.0769s	0.9286 Ops/s	0.9032 Ops/s	$\color{#35bf28}+2.81\%$
test_serial	1.6040s	1.6006s	0.6248 Ops/s	0.6139 Ops/s	$\color{#35bf28}+1.77\%$
test_parallel	1.0499s	1.0375s	0.9639 Ops/s	0.9511 Ops/s	$\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-True-True-True-True]	0.1628ms	42.3161μs	23.6317 KOps/s	22.5026 KOps/s	$\textbf{\color{#35bf28}+5.02\%}$
test_step_mdp_speed[True-True-True-True-False]	49.2010μs	24.3700μs	41.0340 KOps/s	39.8827 KOps/s	$\color{#35bf28}+2.89\%$
test_step_mdp_speed[True-True-True-False-True]	53.0700μs	24.4016μs	40.9810 KOps/s	40.4667 KOps/s	$\color{#35bf28}+1.27\%$
test_step_mdp_speed[True-True-True-False-False]	38.8010μs	13.4833μs	74.1657 KOps/s	72.9797 KOps/s	$\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-False-True-True]	70.9620μs	46.2900μs	21.6029 KOps/s	21.5986 KOps/s	$\color{#35bf28}+0.02\%$
test_step_mdp_speed[True-True-False-True-False]	57.6910μs	26.5420μs	37.6762 KOps/s	36.4182 KOps/s	$\color{#35bf28}+3.45\%$
test_step_mdp_speed[True-True-False-False-True]	55.6210μs	26.5217μs	37.7049 KOps/s	37.3298 KOps/s	$\color{#35bf28}+1.00\%$
test_step_mdp_speed[True-True-False-False-False]	51.0410μs	16.0359μs	62.3603 KOps/s	62.2738 KOps/s	$\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-False-True-True-True]	82.5010μs	47.4381μs	21.0801 KOps/s	20.0805 KOps/s	$\color{#35bf28}+4.98\%$
test_step_mdp_speed[True-False-True-True-False]	62.7110μs	29.1767μs	34.2740 KOps/s	33.3211 KOps/s	$\color{#35bf28}+2.86\%$
test_step_mdp_speed[True-False-True-False-True]	63.4410μs	26.9120μs	37.1582 KOps/s	36.7463 KOps/s	$\color{#35bf28}+1.12\%$
test_step_mdp_speed[True-False-True-False-False]	45.3300μs	15.8577μs	63.0610 KOps/s	62.0875 KOps/s	$\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-False-False-True-True]	84.0810μs	50.2708μs	19.8923 KOps/s	19.2216 KOps/s	$\color{#35bf28}+3.49\%$
test_step_mdp_speed[True-False-False-True-False]	65.8910μs	31.8841μs	31.3636 KOps/s	31.4539 KOps/s	$\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-False-False-False-True]	54.0410μs	29.6149μs	33.7668 KOps/s	33.9100 KOps/s	$\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-False-False-False-False]	75.5210μs	18.7396μs	53.3629 KOps/s	53.1391 KOps/s	$\color{#35bf28}+0.42\%$
test_step_mdp_speed[False-True-True-True-True]	81.9220μs	48.5251μs	20.6079 KOps/s	20.3156 KOps/s	$\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-True-True-True-False]	56.7510μs	29.6549μs	33.7212 KOps/s	33.6330 KOps/s	$\color{#35bf28}+0.26\%$
test_step_mdp_speed[False-True-True-False-True]	2.4329ms	30.9191μs	32.3424 KOps/s	31.6490 KOps/s	$\color{#35bf28}+2.19\%$
test_step_mdp_speed[False-True-True-False-False]	74.1810μs	17.0642μs	58.6020 KOps/s	54.7692 KOps/s	$\textbf{\color{#35bf28}+7.00\%}$
test_step_mdp_speed[False-True-False-True-True]	93.4220μs	51.6477μs	19.3619 KOps/s	19.1816 KOps/s	$\color{#35bf28}+0.94\%$
test_step_mdp_speed[False-True-False-True-False]	65.4210μs	32.2615μs	30.9967 KOps/s	30.1408 KOps/s	$\color{#35bf28}+2.84\%$
test_step_mdp_speed[False-True-False-False-True]	70.1310μs	32.6011μs	30.6738 KOps/s	29.1821 KOps/s	$\textbf{\color{#35bf28}+5.11\%}$
test_step_mdp_speed[False-True-False-False-False]	56.4510μs	20.2753μs	49.3211 KOps/s	46.8909 KOps/s	$\textbf{\color{#35bf28}+5.18\%}$
test_step_mdp_speed[False-False-True-True-True]	88.6610μs	53.8881μs	18.5570 KOps/s	18.0160 KOps/s	$\color{#35bf28}+3.00\%$
test_step_mdp_speed[False-False-True-True-False]	67.7120μs	34.3651μs	29.0993 KOps/s	27.9089 KOps/s	$\color{#35bf28}+4.27\%$
test_step_mdp_speed[False-False-True-False-True]	67.8710μs	32.6430μs	30.6344 KOps/s	28.9190 KOps/s	$\textbf{\color{#35bf28}+5.93\%}$
test_step_mdp_speed[False-False-True-False-False]	57.7810μs	20.6098μs	48.5205 KOps/s	48.0335 KOps/s	$\color{#35bf28}+1.01\%$
test_step_mdp_speed[False-False-False-True-True]	87.5220μs	55.8847μs	17.8940 KOps/s	17.5452 KOps/s	$\color{#35bf28}+1.99\%$
test_step_mdp_speed[False-False-False-True-False]	78.7420μs	36.9766μs	27.0441 KOps/s	26.2182 KOps/s	$\color{#35bf28}+3.15\%$
test_step_mdp_speed[False-False-False-False-True]	76.5110μs	34.8980μs	28.6550 KOps/s	28.0698 KOps/s	$\color{#35bf28}+2.08\%$
test_step_mdp_speed[False-False-False-False-False]	59.3710μs	22.5726μs	44.3015 KOps/s	43.2893 KOps/s	$\color{#35bf28}+2.34\%$
test_values[generalized_advantage_estimate-True-True]	10.1445ms	9.4068ms	106.3064 Ops/s	104.3719 Ops/s	$\color{#35bf28}+1.85\%$
test_values[vec_generalized_advantage_estimate-True-True]	19.7497ms	17.5592ms	56.9502 Ops/s	90.3946 Ops/s	$\textbf{\color{#d91a1a}-37.00\%}$
test_values[td0_return_estimate-False-False]	0.2176ms	0.1287ms	7.7715 KOps/s	7.9065 KOps/s	$\color{#d91a1a}-1.71\%$
test_values[td1_return_estimate-False-False]	27.6491ms	26.3495ms	37.9514 Ops/s	38.6565 Ops/s	$\color{#d91a1a}-1.82\%$
test_values[vec_td1_return_estimate-False-False]	18.6686ms	17.6783ms	56.5666 Ops/s	87.4944 Ops/s	$\textbf{\color{#d91a1a}-35.35\%}$
test_values[td_lambda_return_estimate-True-False]	41.7216ms	40.1014ms	24.9368 Ops/s	25.7980 Ops/s	$\color{#d91a1a}-3.34\%$
test_values[vec_td_lambda_return_estimate-True-False]	18.2375ms	17.7071ms	56.4745 Ops/s	90.1625 Ops/s	$\textbf{\color{#d91a1a}-37.36\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.1627ms	8.0947ms	123.5377 Ops/s	125.4996 Ops/s	$\color{#d91a1a}-1.56\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.8039ms	1.5229ms	656.6632 Ops/s	667.5776 Ops/s	$\color{#d91a1a}-1.63\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4713ms	0.3986ms	2.5086 KOps/s	2.5173 KOps/s	$\color{#d91a1a}-0.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	34.8990ms	33.9833ms	29.4262 Ops/s	33.7450 Ops/s	$\textbf{\color{#d91a1a}-12.80\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.9620ms	1.7036ms	586.9885 Ops/s	583.1734 Ops/s	$\color{#35bf28}+0.65\%$
test_dqn_speed[False-None]	6.4645ms	1.3821ms	723.5400 Ops/s	713.1479 Ops/s	$\color{#35bf28}+1.46\%$
test_dqn_speed[False-backward]	2.0210ms	1.8822ms	531.2870 Ops/s	515.0397 Ops/s	$\color{#35bf28}+3.15\%$
test_dqn_speed[True-None]	0.7186ms	0.5292ms	1.8898 KOps/s	1.9572 KOps/s	$\color{#d91a1a}-3.45\%$
test_dqn_speed[True-backward]	0.9999ms	0.9501ms	1.0525 KOps/s	1.0145 KOps/s	$\color{#35bf28}+3.74\%$
test_dqn_speed[reduce-overhead-None]	0.9117ms	0.5154ms	1.9402 KOps/s	1.9782 KOps/s	$\color{#d91a1a}-1.92\%$
test_dqn_speed[reduce-overhead-backward]	1.0100ms	0.9375ms	1.0666 KOps/s	1.0419 KOps/s	$\color{#35bf28}+2.37\%$
test_ddpg_speed[False-None]	3.1672ms	2.7776ms	360.0204 Ops/s	358.1877 Ops/s	$\color{#35bf28}+0.51\%$
test_ddpg_speed[False-backward]	4.1497ms	4.0295ms	248.1676 Ops/s	249.9197 Ops/s	$\color{#d91a1a}-0.70\%$
test_ddpg_speed[True-None]	1.7294ms	1.3920ms	718.3850 Ops/s	727.7992 Ops/s	$\color{#d91a1a}-1.29\%$
test_ddpg_speed[True-backward]	2.3824ms	2.3344ms	428.3693 Ops/s	398.4147 Ops/s	$\textbf{\color{#35bf28}+7.52\%}$
test_ddpg_speed[reduce-overhead-None]	1.7941ms	1.3842ms	722.4552 Ops/s	712.5490 Ops/s	$\color{#35bf28}+1.39\%$
test_ddpg_speed[reduce-overhead-backward]	2.4782ms	2.3189ms	431.2363 Ops/s	430.8265 Ops/s	$\color{#35bf28}+0.10\%$
test_sac_speed[False-None]	8.1610ms	7.6813ms	130.1868 Ops/s	129.4056 Ops/s	$\color{#35bf28}+0.60\%$
test_sac_speed[False-backward]	11.3561ms	10.9163ms	91.6058 Ops/s	91.1441 Ops/s	$\color{#35bf28}+0.51\%$
test_sac_speed[True-None]	2.4386ms	2.0742ms	482.1212 Ops/s	464.4558 Ops/s	$\color{#35bf28}+3.80\%$
test_sac_speed[True-backward]	4.0473ms	3.9334ms	254.2313 Ops/s	250.0404 Ops/s	$\color{#35bf28}+1.68\%$
test_sac_speed[reduce-overhead-None]	2.4483ms	2.0650ms	484.2683 Ops/s	452.5900 Ops/s	$\textbf{\color{#35bf28}+7.00\%}$
test_sac_speed[reduce-overhead-backward]	4.1070ms	3.9364ms	254.0391 Ops/s	222.9681 Ops/s	$\textbf{\color{#35bf28}+13.94\%}$
test_redq_speed[False-None]	15.5332ms	10.2068ms	97.9743 Ops/s	99.2583 Ops/s	$\color{#d91a1a}-1.29\%$
test_redq_speed[False-backward]	17.8736ms	17.2016ms	58.1343 Ops/s	57.8255 Ops/s	$\color{#35bf28}+0.53\%$
test_redq_speed[True-None]	4.5412ms	4.3277ms	231.0676 Ops/s	226.8624 Ops/s	$\color{#35bf28}+1.85\%$
test_redq_speed[True-backward]	9.8681ms	9.5018ms	105.2431 Ops/s	103.7840 Ops/s	$\color{#35bf28}+1.41\%$
test_redq_speed[reduce-overhead-None]	5.2342ms	4.3404ms	230.3956 Ops/s	234.8153 Ops/s	$\color{#d91a1a}-1.88\%$
test_redq_speed[reduce-overhead-backward]	10.1546ms	9.7673ms	102.3822 Ops/s	105.2226 Ops/s	$\color{#d91a1a}-2.70\%$
test_redq_deprec_speed[False-None]	11.0779ms	10.6378ms	94.0041 Ops/s	90.8470 Ops/s	$\color{#35bf28}+3.48\%$
test_redq_deprec_speed[False-backward]	16.7503ms	15.4033ms	64.9213 Ops/s	62.4177 Ops/s	$\color{#35bf28}+4.01\%$
test_redq_deprec_speed[True-None]	3.6663ms	3.5289ms	283.3782 Ops/s	266.5274 Ops/s	$\textbf{\color{#35bf28}+6.32\%}$
test_redq_deprec_speed[True-backward]	7.5310ms	7.2868ms	137.2347 Ops/s	129.4623 Ops/s	$\textbf{\color{#35bf28}+6.00\%}$
test_redq_deprec_speed[reduce-overhead-None]	3.6932ms	3.5092ms	284.9681 Ops/s	290.2853 Ops/s	$\color{#d91a1a}-1.83\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.5452ms	7.2962ms	137.0584 Ops/s	130.2627 Ops/s	$\textbf{\color{#35bf28}+5.22\%}$
test_td3_speed[False-None]	7.9572ms	7.7255ms	129.4415 Ops/s	130.2884 Ops/s	$\color{#d91a1a}-0.65\%$
test_td3_speed[False-backward]	11.2774ms	10.5735ms	94.5759 Ops/s	94.9652 Ops/s	$\color{#d91a1a}-0.41\%$
test_td3_speed[True-None]	1.8517ms	1.7822ms	561.1182 Ops/s	558.4422 Ops/s	$\color{#35bf28}+0.48\%$
test_td3_speed[True-backward]	3.7284ms	3.5561ms	281.2089 Ops/s	274.8266 Ops/s	$\color{#35bf28}+2.32\%$
test_td3_speed[reduce-overhead-None]	1.7973ms	1.7555ms	569.6395 Ops/s	560.4733 Ops/s	$\color{#35bf28}+1.64\%$
test_td3_speed[reduce-overhead-backward]	3.7320ms	3.5687ms	280.2110 Ops/s	237.0913 Ops/s	$\textbf{\color{#35bf28}+18.19\%}$
test_cql_speed[False-None]	25.9096ms	25.1953ms	39.6899 Ops/s	39.5279 Ops/s	$\color{#35bf28}+0.41\%$
test_cql_speed[False-backward]	39.8606ms	35.1960ms	28.4123 Ops/s	28.6943 Ops/s	$\color{#d91a1a}-0.98\%$
test_cql_speed[True-None]	12.5403ms	12.2481ms	81.6452 Ops/s	85.2125 Ops/s	$\color{#d91a1a}-4.19\%$
test_cql_speed[True-backward]	18.3278ms	17.9768ms	55.6272 Ops/s	59.2643 Ops/s	$\textbf{\color{#d91a1a}-6.14\%}$
test_cql_speed[reduce-overhead-None]	12.6634ms	12.3288ms	81.1110 Ops/s	84.8083 Ops/s	$\color{#d91a1a}-4.36\%$
test_cql_speed[reduce-overhead-backward]	18.4733ms	18.0443ms	55.4191 Ops/s	53.4888 Ops/s	$\color{#35bf28}+3.61\%$
test_a2c_speed[False-None]	5.6698ms	5.2858ms	189.1867 Ops/s	186.3218 Ops/s	$\color{#35bf28}+1.54\%$
test_a2c_speed[False-backward]	12.0280ms	11.7517ms	85.0944 Ops/s	84.6739 Ops/s	$\color{#35bf28}+0.50\%$
test_a2c_speed[True-None]	3.9258ms	3.6562ms	273.5075 Ops/s	280.0569 Ops/s	$\color{#d91a1a}-2.34\%$
test_a2c_speed[True-backward]	9.8616ms	8.6360ms	115.7939 Ops/s	121.5099 Ops/s	$\color{#d91a1a}-4.70\%$
test_a2c_speed[reduce-overhead-None]	4.1250ms	3.6881ms	271.1396 Ops/s	289.9224 Ops/s	$\textbf{\color{#d91a1a}-6.48\%}$
test_a2c_speed[reduce-overhead-backward]	9.0236ms	8.8191ms	113.3903 Ops/s	118.6969 Ops/s	$\color{#d91a1a}-4.47\%$
test_ppo_speed[False-None]	6.0433ms	5.7383ms	174.2681 Ops/s	179.9105 Ops/s	$\color{#d91a1a}-3.14\%$
test_ppo_speed[False-backward]	12.7873ms	12.2044ms	81.9377 Ops/s	81.7590 Ops/s	$\color{#35bf28}+0.22\%$
test_ppo_speed[True-None]	3.8202ms	3.6451ms	274.3375 Ops/s	289.7643 Ops/s	$\textbf{\color{#d91a1a}-5.32\%}$
test_ppo_speed[True-backward]	8.6001ms	8.3300ms	120.0476 Ops/s	115.3532 Ops/s	$\color{#35bf28}+4.07\%$
test_ppo_speed[reduce-overhead-None]	3.8216ms	3.6278ms	275.6473 Ops/s	295.2974 Ops/s	$\textbf{\color{#d91a1a}-6.65\%}$
test_ppo_speed[reduce-overhead-backward]	8.7967ms	8.6065ms	116.1911 Ops/s	119.3717 Ops/s	$\color{#d91a1a}-2.66\%$
test_reinforce_speed[False-None]	4.8882ms	4.4874ms	222.8479 Ops/s	237.4915 Ops/s	$\textbf{\color{#d91a1a}-6.17\%}$
test_reinforce_speed[False-backward]	7.7454ms	7.2960ms	137.0609 Ops/s	141.9642 Ops/s	$\color{#d91a1a}-3.45\%$
test_reinforce_speed[True-None]	2.9882ms	2.8293ms	353.4445 Ops/s	383.0656 Ops/s	$\textbf{\color{#d91a1a}-7.73\%}$
test_reinforce_speed[True-backward]	7.8970ms	7.5714ms	132.0755 Ops/s	128.1198 Ops/s	$\color{#35bf28}+3.09\%$
test_reinforce_speed[reduce-overhead-None]	3.0238ms	2.7900ms	358.4196 Ops/s	355.1204 Ops/s	$\color{#35bf28}+0.93\%$
test_reinforce_speed[reduce-overhead-backward]	8.0854ms	7.8549ms	127.3090 Ops/s	122.6448 Ops/s	$\color{#35bf28}+3.80\%$
test_iql_speed[False-None]	25.1446ms	19.7461ms	50.6430 Ops/s	50.2618 Ops/s	$\color{#35bf28}+0.76\%$
test_iql_speed[False-backward]	35.2724ms	30.0970ms	33.2259 Ops/s	33.4496 Ops/s	$\color{#d91a1a}-0.67\%$
test_iql_speed[True-None]	8.7449ms	8.4522ms	118.3121 Ops/s	115.6211 Ops/s	$\color{#35bf28}+2.33\%$
test_iql_speed[True-backward]	16.9912ms	16.5043ms	60.5904 Ops/s	60.1239 Ops/s	$\color{#35bf28}+0.78\%$
test_iql_speed[reduce-overhead-None]	8.7474ms	8.5254ms	117.2971 Ops/s	117.6008 Ops/s	$\color{#d91a1a}-0.26\%$
test_iql_speed[reduce-overhead-backward]	17.2215ms	16.8683ms	59.2826 Ops/s	58.4823 Ops/s	$\color{#35bf28}+1.37\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.2935ms	5.8632ms	170.5556 Ops/s	170.4058 Ops/s	$\color{#35bf28}+0.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6422ms	0.3601ms	2.7770 KOps/s	2.6443 KOps/s	$\textbf{\color{#35bf28}+5.02\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6361ms	0.3848ms	2.5990 KOps/s	3.9835 KOps/s	$\textbf{\color{#d91a1a}-34.76\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.8097ms	5.5462ms	180.3052 Ops/s	178.5302 Ops/s	$\color{#35bf28}+0.99\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.0133ms	0.3583ms	2.7912 KOps/s	3.1034 KOps/s	$\textbf{\color{#d91a1a}-10.06\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5639ms	0.3539ms	2.8260 KOps/s	4.0417 KOps/s	$\textbf{\color{#d91a1a}-30.08\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.7110ms	1.3947ms	717.0062 Ops/s	815.1113 Ops/s	$\textbf{\color{#d91a1a}-12.04\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.6542ms	1.4009ms	713.8177 Ops/s	888.2461 Ops/s	$\textbf{\color{#d91a1a}-19.64\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	9.5473ms	5.8642ms	170.5261 Ops/s	173.2181 Ops/s	$\color{#d91a1a}-1.55\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.9088ms	0.4852ms	2.0612 KOps/s	2.2049 KOps/s	$\textbf{\color{#d91a1a}-6.52\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8146ms	0.4723ms	2.1171 KOps/s	2.3521 KOps/s	$\textbf{\color{#d91a1a}-9.99\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.6919ms	5.5763ms	179.3313 Ops/s	176.8764 Ops/s	$\color{#35bf28}+1.39\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0034ms	0.3430ms	2.9155 KOps/s	3.1957 KOps/s	$\textbf{\color{#d91a1a}-8.77\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5303ms	0.3280ms	3.0488 KOps/s	3.3946 KOps/s	$\textbf{\color{#d91a1a}-10.19\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.7203ms	5.4948ms	181.9903 Ops/s	178.8000 Ops/s	$\color{#35bf28}+1.78\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6877s	1.2763ms	783.5223 Ops/s	3.1429 KOps/s	$\textbf{\color{#d91a1a}-75.07\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.4489ms	0.2553ms	3.9171 KOps/s	3.7941 KOps/s	$\color{#35bf28}+3.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.8791ms	5.7069ms	175.2267 Ops/s	172.4439 Ops/s	$\color{#35bf28}+1.61\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	2.3989ms	0.4779ms	2.0924 KOps/s	1.9232 KOps/s	$\textbf{\color{#35bf28}+8.80\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6977ms	0.4604ms	2.1721 KOps/s	2.0326 KOps/s	$\textbf{\color{#35bf28}+6.86\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	6.5392ms	4.9739ms	201.0479 Ops/s	197.0971 Ops/s	$\color{#35bf28}+2.00\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	9.4986ms	2.0247ms	493.9021 Ops/s	442.9022 Ops/s	$\textbf{\color{#35bf28}+11.51\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	6.4256ms	1.1622ms	860.4733 Ops/s	785.2643 Ops/s	$\textbf{\color{#35bf28}+9.58\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.4889s	14.6424ms	68.2946 Ops/s	196.7353 Ops/s	$\textbf{\color{#d91a1a}-65.29\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	10.4910ms	2.1283ms	469.8539 Ops/s	506.9608 Ops/s	$\textbf{\color{#d91a1a}-7.32\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.4113ms	1.0755ms	929.7609 Ops/s	827.8119 Ops/s	$\textbf{\color{#35bf28}+12.32\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	8.0597ms	5.2346ms	191.0382 Ops/s	55.3332 Ops/s	$\textbf{\color{#35bf28}+245.25\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	7.4584ms	2.1594ms	463.0900 Ops/s	478.7017 Ops/s	$\color{#d91a1a}-3.26\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	3.6901ms	1.0268ms	973.8564 Ops/s	1.0333 KOps/s	$\textbf{\color{#d91a1a}-5.75\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	34.5236ms	32.1759ms	31.0791 Ops/s	31.2949 Ops/s	$\color{#d91a1a}-0.69\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	18.7646ms	17.0888ms	58.5179 Ops/s	58.8532 Ops/s	$\color{#d91a1a}-0.57\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	34.9005ms	32.4776ms	30.7904 Ops/s	30.4476 Ops/s	$\color{#35bf28}+1.13\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	18.4229ms	16.9231ms	59.0908 Ops/s	58.0737 Ops/s	$\color{#35bf28}+1.75\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	35.4233ms	33.8247ms	29.5642 Ops/s	28.9066 Ops/s	$\color{#35bf28}+2.28\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	19.7345ms	18.6938ms	53.4937 Ops/s	52.0041 Ops/s	$\color{#35bf28}+2.86\%$