[Feature] PPO compatibility with compile

[vmoens]

Stack from ghstack (oldest at bottom):

• [Quality] IMPALA auto-device #2654
• [Feature] timeit.printevery #2653
• -> [Feature] PPO compatibility with compile #2652
• [Feature] IQL compatibility with compile #2649

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2652

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 7 Unrelated Failures

As of commit 99db504 with merge base 6482766 ():

NEW FAILURE - The following job has failed:

• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  Process completed with exit code 1.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cpu (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda11_8 (gh) (trunk failure)
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_4 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_6 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh) (trunk failure)
  AttributeError: _ARRAY_API not found
• Unit-tests on Linux / tests-gpu (3.11, 12.1) / linux-job (gh) (trunk failure)
  test/test_loggers.py::TestMLFlowLogger::test_log_video[steps1]
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (trunk failure)
  test/test_storage_map.py::TestMCTSForest::test_forest_parent[simple-False]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_simple	0.4345s	0.4328s	2.3104 Ops/s	2.1715 Ops/s	$\textbf{\color{#35bf28}+6.40\%}$
test_transformed	0.6174s	0.6112s	1.6360 Ops/s	1.5547 Ops/s	$\textbf{\color{#35bf28}+5.23\%}$
test_serial	1.3572s	1.3543s	0.7384 Ops/s	0.7207 Ops/s	$\color{#35bf28}+2.46\%$
test_parallel	1.3950s	1.3038s	0.7670 Ops/s	0.7566 Ops/s	$\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-True-True-True-True]	0.2190ms	29.6000μs	33.7838 KOps/s	33.1924 KOps/s	$\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-True-True-True-False]	53.1900μs	17.4786μs	57.2128 KOps/s	56.5570 KOps/s	$\color{#35bf28}+1.16\%$
test_step_mdp_speed[True-True-True-False-True]	45.6150μs	16.7632μs	59.6545 KOps/s	58.9490 KOps/s	$\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-True-True-False-False]	35.8970μs	9.8703μs	101.3145 KOps/s	100.9922 KOps/s	$\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-True-False-True-True]	86.2320μs	32.2136μs	31.0428 KOps/s	31.3547 KOps/s	$\color{#d91a1a}-0.99\%$
test_step_mdp_speed[True-True-False-True-False]	55.5240μs	19.5343μs	51.1921 KOps/s	51.5512 KOps/s	$\color{#d91a1a}-0.70\%$
test_step_mdp_speed[True-True-False-False-True]	49.8830μs	18.8519μs	53.0451 KOps/s	52.9004 KOps/s	$\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-True-False-False-False]	33.6230μs	11.8834μs	84.1513 KOps/s	85.1297 KOps/s	$\color{#d91a1a}-1.15\%$
test_step_mdp_speed[True-False-True-True-True]	77.7250μs	33.7351μs	29.6427 KOps/s	29.1488 KOps/s	$\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-False-True-True-False]	59.7820μs	21.4186μs	46.6885 KOps/s	46.3797 KOps/s	$\color{#35bf28}+0.67\%$
test_step_mdp_speed[True-False-True-False-True]	67.3060μs	19.0041μs	52.6203 KOps/s	52.2574 KOps/s	$\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-False-True-False-False]	45.7360μs	11.6293μs	85.9894 KOps/s	83.2994 KOps/s	$\color{#35bf28}+3.23\%$
test_step_mdp_speed[True-False-False-True-True]	74.7900μs	35.7690μs	27.9572 KOps/s	27.6553 KOps/s	$\color{#35bf28}+1.09\%$
test_step_mdp_speed[True-False-False-True-False]	59.3610μs	23.1849μs	43.1315 KOps/s	42.4846 KOps/s	$\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-False-False-False-True]	53.8100μs	20.8242μs	48.0211 KOps/s	48.3068 KOps/s	$\color{#d91a1a}-0.59\%$
test_step_mdp_speed[True-False-False-False-False]	0.6963ms	13.3329μs	75.0024 KOps/s	73.9276 KOps/s	$\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-True-True-True-True]	74.8700μs	33.9372μs	29.4662 KOps/s	29.1145 KOps/s	$\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-True-True-True-False]	57.3370μs	21.4896μs	46.5342 KOps/s	46.1877 KOps/s	$\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-True-True-False-True]	55.8250μs	21.6960μs	46.0915 KOps/s	46.0560 KOps/s	$\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-True-True-False-False]	38.7530μs	13.0214μs	76.7965 KOps/s	75.6224 KOps/s	$\color{#35bf28}+1.55\%$
test_step_mdp_speed[False-True-False-True-True]	78.2360μs	36.2121μs	27.6151 KOps/s	27.7079 KOps/s	$\color{#d91a1a}-0.34\%$
test_step_mdp_speed[False-True-False-True-False]	71.0040μs	23.3997μs	42.7356 KOps/s	42.7511 KOps/s	$\color{#d91a1a}-0.04\%$
test_step_mdp_speed[False-True-False-False-True]	2.6630ms	23.0071μs	43.4649 KOps/s	42.5185 KOps/s	$\color{#35bf28}+2.23\%$
test_step_mdp_speed[False-True-False-False-False]	39.3230μs	14.8866μs	67.1744 KOps/s	66.3061 KOps/s	$\color{#35bf28}+1.31\%$
test_step_mdp_speed[False-False-True-True-True]	81.4720μs	37.3267μs	26.7905 KOps/s	26.2777 KOps/s	$\color{#35bf28}+1.95\%$
test_step_mdp_speed[False-False-True-True-False]	80.7790μs	24.9569μs	40.0690 KOps/s	39.4700 KOps/s	$\color{#35bf28}+1.52\%$
test_step_mdp_speed[False-False-True-False-True]	52.5380μs	22.9945μs	43.4887 KOps/s	43.0719 KOps/s	$\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-False-True-False-False]	47.4990μs	14.8372μs	67.3980 KOps/s	66.0113 KOps/s	$\color{#35bf28}+2.10\%$
test_step_mdp_speed[False-False-False-True-True]	73.1370μs	39.1084μs	25.5700 KOps/s	25.2742 KOps/s	$\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-False-False-True-False]	59.5410μs	26.9432μs	37.1151 KOps/s	37.2273 KOps/s	$\color{#d91a1a}-0.30\%$
test_step_mdp_speed[False-False-False-False-True]	55.8350μs	24.4760μs	40.8563 KOps/s	40.2373 KOps/s	$\color{#35bf28}+1.54\%$
test_step_mdp_speed[False-False-False-False-False]	76.2930μs	16.3240μs	61.2595 KOps/s	59.8124 KOps/s	$\color{#35bf28}+2.42\%$
test_values[generalized_advantage_estimate-True-True]	12.0216ms	9.5232ms	105.0066 Ops/s	103.8931 Ops/s	$\color{#35bf28}+1.07\%$
test_values[vec_generalized_advantage_estimate-True-True]	37.0907ms	33.5642ms	29.7937 Ops/s	27.5852 Ops/s	$\textbf{\color{#35bf28}+8.01\%}$
test_values[td0_return_estimate-False-False]	0.2716ms	0.1786ms	5.5976 KOps/s	5.5582 KOps/s	$\color{#35bf28}+0.71\%$
test_values[td1_return_estimate-False-False]	26.7450ms	23.8721ms	41.8899 Ops/s	41.2767 Ops/s	$\color{#35bf28}+1.49\%$
test_values[vec_td1_return_estimate-False-False]	35.0410ms	33.6098ms	29.7533 Ops/s	27.5059 Ops/s	$\textbf{\color{#35bf28}+8.17\%}$
test_values[td_lambda_return_estimate-True-False]	35.8194ms	34.3144ms	29.1423 Ops/s	28.9001 Ops/s	$\color{#35bf28}+0.84\%$
test_values[vec_td_lambda_return_estimate-True-False]	35.5881ms	33.5775ms	29.7819 Ops/s	27.5773 Ops/s	$\textbf{\color{#35bf28}+7.99\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.5471ms	8.2809ms	120.7600 Ops/s	120.0171 Ops/s	$\color{#35bf28}+0.62\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.9731ms	1.9267ms	519.0341 Ops/s	408.2475 Ops/s	$\textbf{\color{#35bf28}+27.14\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.6052ms	0.3560ms	2.8086 KOps/s	2.7466 KOps/s	$\color{#35bf28}+2.26\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	40.5701ms	39.4667ms	25.3378 Ops/s	21.8844 Ops/s	$\textbf{\color{#35bf28}+15.78\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.8942ms	3.0506ms	327.8094 Ops/s	289.3194 Ops/s	$\textbf{\color{#35bf28}+13.30\%}$
test_dqn_speed[False-None]	5.8617ms	1.3661ms	732.0286 Ops/s	715.7973 Ops/s	$\color{#35bf28}+2.27\%$
test_dqn_speed[False-backward]	1.8986ms	1.8361ms	544.6238 Ops/s	533.1387 Ops/s	$\color{#35bf28}+2.15\%$
test_dqn_speed[True-None]	0.7596ms	0.4613ms	2.1678 KOps/s	2.1363 KOps/s	$\color{#35bf28}+1.48\%$
test_dqn_speed[True-backward]	0.9442ms	0.8742ms	1.1440 KOps/s	1.1228 KOps/s	$\color{#35bf28}+1.89\%$
test_dqn_speed[reduce-overhead-None]	0.5583ms	0.4576ms	2.1852 KOps/s	2.1330 KOps/s	$\color{#35bf28}+2.45\%$
test_dqn_speed[reduce-overhead-backward]	1.0009ms	0.8879ms	1.1263 KOps/s	1.0851 KOps/s	$\color{#35bf28}+3.79\%$
test_ddpg_speed[False-None]	3.5558ms	2.8492ms	350.9717 Ops/s	342.9155 Ops/s	$\color{#35bf28}+2.35\%$
test_ddpg_speed[False-backward]	4.3719ms	3.9762ms	251.4961 Ops/s	248.3760 Ops/s	$\color{#35bf28}+1.26\%$
test_ddpg_speed[True-None]	1.3609ms	0.9929ms	1.0071 KOps/s	989.9345 Ops/s	$\color{#35bf28}+1.74\%$
test_ddpg_speed[True-backward]	1.9740ms	1.8708ms	534.5353 Ops/s	513.0223 Ops/s	$\color{#35bf28}+4.19\%$
test_ddpg_speed[reduce-overhead-None]	1.4247ms	0.9967ms	1.0033 KOps/s	983.0831 Ops/s	$\color{#35bf28}+2.05\%$
test_ddpg_speed[reduce-overhead-backward]	1.9861ms	1.9027ms	525.5761 Ops/s	509.1473 Ops/s	$\color{#35bf28}+3.23\%$
test_sac_speed[False-None]	8.5233ms	7.9597ms	125.6325 Ops/s	119.5938 Ops/s	$\textbf{\color{#35bf28}+5.05\%}$
test_sac_speed[False-backward]	12.8377ms	10.7046ms	93.4178 Ops/s	85.6141 Ops/s	$\textbf{\color{#35bf28}+9.11\%}$
test_sac_speed[True-None]	2.4484ms	1.8182ms	550.0092 Ops/s	545.3414 Ops/s	$\color{#35bf28}+0.86\%$
test_sac_speed[True-backward]	3.7961ms	3.6077ms	277.1832 Ops/s	273.7977 Ops/s	$\color{#35bf28}+1.24\%$
test_sac_speed[reduce-overhead-None]	2.5746ms	1.8643ms	536.3813 Ops/s	537.4816 Ops/s	$\color{#d91a1a}-0.20\%$
test_sac_speed[reduce-overhead-backward]	3.7565ms	3.5873ms	278.7607 Ops/s	272.6579 Ops/s	$\color{#35bf28}+2.24\%$
test_redq_speed[False-None]	0.2374s	15.6682ms	63.8234 Ops/s	63.1258 Ops/s	$\color{#35bf28}+1.11\%$
test_redq_speed[False-backward]	23.4907ms	22.1962ms	45.0529 Ops/s	43.4212 Ops/s	$\color{#35bf28}+3.76\%$
test_redq_speed[True-None]	5.8198ms	4.7751ms	209.4216 Ops/s	207.9806 Ops/s	$\color{#35bf28}+0.69\%$
test_redq_speed[True-backward]	13.4462ms	12.1743ms	82.1401 Ops/s	80.2276 Ops/s	$\color{#35bf28}+2.38\%$
test_redq_speed[reduce-overhead-None]	5.6438ms	4.8722ms	205.2478 Ops/s	205.4934 Ops/s	$\color{#d91a1a}-0.12\%$
test_redq_speed[reduce-overhead-backward]	16.0275ms	12.5208ms	79.8668 Ops/s	79.0934 Ops/s	$\color{#35bf28}+0.98\%$
test_redq_deprec_speed[False-None]	0.2501s	15.9043ms	62.8762 Ops/s	74.6227 Ops/s	$\textbf{\color{#d91a1a}-15.74\%}$
test_redq_deprec_speed[False-backward]	19.3769ms	18.2765ms	54.7149 Ops/s	52.3114 Ops/s	$\color{#35bf28}+4.59\%$
test_redq_deprec_speed[True-None]	3.9692ms	3.5892ms	278.6134 Ops/s	277.6641 Ops/s	$\color{#35bf28}+0.34\%$
test_redq_deprec_speed[True-backward]	9.0902ms	8.0821ms	123.7297 Ops/s	118.7311 Ops/s	$\color{#35bf28}+4.21\%$
test_redq_deprec_speed[reduce-overhead-None]	4.4919ms	3.5977ms	277.9538 Ops/s	273.0833 Ops/s	$\color{#35bf28}+1.78\%$
test_redq_deprec_speed[reduce-overhead-backward]	9.1087ms	8.2611ms	121.0498 Ops/s	121.5866 Ops/s	$\color{#d91a1a}-0.44\%$
test_td3_speed[False-None]	8.2077ms	7.9066ms	126.4763 Ops/s	120.9327 Ops/s	$\color{#35bf28}+4.58\%$
test_td3_speed[False-backward]	11.4600ms	10.4195ms	95.9736 Ops/s	92.4879 Ops/s	$\color{#35bf28}+3.77\%$
test_td3_speed[True-None]	1.8988ms	1.6984ms	588.8062 Ops/s	571.5676 Ops/s	$\color{#35bf28}+3.02\%$
test_td3_speed[True-backward]	3.4231ms	3.3173ms	301.4538 Ops/s	294.9520 Ops/s	$\color{#35bf28}+2.20\%$
test_td3_speed[reduce-overhead-None]	1.8671ms	1.6956ms	589.7580 Ops/s	572.3500 Ops/s	$\color{#35bf28}+3.04\%$
test_td3_speed[reduce-overhead-backward]	3.4254ms	3.3063ms	302.4571 Ops/s	283.8285 Ops/s	$\textbf{\color{#35bf28}+6.56\%}$
test_cql_speed[False-None]	38.4208ms	36.1417ms	27.6689 Ops/s	26.8267 Ops/s	$\color{#35bf28}+3.14\%$
test_cql_speed[False-backward]	48.4618ms	46.0836ms	21.6997 Ops/s	20.9435 Ops/s	$\color{#35bf28}+3.61\%$
test_cql_speed[True-None]	16.8671ms	15.7135ms	63.6394 Ops/s	62.4267 Ops/s	$\color{#35bf28}+1.94\%$
test_cql_speed[True-backward]	23.7196ms	22.5873ms	44.2726 Ops/s	43.5132 Ops/s	$\color{#35bf28}+1.75\%$
test_cql_speed[reduce-overhead-None]	16.5402ms	15.6905ms	63.7328 Ops/s	61.1337 Ops/s	$\color{#35bf28}+4.25\%$
test_cql_speed[reduce-overhead-backward]	23.8140ms	22.6437ms	44.1623 Ops/s	44.2694 Ops/s	$\color{#d91a1a}-0.24\%$
test_a2c_speed[False-None]	8.2212ms	7.5341ms	132.7298 Ops/s	135.4802 Ops/s	$\color{#d91a1a}-2.03\%$
test_a2c_speed[False-backward]	16.2305ms	14.7506ms	67.7938 Ops/s	68.2936 Ops/s	$\color{#d91a1a}-0.73\%$
test_a2c_speed[True-None]	5.1490ms	4.3307ms	230.9107 Ops/s	229.5436 Ops/s	$\color{#35bf28}+0.60\%$
test_a2c_speed[True-backward]	11.4744ms	11.0876ms	90.1908 Ops/s	89.4545 Ops/s	$\color{#35bf28}+0.82\%$
test_a2c_speed[reduce-overhead-None]	5.0120ms	4.2486ms	235.3695 Ops/s	233.5486 Ops/s	$\color{#35bf28}+0.78\%$
test_a2c_speed[reduce-overhead-backward]	13.2784ms	11.4551ms	87.2975 Ops/s	89.6415 Ops/s	$\color{#d91a1a}-2.61\%$
test_ppo_speed[False-None]	8.4754ms	7.6787ms	130.2310 Ops/s	131.2567 Ops/s	$\color{#d91a1a}-0.78\%$
test_ppo_speed[False-backward]	15.8498ms	15.2448ms	65.5959 Ops/s	66.1193 Ops/s	$\color{#d91a1a}-0.79\%$
test_ppo_speed[True-None]	4.2492ms	3.7661ms	265.5261 Ops/s	266.0304 Ops/s	$\color{#d91a1a}-0.19\%$
test_ppo_speed[True-backward]	10.2798ms	9.8255ms	101.7755 Ops/s	101.2624 Ops/s	$\color{#35bf28}+0.51\%$
test_ppo_speed[reduce-overhead-None]	4.4896ms	3.7825ms	264.3755 Ops/s	265.3683 Ops/s	$\color{#d91a1a}-0.37\%$
test_ppo_speed[reduce-overhead-backward]	10.2887ms	9.8621ms	101.3987 Ops/s	103.4452 Ops/s	$\color{#d91a1a}-1.98\%$
test_reinforce_speed[False-None]	7.4287ms	6.6629ms	150.0859 Ops/s	149.0026 Ops/s	$\color{#35bf28}+0.73\%$
test_reinforce_speed[False-backward]	13.2482ms	10.1658ms	98.3689 Ops/s	98.7790 Ops/s	$\color{#d91a1a}-0.42\%$
test_reinforce_speed[True-None]	3.3281ms	2.6472ms	377.7578 Ops/s	371.2639 Ops/s	$\color{#35bf28}+1.75\%$
test_reinforce_speed[True-backward]	9.4411ms	8.6731ms	115.2992 Ops/s	113.6375 Ops/s	$\color{#35bf28}+1.46\%$
test_reinforce_speed[reduce-overhead-None]	3.2013ms	2.6430ms	378.3510 Ops/s	364.7896 Ops/s	$\color{#35bf28}+3.72\%$
test_reinforce_speed[reduce-overhead-backward]	9.1641ms	8.6222ms	115.9791 Ops/s	114.9219 Ops/s	$\color{#35bf28}+0.92\%$
test_iql_speed[False-None]	35.3547ms	32.1442ms	31.1098 Ops/s	29.9856 Ops/s	$\color{#35bf28}+3.75\%$
test_iql_speed[False-backward]	46.7584ms	45.0488ms	22.1982 Ops/s	21.3762 Ops/s	$\color{#35bf28}+3.85\%$
test_iql_speed[True-None]	11.4830ms	10.7528ms	92.9987 Ops/s	91.4041 Ops/s	$\color{#35bf28}+1.74\%$
test_iql_speed[True-backward]	22.9818ms	21.7866ms	45.8997 Ops/s	44.8201 Ops/s	$\color{#35bf28}+2.41\%$
test_iql_speed[reduce-overhead-None]	12.4987ms	10.7572ms	92.9607 Ops/s	91.0896 Ops/s	$\color{#35bf28}+2.05\%$
test_iql_speed[reduce-overhead-backward]	22.8505ms	21.8415ms	45.7845 Ops/s	45.1813 Ops/s	$\color{#35bf28}+1.33\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.5130ms	4.9302ms	202.8307 Ops/s	193.6067 Ops/s	$\color{#35bf28}+4.76\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7157ms	0.5030ms	1.9879 KOps/s	1.8848 KOps/s	$\textbf{\color{#35bf28}+5.47\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.8093ms	0.4818ms	2.0755 KOps/s	2.0321 KOps/s	$\color{#35bf28}+2.13\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.2964ms	4.8575ms	205.8661 Ops/s	200.9711 Ops/s	$\color{#35bf28}+2.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.2807ms	0.4956ms	2.0177 KOps/s	1.8954 KOps/s	$\textbf{\color{#35bf28}+6.45\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7266ms	0.4731ms	2.1139 KOps/s	2.1217 KOps/s	$\color{#d91a1a}-0.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.8946ms	1.6055ms	622.8707 Ops/s	606.8028 Ops/s	$\color{#35bf28}+2.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.3101ms	1.5660ms	638.5650 Ops/s	624.3099 Ops/s	$\color{#35bf28}+2.28\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.2357ms	4.9078ms	203.7576 Ops/s	197.3372 Ops/s	$\color{#35bf28}+3.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1972ms	0.6366ms	1.5708 KOps/s	1.5371 KOps/s	$\color{#35bf28}+2.19\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0022ms	0.6138ms	1.6291 KOps/s	1.6052 KOps/s	$\color{#35bf28}+1.49\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.0559ms	4.7557ms	210.2756 Ops/s	205.8243 Ops/s	$\color{#35bf28}+2.16\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.8145ms	0.5059ms	1.9767 KOps/s	1.9108 KOps/s	$\color{#35bf28}+3.45\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.8234ms	0.4869ms	2.0536 KOps/s	2.0725 KOps/s	$\color{#d91a1a}-0.91\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.4838ms	4.7859ms	208.9453 Ops/s	206.3298 Ops/s	$\color{#35bf28}+1.27\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.1138ms	0.4893ms	2.0436 KOps/s	2.0135 KOps/s	$\color{#35bf28}+1.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6854ms	0.4679ms	2.1371 KOps/s	2.0662 KOps/s	$\color{#35bf28}+3.43\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.1324ms	4.8640ms	205.5909 Ops/s	201.7711 Ops/s	$\color{#35bf28}+1.89\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1953ms	0.6492ms	1.5404 KOps/s	1.5393 KOps/s	$\color{#35bf28}+0.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8602ms	0.6122ms	1.6334 KOps/s	1.5736 KOps/s	$\color{#35bf28}+3.80\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	5.5007ms	4.2277ms	236.5342 Ops/s	37.1477 Ops/s	$\textbf{\color{#35bf28}+536.74\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	8.3579ms	2.4812ms	403.0309 Ops/s	427.5043 Ops/s	$\textbf{\color{#d91a1a}-5.72\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.6762ms	1.3020ms	768.0231 Ops/s	780.0624 Ops/s	$\color{#d91a1a}-1.54\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.4446s	13.0612ms	76.5628 Ops/s	223.4738 Ops/s	$\textbf{\color{#d91a1a}-65.74\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	3.4176ms	2.1719ms	460.4295 Ops/s	396.6475 Ops/s	$\textbf{\color{#35bf28}+16.08\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.9964ms	1.3887ms	720.1010 Ops/s	789.0146 Ops/s	$\textbf{\color{#d91a1a}-8.73\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	5.9587ms	4.4337ms	225.5467 Ops/s	230.4346 Ops/s	$\color{#d91a1a}-2.12\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	6.7544ms	2.4335ms	410.9298 Ops/s	404.1934 Ops/s	$\color{#35bf28}+1.67\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	5.7568ms	1.4768ms	677.1368 Ops/s	652.9999 Ops/s	$\color{#35bf28}+3.70\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	12.7890ms	11.4606ms	87.2553 Ops/s	79.9413 Ops/s	$\textbf{\color{#35bf28}+9.15\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	16.0692ms	14.9115ms	67.0621 Ops/s	64.3777 Ops/s	$\color{#35bf28}+4.17\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	21.8820ms	20.1441ms	49.6423 Ops/s	47.9606 Ops/s	$\color{#35bf28}+3.51\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	16.2705ms	15.2476ms	65.5840 Ops/s	64.1964 Ops/s	$\color{#35bf28}+2.16\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	21.4038ms	20.2038ms	49.4956 Ops/s	49.3275 Ops/s	$\color{#35bf28}+0.34\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	17.5656ms	16.4531ms	60.7788 Ops/s	58.6456 Ops/s	$\color{#35bf28}+3.64\%$