Skip to content

[Feature] Trainer Algorithms - Configuration System #3189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 21 commits into
base: gh/vmoens/152/base
Choose a base branch
from
Open

[Feature] Trainer Algorithms - Configuration System #3189

wants to merge 21 commits into from
+401 −1

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 14, 2025 •
edited
Loading

@vmoens
Update
40c8b63 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Oct 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3189

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 10 Unrelated Failures

As of commit 1cb3227 with merge base 92c20cd (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 14, 2025
Merged
Merged
Merged
Merged
Merged
Open
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2025
vmoens added 7 commits October 16, 2025 15:00
@vmoens
Update
74bfb5a 
[ghstack-poisoned]
@vmoens
Update
bfd08f5 
[ghstack-poisoned]
@vmoens
Update
eb85308 
[ghstack-poisoned]
@vmoens
Update
0d1f7ce 
[ghstack-poisoned]
@vmoens
Update
351326c 
[ghstack-poisoned]
@vmoens
Update
bb95ddb 
[ghstack-poisoned]
@vmoens
Update
5fcf578 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 18, 2025
Open
vmoens added 4 commits October 18, 2025 19:00
@vmoens
Update
67659d3 
[ghstack-poisoned]
@vmoens
Update
1f9b4c8 
[ghstack-poisoned]
@vmoens
Update
36e3097 
[ghstack-poisoned]
@vmoens
Update
5b3b689 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Oct 20, 2025
@vmoens
Update
ddb96c1 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
920f08a 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
e5e892f 
[ghstack-poisoned]
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}19$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 82.7777μs 81.1660μs 12.3204 KOps/s 12.2261 KOps/s $\color{#35bf28}+0.77\%$
test_tensor_to_bytestream_speed[torch.save] 0.1406ms 0.1392ms 7.1817 KOps/s 7.1377 KOps/s $\color{#35bf28}+0.62\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1197s 0.1190s 8.4058 Ops/s 8.3572 Ops/s $\color{#35bf28}+0.58\%$
test_tensor_to_bytestream_speed[numpy] 2.7951μs 2.7892μs 358.5288 KOps/s 350.1754 KOps/s $\color{#35bf28}+2.39\%$
test_tensor_to_bytestream_speed[safetensors] 41.9715μs 41.7377μs 23.9592 KOps/s 24.1916 KOps/s $\color{#d91a1a}-0.96\%$
test_simple 0.5439s 0.5429s 1.8419 Ops/s 1.7520 Ops/s $\textbf{\color{#35bf28}+5.13\%}$
test_transformed 1.2134s 1.1206s 0.8924 Ops/s 0.8827 Ops/s $\color{#35bf28}+1.09\%$
test_serial 1.6447s 1.6405s 0.6096 Ops/s 0.5927 Ops/s $\color{#35bf28}+2.85\%$
test_parallel 1.1801s 1.0967s 0.9118 Ops/s 0.9193 Ops/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[True-True-True-True-True] 0.1706ms 43.0086μs 23.2511 KOps/s 23.1753 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-True-True-True-False] 0.4281ms 24.5923μs 40.6632 KOps/s 39.6455 KOps/s $\color{#35bf28}+2.57\%$
test_step_mdp_speed[True-True-True-False-True] 0.4524ms 24.5409μs 40.7483 KOps/s 41.1850 KOps/s $\color{#d91a1a}-1.06\%$
test_step_mdp_speed[True-True-True-False-False] 43.7120μs 13.6100μs 73.4755 KOps/s 72.9546 KOps/s $\color{#35bf28}+0.71\%$
test_step_mdp_speed[True-True-False-True-True] 0.4696ms 46.0288μs 21.7255 KOps/s 21.8066 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-True-False-True-False] 0.4433ms 27.2260μs 36.7296 KOps/s 37.9913 KOps/s $\color{#d91a1a}-3.32\%$
test_step_mdp_speed[True-True-False-False-True] 63.9030μs 26.6017μs 37.5915 KOps/s 36.6169 KOps/s $\color{#35bf28}+2.66\%$
test_step_mdp_speed[True-True-False-False-False] 47.1620μs 16.1399μs 61.9584 KOps/s 61.4186 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[True-False-True-True-True] 0.4612ms 49.2858μs 20.2898 KOps/s 20.2212 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-False-True-True-False] 0.4538ms 29.8909μs 33.4550 KOps/s 33.1781 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-False-True-False-True] 55.9520μs 27.1434μs 36.8413 KOps/s 36.9615 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-False-True-False-False] 0.4227ms 16.3261μs 61.2517 KOps/s 60.8903 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-False-False-True-True] 0.4563ms 51.8500μs 19.2864 KOps/s 18.9433 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[True-False-False-True-False] 0.4350ms 32.3776μs 30.8856 KOps/s 30.5522 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[True-False-False-False-True] 62.8520μs 29.7930μs 33.5650 KOps/s 33.6578 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-False-False-False-False] 0.4369ms 18.8471μs 53.0585 KOps/s 53.1807 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-True-True-True-True] 0.4577ms 49.6783μs 20.1295 KOps/s 20.5650 KOps/s $\color{#d91a1a}-2.12\%$
test_step_mdp_speed[False-True-True-True-False] 0.4366ms 29.9153μs 33.4277 KOps/s 33.4913 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[False-True-True-False-True] 2.3971ms 31.7522μs 31.4939 KOps/s 32.0107 KOps/s $\color{#d91a1a}-1.61\%$
test_step_mdp_speed[False-True-True-False-False] 0.4471ms 17.8488μs 56.0263 KOps/s 55.5712 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-True-False-True-True] 99.0640μs 51.8618μs 19.2820 KOps/s 19.7562 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[False-True-False-True-False] 91.5440μs 32.1932μs 31.0624 KOps/s 30.6137 KOps/s $\color{#35bf28}+1.47\%$
test_step_mdp_speed[False-True-False-False-True] 63.6530μs 33.7094μs 29.6653 KOps/s 29.4971 KOps/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[False-True-False-False-False] 55.3320μs 21.0695μs 47.4619 KOps/s 48.6289 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[False-False-True-True-True] 92.7640μs 53.9296μs 18.5427 KOps/s 18.6715 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-False-True-True-False] 66.1330μs 35.2647μs 28.3570 KOps/s 28.8505 KOps/s $\color{#d91a1a}-1.71\%$
test_step_mdp_speed[False-False-True-False-True] 73.4830μs 34.3535μs 29.1092 KOps/s 29.4565 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[False-False-True-False-False] 57.1520μs 20.7177μs 48.2679 KOps/s 48.4744 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[False-False-False-True-True] 0.1247ms 56.4941μs 17.7010 KOps/s 17.6632 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-False-False-True-False] 66.0230μs 37.8018μs 26.4538 KOps/s 26.5408 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-False-False-False-True] 77.6030μs 35.7819μs 27.9471 KOps/s 27.7001 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-False-False-False-False] 55.8820μs 23.2878μs 42.9409 KOps/s 43.8463 KOps/s $\color{#d91a1a}-2.07\%$
test_values[generalized_advantage_estimate-True-True] 10.6027ms 10.3382ms 96.7286 Ops/s 96.0873 Ops/s $\color{#35bf28}+0.67\%$
test_values[vec_generalized_advantage_estimate-True-True] 15.1600ms 11.1160ms 89.9601 Ops/s 57.3368 Ops/s $\textbf{\color{#35bf28}+56.90\%}$
test_values[td0_return_estimate-False-False] 0.2409ms 0.1314ms 7.6106 KOps/s 7.7061 KOps/s $\color{#d91a1a}-1.24\%$
test_values[td1_return_estimate-False-False] 28.8546ms 28.5724ms 34.9988 Ops/s 35.5255 Ops/s $\color{#d91a1a}-1.48\%$
test_values[vec_td1_return_estimate-False-False] 11.3897ms 11.1808ms 89.4392 Ops/s 56.9853 Ops/s $\textbf{\color{#35bf28}+56.95\%}$
test_values[td_lambda_return_estimate-True-False] 43.1514ms 42.4144ms 23.5769 Ops/s 23.8829 Ops/s $\color{#d91a1a}-1.28\%$
test_values[vec_td_lambda_return_estimate-True-False] 11.4184ms 11.1619ms 89.5901 Ops/s 56.8430 Ops/s $\textbf{\color{#35bf28}+57.61\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.0335ms 8.9269ms 112.0211 Ops/s 112.5268 Ops/s $\color{#d91a1a}-0.45\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7060ms 1.5385ms 649.9634 Ops/s 625.4777 Ops/s $\color{#35bf28}+3.91\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4578ms 0.4144ms 2.4130 KOps/s 2.4106 KOps/s $\color{#35bf28}+0.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 30.1460ms 26.0943ms 38.3225 Ops/s 28.6463 Ops/s $\textbf{\color{#35bf28}+33.78\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1180ms 1.7229ms 580.4199 Ops/s 580.4008 Ops/s $+0.00\%$
test_dqn_speed[False-None] 1.6133ms 1.4123ms 708.0423 Ops/s 708.2596 Ops/s $\color{#d91a1a}-0.03\%$
test_dqn_speed[False-backward] 2.2400ms 1.9409ms 515.2172 Ops/s 520.0804 Ops/s $\color{#d91a1a}-0.94\%$
test_dqn_speed[True-None] 1.2590ms 0.5265ms 1.8992 KOps/s 1.8741 KOps/s $\color{#35bf28}+1.34\%$
test_dqn_speed[True-backward] 1.0180ms 0.9720ms 1.0288 KOps/s 1.0161 KOps/s $\color{#35bf28}+1.25\%$
test_dqn_speed[reduce-overhead-None] 0.6489ms 0.5118ms 1.9538 KOps/s 1.9468 KOps/s $\color{#35bf28}+0.36\%$
test_dqn_speed[reduce-overhead-backward] 0.9820ms 0.9544ms 1.0478 KOps/s 1.0423 KOps/s $\color{#35bf28}+0.53\%$
test_ddpg_speed[False-None] 3.2743ms 2.9007ms 344.7430 Ops/s 350.6422 Ops/s $\color{#d91a1a}-1.68\%$
test_ddpg_speed[False-backward] 4.2187ms 4.1248ms 242.4334 Ops/s 244.3138 Ops/s $\color{#d91a1a}-0.77\%$
test_ddpg_speed[True-None] 1.8191ms 1.3939ms 717.4279 Ops/s 704.4952 Ops/s $\color{#35bf28}+1.84\%$
test_ddpg_speed[True-backward] 2.4259ms 2.3790ms 420.3460 Ops/s 397.8634 Ops/s $\textbf{\color{#35bf28}+5.65\%}$
test_ddpg_speed[reduce-overhead-None] 1.4894ms 1.3834ms 722.8745 Ops/s 718.1654 Ops/s $\color{#35bf28}+0.66\%$
test_ddpg_speed[reduce-overhead-backward] 2.4195ms 2.3681ms 422.2742 Ops/s 419.1791 Ops/s $\color{#35bf28}+0.74\%$
test_sac_speed[False-None] 8.4585ms 7.9085ms 126.4456 Ops/s 127.3629 Ops/s $\color{#d91a1a}-0.72\%$
test_sac_speed[False-backward] 11.6716ms 11.2053ms 89.2432 Ops/s 89.7373 Ops/s $\color{#d91a1a}-0.55\%$
test_sac_speed[True-None] 2.2800ms 2.1206ms 471.5559 Ops/s 472.9185 Ops/s $\color{#d91a1a}-0.29\%$
test_sac_speed[True-backward] 4.1757ms 4.0610ms 246.2421 Ops/s 245.8042 Ops/s $\color{#35bf28}+0.18\%$
test_sac_speed[reduce-overhead-None] 2.6077ms 2.1302ms 469.4418 Ops/s 447.4015 Ops/s $\color{#35bf28}+4.93\%$
test_sac_speed[reduce-overhead-backward] 4.1398ms 4.0548ms 246.6198 Ops/s 220.3284 Ops/s $\textbf{\color{#35bf28}+11.93\%}$
test_redq_speed[False-None] 10.9293ms 10.3806ms 96.3339 Ops/s 94.7355 Ops/s $\color{#35bf28}+1.69\%$
test_redq_speed[False-backward] 18.5032ms 17.8813ms 55.9242 Ops/s 54.3239 Ops/s $\color{#35bf28}+2.95\%$
test_redq_speed[True-None] 4.5861ms 4.3686ms 228.9085 Ops/s 232.3328 Ops/s $\color{#d91a1a}-1.47\%$
test_redq_speed[True-backward] 10.2701ms 9.9079ms 100.9298 Ops/s 103.5926 Ops/s $\color{#d91a1a}-2.57\%$
test_redq_speed[reduce-overhead-None] 4.8519ms 4.3202ms 231.4684 Ops/s 232.4497 Ops/s $\color{#d91a1a}-0.42\%$
test_redq_speed[reduce-overhead-backward] 10.5507ms 9.9700ms 100.3012 Ops/s 100.6907 Ops/s $\color{#d91a1a}-0.39\%$
test_redq_deprec_speed[False-None] 11.5358ms 11.0833ms 90.2256 Ops/s 90.3055 Ops/s $\color{#d91a1a}-0.09\%$
test_redq_deprec_speed[False-backward] 17.6920ms 16.0998ms 62.1126 Ops/s 62.0632 Ops/s $\color{#35bf28}+0.08\%$
test_redq_deprec_speed[True-None] 4.1220ms 3.6324ms 275.3025 Ops/s 267.3334 Ops/s $\color{#35bf28}+2.98\%$
test_redq_deprec_speed[True-backward] 7.9569ms 7.7092ms 129.7157 Ops/s 128.7160 Ops/s $\color{#35bf28}+0.78\%$
test_redq_deprec_speed[reduce-overhead-None] 3.7414ms 3.5805ms 279.2916 Ops/s 264.7277 Ops/s $\textbf{\color{#35bf28}+5.50\%}$
test_redq_deprec_speed[reduce-overhead-backward] 7.8139ms 7.6607ms 130.5365 Ops/s 128.2666 Ops/s $\color{#35bf28}+1.77\%$
test_td3_speed[False-None] 8.2974ms 7.9350ms 126.0232 Ops/s 126.3218 Ops/s $\color{#d91a1a}-0.24\%$
test_td3_speed[False-backward] 11.3825ms 10.8527ms 92.1432 Ops/s 92.2878 Ops/s $\color{#d91a1a}-0.16\%$
test_td3_speed[True-None] 1.8433ms 1.7949ms 557.1272 Ops/s 551.6778 Ops/s $\color{#35bf28}+0.99\%$
test_td3_speed[True-backward] 3.7117ms 3.5914ms 278.4454 Ops/s 257.1948 Ops/s $\textbf{\color{#35bf28}+8.26\%}$
test_td3_speed[reduce-overhead-None] 1.8322ms 1.7739ms 563.7376 Ops/s 581.3039 Ops/s $\color{#d91a1a}-3.02\%$
test_td3_speed[reduce-overhead-backward] 3.8127ms 3.6723ms 272.3083 Ops/s 241.5691 Ops/s $\textbf{\color{#35bf28}+12.72\%}$
test_cql_speed[False-None] 30.8036ms 26.1615ms 38.2241 Ops/s 38.3960 Ops/s $\color{#d91a1a}-0.45\%$
test_cql_speed[False-backward] 40.9896ms 36.7633ms 27.2010 Ops/s 28.0286 Ops/s $\color{#d91a1a}-2.95\%$
test_cql_speed[True-None] 12.8087ms 12.3710ms 80.8343 Ops/s 77.2202 Ops/s $\color{#35bf28}+4.68\%$
test_cql_speed[True-backward] 18.8420ms 18.4160ms 54.3006 Ops/s 55.0119 Ops/s $\color{#d91a1a}-1.29\%$
test_cql_speed[reduce-overhead-None] 13.0402ms 12.4742ms 80.1654 Ops/s 79.6299 Ops/s $\color{#35bf28}+0.67\%$
test_cql_speed[reduce-overhead-backward] 18.9664ms 18.5858ms 53.8046 Ops/s 55.1271 Ops/s $\color{#d91a1a}-2.40\%$
test_a2c_speed[False-None] 5.8674ms 5.4387ms 183.8665 Ops/s 179.0839 Ops/s $\color{#35bf28}+2.67\%$
test_a2c_speed[False-backward] 12.2567ms 11.9561ms 83.6390 Ops/s 80.7260 Ops/s $\color{#35bf28}+3.61\%$
test_a2c_speed[True-None] 3.8875ms 3.7180ms 268.9591 Ops/s 262.0984 Ops/s $\color{#35bf28}+2.62\%$
test_a2c_speed[True-backward] 8.9905ms 8.5784ms 116.5713 Ops/s 113.3952 Ops/s $\color{#35bf28}+2.80\%$
test_a2c_speed[reduce-overhead-None] 3.8464ms 3.6987ms 270.3623 Ops/s 264.5311 Ops/s $\color{#35bf28}+2.20\%$
test_a2c_speed[reduce-overhead-backward] 9.1372ms 8.8825ms 112.5807 Ops/s 111.7563 Ops/s $\color{#35bf28}+0.74\%$
test_ppo_speed[False-None] 6.0154ms 5.8224ms 171.7510 Ops/s 165.9313 Ops/s $\color{#35bf28}+3.51\%$
test_ppo_speed[False-backward] 12.9064ms 12.6389ms 79.1205 Ops/s 77.6774 Ops/s $\color{#35bf28}+1.86\%$
test_ppo_speed[True-None] 4.1488ms 3.6230ms 276.0139 Ops/s 272.3504 Ops/s $\color{#35bf28}+1.35\%$
test_ppo_speed[True-backward] 8.7143ms 8.5528ms 116.9211 Ops/s 107.9130 Ops/s $\textbf{\color{#35bf28}+8.35\%}$
test_ppo_speed[reduce-overhead-None] 3.7671ms 3.5928ms 278.3310 Ops/s 266.8167 Ops/s $\color{#35bf28}+4.32\%$
test_ppo_speed[reduce-overhead-backward] 9.1772ms 8.7570ms 114.1948 Ops/s 111.1002 Ops/s $\color{#35bf28}+2.79\%$
test_reinforce_speed[False-None] 5.0537ms 4.5631ms 219.1515 Ops/s 217.0484 Ops/s $\color{#35bf28}+0.97\%$
test_reinforce_speed[False-backward] 7.7343ms 7.4885ms 133.5380 Ops/s 132.4782 Ops/s $\color{#35bf28}+0.80\%$
test_reinforce_speed[True-None] 3.3870ms 2.8741ms 347.9365 Ops/s 343.9532 Ops/s $\color{#35bf28}+1.16\%$
test_reinforce_speed[True-backward] 7.9716ms 7.7393ms 129.2111 Ops/s 127.8355 Ops/s $\color{#35bf28}+1.08\%$
test_reinforce_speed[reduce-overhead-None] 3.0155ms 2.8532ms 350.4856 Ops/s 347.2106 Ops/s $\color{#35bf28}+0.94\%$
test_reinforce_speed[reduce-overhead-backward] 8.2209ms 7.9330ms 126.0551 Ops/s 124.1933 Ops/s $\color{#35bf28}+1.50\%$
test_iql_speed[False-None] 20.6819ms 20.0462ms 49.8849 Ops/s 47.3547 Ops/s $\textbf{\color{#35bf28}+5.34\%}$
test_iql_speed[False-backward] 31.5671ms 30.7487ms 32.5217 Ops/s 32.2146 Ops/s $\color{#35bf28}+0.95\%$
test_iql_speed[True-None] 8.6914ms 8.4974ms 117.6834 Ops/s 117.0241 Ops/s $\color{#35bf28}+0.56\%$
test_iql_speed[True-backward] 17.3645ms 16.9045ms 59.1558 Ops/s 57.3857 Ops/s $\color{#35bf28}+3.08\%$
test_iql_speed[reduce-overhead-None] 8.9594ms 8.5801ms 116.5491 Ops/s 115.1117 Ops/s $\color{#35bf28}+1.25\%$
test_iql_speed[reduce-overhead-backward] 17.7836ms 17.3506ms 57.6348 Ops/s 57.9750 Ops/s $\color{#d91a1a}-0.59\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.4889ms 5.8991ms 169.5188 Ops/s 172.2512 Ops/s $\color{#d91a1a}-1.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5888ms 0.3543ms 2.8222 KOps/s 3.6087 KOps/s $\textbf{\color{#d91a1a}-21.80\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5635ms 0.3077ms 3.2494 KOps/s 3.3079 KOps/s $\color{#d91a1a}-1.77\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9667ms 5.6358ms 177.4373 Ops/s 181.4237 Ops/s $\color{#d91a1a}-2.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.1012ms 0.4054ms 2.4669 KOps/s 2.6658 KOps/s $\textbf{\color{#d91a1a}-7.46\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4995ms 0.2929ms 3.4138 KOps/s 2.7967 KOps/s $\textbf{\color{#35bf28}+22.06\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7272ms 1.4201ms 704.1944 Ops/s 651.1935 Ops/s $\textbf{\color{#35bf28}+8.14\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5712ms 1.3101ms 763.3269 Ops/s 703.3366 Ops/s $\textbf{\color{#35bf28}+8.53\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.8057ms 5.9723ms 167.4393 Ops/s 176.5644 Ops/s $\textbf{\color{#d91a1a}-5.17\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.0717ms 0.4911ms 2.0361 KOps/s 2.2575 KOps/s $\textbf{\color{#d91a1a}-9.81\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7604ms 0.5172ms 1.9335 KOps/s 2.4022 KOps/s $\textbf{\color{#d91a1a}-19.51\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9198ms 5.6637ms 176.5629 Ops/s 179.6203 Ops/s $\color{#d91a1a}-1.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.6721ms 0.3549ms 2.8175 KOps/s 849.5315 Ops/s $\textbf{\color{#35bf28}+231.65\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4775ms 0.2826ms 3.5391 KOps/s 2.4877 KOps/s $\textbf{\color{#35bf28}+42.26\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.7841ms 5.5583ms 179.9113 Ops/s 177.1577 Ops/s $\color{#35bf28}+1.55\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0003ms 0.3303ms 3.0277 KOps/s 3.5388 KOps/s $\textbf{\color{#d91a1a}-14.44\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4911ms 0.3170ms 3.1543 KOps/s 3.8272 KOps/s $\textbf{\color{#d91a1a}-17.58\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1716ms 5.7463ms 174.0250 Ops/s 172.7372 Ops/s $\color{#35bf28}+0.75\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2639ms 0.4652ms 2.1497 KOps/s 2.1920 KOps/s $\color{#d91a1a}-1.93\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7255ms 0.4508ms 2.2181 KOps/s 2.2225 KOps/s $\color{#d91a1a}-0.20\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.6701ms 5.0610ms 197.5908 Ops/s 195.3332 Ops/s $\color{#35bf28}+1.16\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 7.5069ms 2.3211ms 430.8351 Ops/s 439.4758 Ops/s $\color{#d91a1a}-1.97\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.2925ms 1.2553ms 796.6492 Ops/s 839.8820 Ops/s $\textbf{\color{#d91a1a}-5.15\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5087s 15.2135ms 65.7312 Ops/s 55.9126 Ops/s $\textbf{\color{#35bf28}+17.56\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.5596ms 2.0285ms 492.9742 Ops/s 512.0283 Ops/s $\color{#d91a1a}-3.72\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.2833ms 1.1530ms 867.2720 Ops/s 823.0308 Ops/s $\textbf{\color{#35bf28}+5.38\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.8934ms 5.2808ms 189.3643 Ops/s 184.7654 Ops/s $\color{#35bf28}+2.49\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.8809ms 2.1998ms 454.5941 Ops/s 445.9690 Ops/s $\color{#35bf28}+1.93\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.9509ms 1.3553ms 737.8581 Ops/s 729.7457 Ops/s $\color{#35bf28}+1.11\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 34.5818ms 32.4408ms 30.8254 Ops/s 30.0265 Ops/s $\color{#35bf28}+2.66\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.6701ms 17.3246ms 57.7214 Ops/s 57.5572 Ops/s $\color{#35bf28}+0.29\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 35.5846ms 33.4139ms 29.9277 Ops/s 29.0074 Ops/s $\color{#35bf28}+3.17\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.5671ms 17.7180ms 56.4399 Ops/s 56.5557 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 36.2616ms 34.9292ms 28.6293 Ops/s 28.0475 Ops/s $\color{#35bf28}+2.07\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 19.9100ms 18.8830ms 52.9576 Ops/s 51.5339 Ops/s $\color{#35bf28}+2.76\%$

vmoens added 6 commits October 19, 2025 20:56
@vmoens
Update
1591d0e 
[ghstack-poisoned]
@vmoens
Update
2c479eb 
[ghstack-poisoned]
@vmoens
Update
1d4c308 
[ghstack-poisoned]
@vmoens
Update
3092841 
[ghstack-poisoned]
@vmoens
Update
323ed9d 
[ghstack-poisoned]
@vmoens
Update
1cb3227 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens