Skip to content

[Feature] SAC Trainer #3191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 19 commits into
base: gh/vmoens/154/base
Choose a base branch
from
Open

[Feature] SAC Trainer #3191

wants to merge 19 commits into from
+4 −0

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 14, 2025 •
edited
Loading

@vmoens
Update
4d6443e 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Oct 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3191

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 6 New Failures, 10 Unrelated Failures

As of commit dfaa0fa with merge base e7ec9c3 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 14, 2025
Merged
Merged
Merged
Merged
Merged
Open
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2025
vmoens added 7 commits October 16, 2025 15:00
@vmoens
Update
ec4e903 
[ghstack-poisoned]
@vmoens
Update
58b15b5 
[ghstack-poisoned]
@vmoens
Update
3e42d1f 
[ghstack-poisoned]
@vmoens
Update
3ffb582 
[ghstack-poisoned]
@vmoens
Update
b0b8b39 
[ghstack-poisoned]
@vmoens
Update
3b61b27 
[ghstack-poisoned]
@vmoens
Update
fd9dec0 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 18, 2025
Open
vmoens added 4 commits October 18, 2025 19:00
@vmoens
Update
26026d4 
[ghstack-poisoned]
@vmoens
Update
a411944 
[ghstack-poisoned]
@vmoens
Update
7739792 
[ghstack-poisoned]
@vmoens
Update
f9a29c8 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Oct 20, 2025
@vmoens
Update
579b61b 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
a3a0abc 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Open
@vmoens
Update
072605a 
[ghstack-poisoned]
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}17$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 83.3236μs 81.3857μs 12.2872 KOps/s 12.1979 KOps/s $\color{#35bf28}+0.73\%$
test_tensor_to_bytestream_speed[torch.save] 0.1402ms 0.1395ms 7.1674 KOps/s 6.9899 KOps/s $\color{#35bf28}+2.54\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1288s 0.1282s 7.8032 Ops/s 7.7270 Ops/s $\color{#35bf28}+0.99\%$
test_tensor_to_bytestream_speed[numpy] 2.8271μs 2.8242μs 354.0878 KOps/s 339.2661 KOps/s $\color{#35bf28}+4.37\%$
test_tensor_to_bytestream_speed[safetensors] 42.5564μs 42.3262μs 23.6260 KOps/s 23.0134 KOps/s $\color{#35bf28}+2.66\%$
test_simple 0.5553s 0.5540s 1.8049 Ops/s 1.7118 Ops/s $\textbf{\color{#35bf28}+5.44\%}$
test_transformed 1.1228s 1.1219s 0.8914 Ops/s 0.8614 Ops/s $\color{#35bf28}+3.48\%$
test_serial 1.6807s 1.6748s 0.5971 Ops/s 0.5817 Ops/s $\color{#35bf28}+2.65\%$
test_parallel 1.1841s 1.1065s 0.9038 Ops/s 0.8929 Ops/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-True-True-True-True] 0.1689ms 44.6101μs 22.4165 KOps/s 21.8490 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[True-True-True-True-False] 99.6520μs 25.7045μs 38.9036 KOps/s 38.9651 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[True-True-True-False-True] 54.3510μs 25.5042μs 39.2093 KOps/s 39.2560 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-True-True-False-False] 44.0610μs 14.0004μs 71.4266 KOps/s 71.3453 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-True-False-True-True] 89.4420μs 48.9525μs 20.4280 KOps/s 20.4037 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-True-False-True-False] 65.4010μs 28.1554μs 35.5172 KOps/s 35.0921 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-True-False-False-True] 56.2610μs 28.2087μs 35.4501 KOps/s 35.1501 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-True-False-False-False] 49.4910μs 16.7998μs 59.5245 KOps/s 58.7234 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[True-False-True-True-True] 86.1620μs 52.7125μs 18.9708 KOps/s 19.3832 KOps/s $\color{#d91a1a}-2.13\%$
test_step_mdp_speed[True-False-True-True-False] 63.1220μs 31.2876μs 31.9615 KOps/s 31.9049 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-False-True-False-True] 67.2410μs 28.6377μs 34.9190 KOps/s 34.9280 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[True-False-True-False-False] 0.1271ms 16.4140μs 60.9237 KOps/s 59.1975 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[True-False-False-True-True] 0.4645ms 54.3271μs 18.4070 KOps/s 18.4107 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-False-False-True-False] 0.4377ms 34.2628μs 29.1862 KOps/s 29.4962 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[True-False-False-False-True] 65.0710μs 30.3386μs 32.9613 KOps/s 32.2309 KOps/s $\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-False-False-False-False] 0.4270ms 19.7189μs 50.7129 KOps/s 51.1340 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[False-True-True-True-True] 0.4550ms 51.8096μs 19.3014 KOps/s 19.3410 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-True-True-True-False] 0.4293ms 31.4907μs 31.7554 KOps/s 31.9322 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-True-True-False-True] 2.3628ms 32.9141μs 30.3821 KOps/s 30.6386 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[False-True-True-False-False] 0.4419ms 18.5555μs 53.8923 KOps/s 53.6971 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-True-False-True-True] 0.4714ms 54.1357μs 18.4721 KOps/s 18.5746 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-True-False-True-False] 0.4425ms 34.2373μs 29.2080 KOps/s 29.4540 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[False-True-False-False-True] 74.9210μs 35.0364μs 28.5417 KOps/s 28.6540 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[False-True-False-False-False] 0.4308ms 21.3065μs 46.9340 KOps/s 46.7953 KOps/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[False-False-True-True-True] 0.4692ms 57.9438μs 17.2581 KOps/s 17.3914 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[False-False-True-True-False] 0.4435ms 36.8746μs 27.1190 KOps/s 27.2666 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[False-False-True-False-True] 0.4372ms 34.9241μs 28.6335 KOps/s 28.5367 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[False-False-True-False-False] 52.2920μs 21.3406μs 46.8591 KOps/s 46.6152 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-False-False-True-True] 0.4738ms 59.6247μs 16.7716 KOps/s 16.9260 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[False-False-False-True-False] 0.4436ms 39.2392μs 25.4847 KOps/s 25.2044 KOps/s $\color{#35bf28}+1.11\%$
test_step_mdp_speed[False-False-False-False-True] 0.4389ms 36.2312μs 27.6005 KOps/s 26.7273 KOps/s $\color{#35bf28}+3.27\%$
test_step_mdp_speed[False-False-False-False-False] 41.8610μs 24.0428μs 41.5925 KOps/s 42.3450 KOps/s $\color{#d91a1a}-1.78\%$
test_values[generalized_advantage_estimate-True-True] 10.6735ms 10.2227ms 97.8216 Ops/s 94.0274 Ops/s $\color{#35bf28}+4.04\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.8686ms 17.5107ms 57.1080 Ops/s 90.5994 Ops/s $\textbf{\color{#d91a1a}-36.97\%}$
test_values[td0_return_estimate-False-False] 0.2290ms 0.1339ms 7.4663 KOps/s 7.7914 KOps/s $\color{#d91a1a}-4.17\%$
test_values[td1_return_estimate-False-False] 29.0540ms 28.2318ms 35.4211 Ops/s 34.3629 Ops/s $\color{#35bf28}+3.08\%$
test_values[vec_td1_return_estimate-False-False] 18.9892ms 17.6230ms 56.7441 Ops/s 89.1800 Ops/s $\textbf{\color{#d91a1a}-36.37\%}$
test_values[td_lambda_return_estimate-True-False] 43.0954ms 42.4248ms 23.5711 Ops/s 22.8997 Ops/s $\color{#35bf28}+2.93\%$
test_values[vec_td_lambda_return_estimate-True-False] 18.1695ms 17.6733ms 56.5827 Ops/s 89.4053 Ops/s $\textbf{\color{#d91a1a}-36.71\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.4053ms 8.8606ms 112.8596 Ops/s 109.0377 Ops/s $\color{#35bf28}+3.51\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7924ms 1.5084ms 662.9544 Ops/s 649.9482 Ops/s $\color{#35bf28}+2.00\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8367ms 0.4174ms 2.3957 KOps/s 2.3716 KOps/s $\color{#35bf28}+1.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.7884ms 33.9645ms 29.4425 Ops/s 37.1174 Ops/s $\textbf{\color{#d91a1a}-20.68\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.9597ms 1.7272ms 578.9680 Ops/s 578.5712 Ops/s $\color{#35bf28}+0.07\%$
test_dqn_speed[False-None] 6.5514ms 1.4423ms 693.3608 Ops/s 693.8178 Ops/s $\color{#d91a1a}-0.07\%$
test_dqn_speed[False-backward] 2.3682ms 1.9498ms 512.8826 Ops/s 516.6677 Ops/s $\color{#d91a1a}-0.73\%$
test_dqn_speed[True-None] 0.6719ms 0.5244ms 1.9069 KOps/s 1.8409 KOps/s $\color{#35bf28}+3.59\%$
test_dqn_speed[True-backward] 1.0302ms 0.9879ms 1.0123 KOps/s 924.7285 Ops/s $\textbf{\color{#35bf28}+9.47\%}$
test_dqn_speed[reduce-overhead-None] 0.8216ms 0.5167ms 1.9353 KOps/s 1.9074 KOps/s $\color{#35bf28}+1.46\%$
test_dqn_speed[reduce-overhead-backward] 1.0279ms 0.9702ms 1.0307 KOps/s 1.0162 KOps/s $\color{#35bf28}+1.43\%$
test_ddpg_speed[False-None] 3.3290ms 2.8999ms 344.8354 Ops/s 345.6875 Ops/s $\color{#d91a1a}-0.25\%$
test_ddpg_speed[False-backward] 4.2505ms 4.1504ms 240.9381 Ops/s 243.8776 Ops/s $\color{#d91a1a}-1.21\%$
test_ddpg_speed[True-None] 1.7724ms 1.4102ms 709.0981 Ops/s 648.7086 Ops/s $\textbf{\color{#35bf28}+9.31\%}$
test_ddpg_speed[True-backward] 2.7985ms 2.4285ms 411.7842 Ops/s 411.9572 Ops/s $\color{#d91a1a}-0.04\%$
test_ddpg_speed[reduce-overhead-None] 1.7968ms 1.4033ms 712.6222 Ops/s 689.5773 Ops/s $\color{#35bf28}+3.34\%$
test_ddpg_speed[reduce-overhead-backward] 2.4562ms 2.4126ms 414.4917 Ops/s 355.6828 Ops/s $\textbf{\color{#35bf28}+16.53\%}$
test_sac_speed[False-None] 8.3715ms 7.9582ms 125.6572 Ops/s 126.6375 Ops/s $\color{#d91a1a}-0.77\%$
test_sac_speed[False-backward] 11.5967ms 11.2613ms 88.7999 Ops/s 89.2607 Ops/s $\color{#d91a1a}-0.52\%$
test_sac_speed[True-None] 2.5386ms 2.1233ms 470.9697 Ops/s 463.8317 Ops/s $\color{#35bf28}+1.54\%$
test_sac_speed[True-backward] 4.2604ms 4.0739ms 245.4654 Ops/s 233.7171 Ops/s $\textbf{\color{#35bf28}+5.03\%}$
test_sac_speed[reduce-overhead-None] 2.3327ms 2.1146ms 472.9070 Ops/s 462.3207 Ops/s $\color{#35bf28}+2.29\%$
test_sac_speed[reduce-overhead-backward] 4.2048ms 4.0669ms 245.8874 Ops/s 230.8396 Ops/s $\textbf{\color{#35bf28}+6.52\%}$
test_redq_speed[False-None] 13.7552ms 10.5655ms 94.6472 Ops/s 95.9440 Ops/s $\color{#d91a1a}-1.35\%$
test_redq_speed[False-backward] 22.5980ms 18.0554ms 55.3852 Ops/s 55.9013 Ops/s $\color{#d91a1a}-0.92\%$
test_redq_speed[True-None] 4.5478ms 4.3129ms 231.8614 Ops/s 226.6286 Ops/s $\color{#35bf28}+2.31\%$
test_redq_speed[True-backward] 10.1620ms 9.7544ms 102.5173 Ops/s 102.1908 Ops/s $\color{#35bf28}+0.32\%$
test_redq_speed[reduce-overhead-None] 4.7083ms 4.2529ms 235.1364 Ops/s 217.0958 Ops/s $\textbf{\color{#35bf28}+8.31\%}$
test_redq_speed[reduce-overhead-backward] 10.3580ms 9.9244ms 100.7616 Ops/s 99.7008 Ops/s $\color{#35bf28}+1.06\%$
test_redq_deprec_speed[False-None] 11.2815ms 10.9264ms 91.5218 Ops/s 88.8002 Ops/s $\color{#35bf28}+3.06\%$
test_redq_deprec_speed[False-backward] 16.0578ms 15.6797ms 63.7766 Ops/s 62.1426 Ops/s $\color{#35bf28}+2.63\%$
test_redq_deprec_speed[True-None] 4.0438ms 3.6105ms 276.9725 Ops/s 275.6839 Ops/s $\color{#35bf28}+0.47\%$
test_redq_deprec_speed[True-backward] 8.0062ms 7.6136ms 131.3443 Ops/s 128.6043 Ops/s $\color{#35bf28}+2.13\%$
test_redq_deprec_speed[reduce-overhead-None] 4.0731ms 3.5813ms 279.2285 Ops/s 282.9439 Ops/s $\color{#d91a1a}-1.31\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.9359ms 7.6150ms 131.3198 Ops/s 130.5432 Ops/s $\color{#35bf28}+0.59\%$
test_td3_speed[False-None] 8.2050ms 7.9934ms 125.1033 Ops/s 125.3263 Ops/s $\color{#d91a1a}-0.18\%$
test_td3_speed[False-backward] 11.5862ms 10.9466ms 91.3525 Ops/s 92.1381 Ops/s $\color{#d91a1a}-0.85\%$
test_td3_speed[True-None] 1.8511ms 1.8094ms 552.6555 Ops/s 549.5981 Ops/s $\color{#35bf28}+0.56\%$
test_td3_speed[True-backward] 3.8069ms 3.6481ms 274.1128 Ops/s 271.8615 Ops/s $\color{#35bf28}+0.83\%$
test_td3_speed[reduce-overhead-None] 1.8772ms 1.7972ms 556.4188 Ops/s 563.6270 Ops/s $\color{#d91a1a}-1.28\%$
test_td3_speed[reduce-overhead-backward] 3.7887ms 3.6583ms 273.3474 Ops/s 268.5132 Ops/s $\color{#35bf28}+1.80\%$
test_cql_speed[False-None] 28.7112ms 25.8518ms 38.6821 Ops/s 38.5585 Ops/s $\color{#35bf28}+0.32\%$
test_cql_speed[False-backward] 37.7741ms 35.1823ms 28.4234 Ops/s 28.3807 Ops/s $\color{#35bf28}+0.15\%$
test_cql_speed[True-None] 12.9040ms 12.3874ms 80.7273 Ops/s 81.1772 Ops/s $\color{#d91a1a}-0.55\%$
test_cql_speed[True-backward] 18.4526ms 18.1124ms 55.2107 Ops/s 56.8421 Ops/s $\color{#d91a1a}-2.87\%$
test_cql_speed[reduce-overhead-None] 12.8771ms 12.4944ms 80.0358 Ops/s 81.4791 Ops/s $\color{#d91a1a}-1.77\%$
test_cql_speed[reduce-overhead-backward] 18.6456ms 18.2628ms 54.7560 Ops/s 57.2652 Ops/s $\color{#d91a1a}-4.38\%$
test_a2c_speed[False-None] 5.7552ms 5.4089ms 184.8802 Ops/s 183.2657 Ops/s $\color{#35bf28}+0.88\%$
test_a2c_speed[False-backward] 12.3882ms 11.6966ms 85.4949 Ops/s 84.4372 Ops/s $\color{#35bf28}+1.25\%$
test_a2c_speed[True-None] 4.0991ms 3.6796ms 271.7664 Ops/s 269.4980 Ops/s $\color{#35bf28}+0.84\%$
test_a2c_speed[True-backward] 9.2975ms 8.6573ms 115.5095 Ops/s 114.2543 Ops/s $\color{#35bf28}+1.10\%$
test_a2c_speed[reduce-overhead-None] 3.9096ms 3.6772ms 271.9476 Ops/s 266.4064 Ops/s $\color{#35bf28}+2.08\%$
test_a2c_speed[reduce-overhead-backward] 9.0138ms 8.7703ms 114.0208 Ops/s 106.7862 Ops/s $\textbf{\color{#35bf28}+6.77\%}$
test_ppo_speed[False-None] 6.0610ms 5.8650ms 170.5019 Ops/s 167.4951 Ops/s $\color{#35bf28}+1.80\%$
test_ppo_speed[False-backward] 12.6823ms 12.4682ms 80.2037 Ops/s 78.7959 Ops/s $\color{#35bf28}+1.79\%$
test_ppo_speed[True-None] 4.0434ms 3.6729ms 272.2627 Ops/s 260.1315 Ops/s $\color{#35bf28}+4.66\%$
test_ppo_speed[True-backward] 9.4270ms 8.4482ms 118.3687 Ops/s 117.8241 Ops/s $\color{#35bf28}+0.46\%$
test_ppo_speed[reduce-overhead-None] 4.0299ms 3.6255ms 275.8220 Ops/s 275.2946 Ops/s $\color{#35bf28}+0.19\%$
test_ppo_speed[reduce-overhead-backward] 9.0473ms 8.6586ms 115.4923 Ops/s 111.8089 Ops/s $\color{#35bf28}+3.29\%$
test_reinforce_speed[False-None] 4.7400ms 4.5528ms 219.6468 Ops/s 218.6422 Ops/s $\color{#35bf28}+0.46\%$
test_reinforce_speed[False-backward] 7.7122ms 7.3665ms 135.7504 Ops/s 136.0329 Ops/s $\color{#d91a1a}-0.21\%$
test_reinforce_speed[True-None] 3.0813ms 2.8860ms 346.4955 Ops/s 341.0571 Ops/s $\color{#35bf28}+1.59\%$
test_reinforce_speed[True-backward] 8.0394ms 7.7575ms 128.9075 Ops/s 119.9060 Ops/s $\textbf{\color{#35bf28}+7.51\%}$
test_reinforce_speed[reduce-overhead-None] 3.4124ms 2.8565ms 350.0736 Ops/s 340.2845 Ops/s $\color{#35bf28}+2.88\%$
test_reinforce_speed[reduce-overhead-backward] 8.2152ms 7.9489ms 125.8039 Ops/s 118.9776 Ops/s $\textbf{\color{#35bf28}+5.74\%}$
test_iql_speed[False-None] 25.4922ms 20.1816ms 49.5500 Ops/s 48.1755 Ops/s $\color{#35bf28}+2.85\%$
test_iql_speed[False-backward] 31.1482ms 30.4248ms 32.8679 Ops/s 32.7701 Ops/s $\color{#35bf28}+0.30\%$
test_iql_speed[True-None] 8.8685ms 8.4944ms 117.7245 Ops/s 115.6849 Ops/s $\color{#35bf28}+1.76\%$
test_iql_speed[True-backward] 17.0848ms 16.7539ms 59.6875 Ops/s 59.4755 Ops/s $\color{#35bf28}+0.36\%$
test_iql_speed[reduce-overhead-None] 9.2403ms 8.5900ms 116.4149 Ops/s 111.8933 Ops/s $\color{#35bf28}+4.04\%$
test_iql_speed[reduce-overhead-backward] 17.6074ms 17.2793ms 57.8729 Ops/s 58.2766 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.4650ms 6.0668ms 164.8325 Ops/s 163.6072 Ops/s $\color{#35bf28}+0.75\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6182ms 0.3402ms 2.9396 KOps/s 2.9339 KOps/s $\color{#35bf28}+0.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6949ms 0.3009ms 3.3229 KOps/s 3.6589 KOps/s $\textbf{\color{#d91a1a}-9.18\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1534ms 5.8244ms 171.6913 Ops/s 171.6428 Ops/s $\color{#35bf28}+0.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9336ms 0.3142ms 3.1827 KOps/s 3.1032 KOps/s $\color{#35bf28}+2.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4953ms 0.2950ms 3.3903 KOps/s 3.0424 KOps/s $\textbf{\color{#35bf28}+11.44\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6099ms 1.3279ms 753.0883 Ops/s 725.4257 Ops/s $\color{#35bf28}+3.81\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4413ms 1.1962ms 835.9931 Ops/s 776.1521 Ops/s $\textbf{\color{#35bf28}+7.71\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.4060ms 6.1040ms 163.8277 Ops/s 167.3673 Ops/s $\color{#d91a1a}-2.11\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0494ms 0.4513ms 2.2160 KOps/s 2.2321 KOps/s $\color{#d91a1a}-0.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8316ms 0.4101ms 2.4383 KOps/s 2.3097 KOps/s $\textbf{\color{#35bf28}+5.57\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.0234ms 5.8595ms 170.6616 Ops/s 171.5482 Ops/s $\color{#d91a1a}-0.52\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5802ms 0.2803ms 3.5671 KOps/s 3.5572 KOps/s $\color{#35bf28}+0.28\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4810ms 0.2588ms 3.8643 KOps/s 3.2659 KOps/s $\textbf{\color{#35bf28}+18.32\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9182ms 5.7758ms 173.1353 Ops/s 173.5111 Ops/s $\color{#d91a1a}-0.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8047ms 0.2786ms 3.5900 KOps/s 3.2444 KOps/s $\textbf{\color{#35bf28}+10.65\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4590ms 0.2582ms 3.8728 KOps/s 3.1343 KOps/s $\textbf{\color{#35bf28}+23.56\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0547ms 5.9081ms 169.2602 Ops/s 166.3356 Ops/s $\color{#35bf28}+1.76\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9308ms 0.4305ms 2.3227 KOps/s 2.2946 KOps/s $\color{#35bf28}+1.23\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6638ms 0.4194ms 2.3845 KOps/s 2.4047 KOps/s $\color{#d91a1a}-0.84\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.6357ms 5.0887ms 196.5157 Ops/s 193.7235 Ops/s $\color{#35bf28}+1.44\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.8319ms 2.3118ms 432.5682 Ops/s 481.6307 Ops/s $\textbf{\color{#d91a1a}-10.19\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 3.3832ms 1.1234ms 890.1550 Ops/s 882.5453 Ops/s $\color{#35bf28}+0.86\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5267s 15.5862ms 64.1593 Ops/s 189.6173 Ops/s $\textbf{\color{#d91a1a}-66.16\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 2.2668ms 1.3776ms 725.8968 Ops/s 722.6047 Ops/s $\color{#35bf28}+0.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.0195ms 1.0000ms 999.9795 Ops/s 1.0272 KOps/s $\color{#d91a1a}-2.65\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 7.3447ms 5.2770ms 189.5002 Ops/s 55.3893 Ops/s $\textbf{\color{#35bf28}+242.12\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.0801ms 2.1952ms 455.5383 Ops/s 467.4085 Ops/s $\color{#d91a1a}-2.54\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.6095ms 1.3806ms 724.3420 Ops/s 979.7791 Ops/s $\textbf{\color{#d91a1a}-26.07\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 36.3636ms 33.6570ms 29.7115 Ops/s 29.8152 Ops/s $\color{#d91a1a}-0.35\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.3630ms 17.9051ms 55.8499 Ops/s 55.2656 Ops/s $\color{#35bf28}+1.06\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 35.8899ms 34.1783ms 29.2584 Ops/s 28.7481 Ops/s $\color{#35bf28}+1.77\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.4610ms 17.8498ms 56.0231 Ops/s 55.4328 Ops/s $\color{#35bf28}+1.06\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 38.7256ms 35.8515ms 27.8929 Ops/s 27.7827 Ops/s $\color{#35bf28}+0.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.8542ms 19.3758ms 51.6107 Ops/s 52.0868 Ops/s $\color{#d91a1a}-0.91\%$

vmoens added 4 commits October 19, 2025 20:56
@vmoens
Update
39f3850 
[ghstack-poisoned]
@vmoens
Update
70d35e9 
[ghstack-poisoned]
@vmoens
Update
72d9116 
[ghstack-poisoned]
@vmoens
Update
dfaa0fa 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens