Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Example] RNN-based policy example #2675

Merged
merged 1 commit into from
Dec 20, 2024
Merged

[Example] RNN-based policy example #2675

merged 1 commit into from
Dec 20, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Dec 20, 2024 •
edited
Loading

Stack from ghstack (oldest at bottom):

@vmoens
Update
943a1f1 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Dec 20, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2675

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 18 Unrelated Failures

As of commit 943a1f1 with merge base ab4250e (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Dec 20, 2024
@vmoens
[Example] RNN-based policy example
1a79aac 
ghstack-source-id: ef0087e9b5cba40be428f57ef70ecd2f63483d03
Pull Request resolved: #2675
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 20, 2024
@vmoens vmoens merged commit 943a1f1 into gh/vmoens/64/base Dec 20, 2024
17 of 29 checks passed
vmoens added a commit that referenced this pull request Dec 20, 2024
@vmoens
[Example] RNN-based policy example
d009835 
ghstack-source-id: ef0087e9b5cba40be428f57ef70ecd2f63483d03
Pull Request resolved: #2675
@vmoens vmoens deleted the gh/vmoens/64/head branch December 20, 2024 12:12
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}48$. Worsened: $\large\color{#d91a1a}11$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.4373s 0.4350s 2.2986 Ops/s 2.2067 Ops/s $\color{#35bf28}+4.16\%$
test_transformed 0.6233s 0.6168s 1.6212 Ops/s 1.6279 Ops/s $\color{#d91a1a}-0.41\%$
test_serial 1.3712s 1.3672s 0.7314 Ops/s 0.7184 Ops/s $\color{#35bf28}+1.81\%$
test_parallel 1.2940s 1.2086s 0.8274 Ops/s 0.8095 Ops/s $\color{#35bf28}+2.22\%$
test_step_mdp_speed[True-True-True-True-True] 0.2368ms 30.9784μs 32.2806 KOps/s 30.4573 KOps/s $\textbf{\color{#35bf28}+5.99\%}$
test_step_mdp_speed[True-True-True-True-False] 60.8450μs 18.2227μs 54.8767 KOps/s 52.2133 KOps/s $\textbf{\color{#35bf28}+5.10\%}$
test_step_mdp_speed[True-True-True-False-True] 48.5510μs 17.5243μs 57.0637 KOps/s 53.5075 KOps/s $\textbf{\color{#35bf28}+6.65\%}$
test_step_mdp_speed[True-True-True-False-False] 37.4800μs 10.3233μs 96.8686 KOps/s 91.3770 KOps/s $\textbf{\color{#35bf28}+6.01\%}$
test_step_mdp_speed[True-True-False-True-True] 73.3270μs 33.2946μs 30.0349 KOps/s 28.1818 KOps/s $\textbf{\color{#35bf28}+6.58\%}$
test_step_mdp_speed[True-True-False-True-False] 88.7170μs 20.1032μs 49.7434 KOps/s 47.0200 KOps/s $\textbf{\color{#35bf28}+5.79\%}$
test_step_mdp_speed[True-True-False-False-True] 64.7320μs 19.1466μs 52.2285 KOps/s 48.6061 KOps/s $\textbf{\color{#35bf28}+7.45\%}$
test_step_mdp_speed[True-True-False-False-False] 67.8770μs 12.2098μs 81.9017 KOps/s 77.4645 KOps/s $\textbf{\color{#35bf28}+5.73\%}$
test_step_mdp_speed[True-False-True-True-True] 67.6270μs 34.4038μs 29.0666 KOps/s 26.5221 KOps/s $\textbf{\color{#35bf28}+9.59\%}$
test_step_mdp_speed[True-False-True-True-False] 52.8500μs 22.3581μs 44.7264 KOps/s 41.9424 KOps/s $\textbf{\color{#35bf28}+6.64\%}$
test_step_mdp_speed[True-False-True-False-True] 63.6090μs 19.1624μs 52.1855 KOps/s 47.9356 KOps/s $\textbf{\color{#35bf28}+8.87\%}$
test_step_mdp_speed[True-False-True-False-False] 46.1860μs 12.1749μs 82.1361 KOps/s 76.4260 KOps/s $\textbf{\color{#35bf28}+7.47\%}$
test_step_mdp_speed[True-False-False-True-True] 72.3660μs 36.7133μs 27.2381 KOps/s 25.2315 KOps/s $\textbf{\color{#35bf28}+7.95\%}$
test_step_mdp_speed[True-False-False-True-False] 61.5250μs 24.3851μs 41.0086 KOps/s 38.5356 KOps/s $\textbf{\color{#35bf28}+6.42\%}$
test_step_mdp_speed[True-False-False-False-True] 54.2420μs 20.8887μs 47.8728 KOps/s 43.4357 KOps/s $\textbf{\color{#35bf28}+10.22\%}$
test_step_mdp_speed[True-False-False-False-False] 41.5080μs 14.1014μs 70.9147 KOps/s 66.4197 KOps/s $\textbf{\color{#35bf28}+6.77\%}$
test_step_mdp_speed[False-True-True-True-True] 68.4490μs 35.2200μs 28.3930 KOps/s 26.5321 KOps/s $\textbf{\color{#35bf28}+7.01\%}$
test_step_mdp_speed[False-True-True-True-False] 90.5400μs 22.4999μs 44.4446 KOps/s 41.8636 KOps/s $\textbf{\color{#35bf28}+6.17\%}$
test_step_mdp_speed[False-True-True-False-True] 58.6300μs 22.2718μs 44.8999 KOps/s 41.5910 KOps/s $\textbf{\color{#35bf28}+7.96\%}$
test_step_mdp_speed[False-True-True-False-False] 37.1900μs 13.5829μs 73.6219 KOps/s 68.7715 KOps/s $\textbf{\color{#35bf28}+7.05\%}$
test_step_mdp_speed[False-True-False-True-True] 74.7500μs 36.7084μs 27.2418 KOps/s 25.0479 KOps/s $\textbf{\color{#35bf28}+8.76\%}$
test_step_mdp_speed[False-True-False-True-False] 70.4330μs 24.2497μs 41.2377 KOps/s 38.4625 KOps/s $\textbf{\color{#35bf28}+7.22\%}$
test_step_mdp_speed[False-True-False-False-True] 2.6586ms 24.3536μs 41.0617 KOps/s 38.5860 KOps/s $\textbf{\color{#35bf28}+6.42\%}$
test_step_mdp_speed[False-True-False-False-False] 43.8820μs 15.7138μs 63.6383 KOps/s 59.6903 KOps/s $\textbf{\color{#35bf28}+6.61\%}$
test_step_mdp_speed[False-False-True-True-True] 85.4300μs 38.7867μs 25.7820 KOps/s 23.7405 KOps/s $\textbf{\color{#35bf28}+8.60\%}$
test_step_mdp_speed[False-False-True-True-False] 66.9560μs 26.0893μs 38.3299 KOps/s 35.3627 KOps/s $\textbf{\color{#35bf28}+8.39\%}$
test_step_mdp_speed[False-False-True-False-True] 52.5580μs 24.0137μs 41.6430 KOps/s 38.5103 KOps/s $\textbf{\color{#35bf28}+8.13\%}$
test_step_mdp_speed[False-False-True-False-False] 54.8730μs 15.5572μs 64.2788 KOps/s 60.2663 KOps/s $\textbf{\color{#35bf28}+6.66\%}$
test_step_mdp_speed[False-False-False-True-True] 0.1025ms 40.1572μs 24.9021 KOps/s 23.1115 KOps/s $\textbf{\color{#35bf28}+7.75\%}$
test_step_mdp_speed[False-False-False-True-False] 64.9920μs 27.7118μs 36.0858 KOps/s 33.8273 KOps/s $\textbf{\color{#35bf28}+6.68\%}$
test_step_mdp_speed[False-False-False-False-True] 79.5990μs 25.8536μs 38.6794 KOps/s 36.1210 KOps/s $\textbf{\color{#35bf28}+7.08\%}$
test_step_mdp_speed[False-False-False-False-False] 0.2767ms 17.2144μs 58.0909 KOps/s 53.5688 KOps/s $\textbf{\color{#35bf28}+8.44\%}$
test_values[generalized_advantage_estimate-True-True] 9.9141ms 9.6711ms 103.4013 Ops/s 101.2812 Ops/s $\color{#35bf28}+2.09\%$
test_values[vec_generalized_advantage_estimate-True-True] 39.2915ms 35.7448ms 27.9761 Ops/s 29.8689 Ops/s $\textbf{\color{#d91a1a}-6.34\%}$
test_values[td0_return_estimate-False-False] 0.2445ms 0.1762ms 5.6749 KOps/s 5.6789 KOps/s $\color{#d91a1a}-0.07\%$
test_values[td1_return_estimate-False-False] 24.6835ms 24.0532ms 41.5745 Ops/s 40.8248 Ops/s $\color{#35bf28}+1.84\%$
test_values[vec_td1_return_estimate-False-False] 39.2620ms 35.8902ms 27.8627 Ops/s 29.7942 Ops/s $\textbf{\color{#d91a1a}-6.48\%}$
test_values[td_lambda_return_estimate-True-False] 39.0392ms 35.0037ms 28.5684 Ops/s 28.4205 Ops/s $\color{#35bf28}+0.52\%$
test_values[vec_td_lambda_return_estimate-True-False] 37.5116ms 35.8800ms 27.8707 Ops/s 29.7765 Ops/s $\textbf{\color{#d91a1a}-6.40\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.6826ms 8.5262ms 117.2850 Ops/s 116.1673 Ops/s $\color{#35bf28}+0.96\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 4.5122ms 1.9760ms 506.0660 Ops/s 551.0333 Ops/s $\textbf{\color{#d91a1a}-8.16\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4667ms 0.3569ms 2.8018 KOps/s 2.7510 KOps/s $\color{#35bf28}+1.85\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.5572ms 44.4577ms 22.4933 Ops/s 25.6517 Ops/s $\textbf{\color{#d91a1a}-12.31\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.1816ms 3.0470ms 328.1870 Ops/s 329.0783 Ops/s $\color{#d91a1a}-0.27\%$
test_dqn_speed[False-None] 6.3367ms 1.4502ms 689.5564 Ops/s 702.0668 Ops/s $\color{#d91a1a}-1.78\%$
test_dqn_speed[False-backward] 2.0238ms 1.9218ms 520.3488 Ops/s 532.5707 Ops/s $\color{#d91a1a}-2.29\%$
test_dqn_speed[True-None] 0.7853ms 0.4904ms 2.0390 KOps/s 2.0631 KOps/s $\color{#d91a1a}-1.16\%$
test_dqn_speed[True-backward] 1.2663ms 0.9833ms 1.0170 KOps/s 993.4840 Ops/s $\color{#35bf28}+2.37\%$
test_dqn_speed[reduce-overhead-None] 1.4125ms 0.4929ms 2.0289 KOps/s 2.0799 KOps/s $\color{#d91a1a}-2.45\%$
test_dqn_speed[reduce-overhead-backward] 0.9943ms 0.9377ms 1.0665 KOps/s 1.0812 KOps/s $\color{#d91a1a}-1.36\%$
test_ddpg_speed[False-None] 3.8615ms 3.0238ms 330.7059 Ops/s 340.6858 Ops/s $\color{#d91a1a}-2.93\%$
test_ddpg_speed[False-backward] 4.3185ms 4.1397ms 241.5623 Ops/s 224.9284 Ops/s $\textbf{\color{#35bf28}+7.40\%}$
test_ddpg_speed[True-None] 1.2524ms 1.0169ms 983.3874 Ops/s 971.3241 Ops/s $\color{#35bf28}+1.24\%$
test_ddpg_speed[True-backward] 2.0024ms 1.9130ms 522.7269 Ops/s 515.7080 Ops/s $\color{#35bf28}+1.36\%$
test_ddpg_speed[reduce-overhead-None] 1.4041ms 1.0192ms 981.2056 Ops/s 971.1456 Ops/s $\color{#35bf28}+1.04\%$
test_ddpg_speed[reduce-overhead-backward] 2.2082ms 1.9322ms 517.5529 Ops/s 516.2743 Ops/s $\color{#35bf28}+0.25\%$
test_sac_speed[False-None] 10.0000ms 8.2002ms 121.9488 Ops/s 118.6048 Ops/s $\color{#35bf28}+2.82\%$
test_sac_speed[False-backward] 11.1765ms 10.9278ms 91.5093 Ops/s 91.8631 Ops/s $\color{#d91a1a}-0.39\%$
test_sac_speed[True-None] 2.2863ms 1.8632ms 536.7198 Ops/s 541.2338 Ops/s $\color{#d91a1a}-0.83\%$
test_sac_speed[True-backward] 3.6706ms 3.5534ms 281.4200 Ops/s 285.3207 Ops/s $\color{#d91a1a}-1.37\%$
test_sac_speed[reduce-overhead-None] 2.0708ms 1.8610ms 537.3459 Ops/s 531.7854 Ops/s $\color{#35bf28}+1.05\%$
test_sac_speed[reduce-overhead-backward] 4.0780ms 3.7708ms 265.1949 Ops/s 284.0531 Ops/s $\textbf{\color{#d91a1a}-6.64\%}$
test_redq_speed[False-None] 15.4379ms 13.5798ms 73.6387 Ops/s 76.9813 Ops/s $\color{#d91a1a}-4.34\%$
test_redq_speed[False-backward] 25.9174ms 23.2562ms 42.9993 Ops/s 44.7187 Ops/s $\color{#d91a1a}-3.84\%$
test_redq_speed[True-None] 6.3412ms 5.2339ms 191.0635 Ops/s 218.9459 Ops/s $\textbf{\color{#d91a1a}-12.73\%}$
test_redq_speed[True-backward] 13.3854ms 13.0806ms 76.4490 Ops/s 76.1898 Ops/s $\color{#35bf28}+0.34\%$
test_redq_speed[reduce-overhead-None] 9.7897ms 4.7088ms 212.3668 Ops/s 185.8404 Ops/s $\textbf{\color{#35bf28}+14.27\%}$
test_redq_speed[reduce-overhead-backward] 14.1335ms 12.4241ms 80.4891 Ops/s 83.9892 Ops/s $\color{#d91a1a}-4.17\%$
test_redq_deprec_speed[False-None] 15.3085ms 13.9522ms 71.6735 Ops/s 76.7849 Ops/s $\textbf{\color{#d91a1a}-6.66\%}$
test_redq_deprec_speed[False-backward] 20.7380ms 18.5581ms 53.8848 Ops/s 53.8418 Ops/s $\color{#35bf28}+0.08\%$
test_redq_deprec_speed[True-None] 4.3935ms 3.8159ms 262.0631 Ops/s 262.1121 Ops/s $\color{#d91a1a}-0.02\%$
test_redq_deprec_speed[True-backward] 8.5590ms 8.0669ms 123.9626 Ops/s 113.5323 Ops/s $\textbf{\color{#35bf28}+9.19\%}$
test_redq_deprec_speed[reduce-overhead-None] 4.2413ms 3.5725ms 279.9141 Ops/s 247.7115 Ops/s $\textbf{\color{#35bf28}+13.00\%}$
test_redq_deprec_speed[reduce-overhead-backward] 8.5535ms 7.9793ms 125.3241 Ops/s 120.9356 Ops/s $\color{#35bf28}+3.63\%$
test_td3_speed[False-None] 10.1606ms 8.1753ms 122.3196 Ops/s 119.1276 Ops/s $\color{#35bf28}+2.68\%$
test_td3_speed[False-backward] 14.8548ms 11.2709ms 88.7243 Ops/s 91.0051 Ops/s $\color{#d91a1a}-2.51\%$
test_td3_speed[True-None] 1.9313ms 1.7323ms 577.2685 Ops/s 575.1523 Ops/s $\color{#35bf28}+0.37\%$
test_td3_speed[True-backward] 3.6349ms 3.4401ms 290.6892 Ops/s 300.8588 Ops/s $\color{#d91a1a}-3.38\%$
test_td3_speed[reduce-overhead-None] 1.9560ms 1.7617ms 567.6346 Ops/s 568.5318 Ops/s $\color{#d91a1a}-0.16\%$
test_td3_speed[reduce-overhead-backward] 4.0053ms 3.4485ms 289.9807 Ops/s 280.7026 Ops/s $\color{#35bf28}+3.31\%$
test_cql_speed[False-None] 39.8986ms 37.5863ms 26.6054 Ops/s 26.3098 Ops/s $\color{#35bf28}+1.12\%$
test_cql_speed[False-backward] 58.0525ms 47.5607ms 21.0258 Ops/s 20.4836 Ops/s $\color{#35bf28}+2.65\%$
test_cql_speed[True-None] 16.4194ms 15.9848ms 62.5596 Ops/s 63.0623 Ops/s $\color{#d91a1a}-0.80\%$
test_cql_speed[True-backward] 24.5122ms 22.7957ms 43.8680 Ops/s 42.9098 Ops/s $\color{#35bf28}+2.23\%$
test_cql_speed[reduce-overhead-None] 16.9603ms 15.5527ms 64.2973 Ops/s 63.3387 Ops/s $\color{#35bf28}+1.51\%$
test_cql_speed[reduce-overhead-backward] 24.4626ms 23.0053ms 43.4683 Ops/s 43.6760 Ops/s $\color{#d91a1a}-0.48\%$
test_a2c_speed[False-None] 7.9735ms 7.4083ms 134.9828 Ops/s 136.5355 Ops/s $\color{#d91a1a}-1.14\%$
test_a2c_speed[False-backward] 17.1946ms 14.9712ms 66.7948 Ops/s 68.8558 Ops/s $\color{#d91a1a}-2.99\%$
test_a2c_speed[True-None] 4.6886ms 4.2178ms 237.0929 Ops/s 226.9465 Ops/s $\color{#35bf28}+4.47\%$
test_a2c_speed[True-backward] 12.5098ms 10.9637ms 91.2103 Ops/s 89.3733 Ops/s $\color{#35bf28}+2.06\%$
test_a2c_speed[reduce-overhead-None] 5.0566ms 4.3084ms 232.1044 Ops/s 217.5765 Ops/s $\textbf{\color{#35bf28}+6.68\%}$
test_a2c_speed[reduce-overhead-backward] 11.2237ms 10.6607ms 93.8021 Ops/s 90.8691 Ops/s $\color{#35bf28}+3.23\%$
test_ppo_speed[False-None] 9.3483ms 7.6930ms 129.9891 Ops/s 131.2792 Ops/s $\color{#d91a1a}-0.98\%$
test_ppo_speed[False-backward] 15.4079ms 14.5847ms 68.5651 Ops/s 65.0541 Ops/s $\textbf{\color{#35bf28}+5.40\%}$
test_ppo_speed[True-None] 4.5216ms 3.7779ms 264.6956 Ops/s 265.3335 Ops/s $\color{#d91a1a}-0.24\%$
test_ppo_speed[True-backward] 9.8828ms 9.5261ms 104.9747 Ops/s 85.2485 Ops/s $\textbf{\color{#35bf28}+23.14\%}$
test_ppo_speed[reduce-overhead-None] 5.4807ms 3.7704ms 265.2250 Ops/s 257.7327 Ops/s $\color{#35bf28}+2.91\%$
test_ppo_speed[reduce-overhead-backward] 10.6259ms 9.9820ms 100.1804 Ops/s 96.9047 Ops/s $\color{#35bf28}+3.38\%$
test_reinforce_speed[False-None] 8.1389ms 6.8953ms 145.0269 Ops/s 149.6916 Ops/s $\color{#d91a1a}-3.12\%$
test_reinforce_speed[False-backward] 10.2808ms 9.9980ms 100.0197 Ops/s 96.3446 Ops/s $\color{#35bf28}+3.81\%$
test_reinforce_speed[True-None] 3.0565ms 2.7177ms 367.9532 Ops/s 369.8985 Ops/s $\color{#d91a1a}-0.53\%$
test_reinforce_speed[True-backward] 9.9436ms 8.8376ms 113.1531 Ops/s 113.0642 Ops/s $\color{#35bf28}+0.08\%$
test_reinforce_speed[reduce-overhead-None] 3.0389ms 2.6974ms 370.7222 Ops/s 368.3016 Ops/s $\color{#35bf28}+0.66\%$
test_reinforce_speed[reduce-overhead-backward] 9.3696ms 8.5577ms 116.8544 Ops/s 111.4331 Ops/s $\color{#35bf28}+4.87\%$
test_iql_speed[False-None] 35.5122ms 33.0487ms 30.2584 Ops/s 29.9552 Ops/s $\color{#35bf28}+1.01\%$
test_iql_speed[False-backward] 49.6080ms 46.2674ms 21.6135 Ops/s 21.8557 Ops/s $\color{#d91a1a}-1.11\%$
test_iql_speed[True-None] 11.4270ms 10.7359ms 93.1456 Ops/s 92.8738 Ops/s $\color{#35bf28}+0.29\%$
test_iql_speed[True-backward] 22.8313ms 22.2195ms 45.0055 Ops/s 45.0045 Ops/s $+0.00\%$
test_iql_speed[reduce-overhead-None] 11.7607ms 10.6706ms 93.7156 Ops/s 88.2380 Ops/s $\textbf{\color{#35bf28}+6.21\%}$
test_iql_speed[reduce-overhead-backward] 23.2991ms 21.6416ms 46.2073 Ops/s 43.6213 Ops/s $\textbf{\color{#35bf28}+5.93\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.5152ms 5.0626ms 197.5264 Ops/s 200.4409 Ops/s $\color{#d91a1a}-1.45\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8601ms 0.5183ms 1.9292 KOps/s 1.8465 KOps/s $\color{#35bf28}+4.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9320ms 0.4965ms 2.0140 KOps/s 1.9182 KOps/s $\color{#35bf28}+5.00\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.9457ms 4.6290ms 216.0279 Ops/s 207.7189 Ops/s $\color{#35bf28}+4.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.3468s 0.7712ms 1.2967 KOps/s 1.9361 KOps/s $\textbf{\color{#d91a1a}-33.02\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9166ms 0.4806ms 2.0809 KOps/s 1.9756 KOps/s $\textbf{\color{#35bf28}+5.33\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.9159ms 1.6553ms 604.1365 Ops/s 590.8255 Ops/s $\color{#35bf28}+2.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1386ms 1.5540ms 643.4983 Ops/s 627.0667 Ops/s $\color{#35bf28}+2.62\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 10.7006ms 4.9575ms 201.7144 Ops/s 199.4555 Ops/s $\color{#35bf28}+1.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.4081ms 0.6425ms 1.5564 KOps/s 1.5139 KOps/s $\color{#35bf28}+2.81\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9429ms 0.6217ms 1.6086 KOps/s 1.5938 KOps/s $\color{#35bf28}+0.93\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.3722ms 4.7739ms 209.4727 Ops/s 214.7791 Ops/s $\color{#d91a1a}-2.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.7708ms 0.5183ms 1.9294 KOps/s 1.9027 KOps/s $\color{#35bf28}+1.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7952ms 0.4973ms 2.0107 KOps/s 2.0211 KOps/s $\color{#d91a1a}-0.51\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.7473ms 4.7623ms 209.9816 Ops/s 209.1860 Ops/s $\color{#35bf28}+0.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.5581ms 0.5028ms 1.9888 KOps/s 1.8452 KOps/s $\textbf{\color{#35bf28}+7.78\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7146ms 0.4792ms 2.0867 KOps/s 1.9801 KOps/s $\textbf{\color{#35bf28}+5.38\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2607ms 4.8760ms 205.0853 Ops/s 205.1407 Ops/s $\color{#d91a1a}-0.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3287ms 0.6794ms 1.4719 KOps/s 1.5269 KOps/s $\color{#d91a1a}-3.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8887ms 0.6353ms 1.5740 KOps/s 1.5390 KOps/s $\color{#35bf28}+2.27\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4630s 13.5120ms 74.0085 Ops/s 37.3423 Ops/s $\textbf{\color{#35bf28}+98.19\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 4.9871ms 2.2644ms 441.6251 Ops/s 466.3031 Ops/s $\textbf{\color{#d91a1a}-5.29\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.0645ms 1.3717ms 729.0413 Ops/s 687.4367 Ops/s $\textbf{\color{#35bf28}+6.05\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 5.6991ms 4.2820ms 233.5350 Ops/s 225.0492 Ops/s $\color{#35bf28}+3.77\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 7.2859ms 2.3389ms 427.5558 Ops/s 423.0684 Ops/s $\color{#35bf28}+1.06\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.6781ms 1.3945ms 717.1055 Ops/s 721.9701 Ops/s $\color{#d91a1a}-0.67\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4193s 12.9772ms 77.0583 Ops/s 230.8265 Ops/s $\textbf{\color{#d91a1a}-66.62\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 7.5087ms 2.4425ms 409.4206 Ops/s 404.2339 Ops/s $\color{#35bf28}+1.28\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.1655ms 1.4480ms 690.6122 Ops/s 649.0591 Ops/s $\textbf{\color{#35bf28}+6.40\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 13.8901ms 13.3288ms 75.0255 Ops/s 70.1262 Ops/s $\textbf{\color{#35bf28}+6.99\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 16.2747ms 15.2112ms 65.7410 Ops/s 64.6424 Ops/s $\color{#35bf28}+1.70\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 22.6784ms 21.8663ms 45.7324 Ops/s 43.9510 Ops/s $\color{#35bf28}+4.05\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 16.4332ms 15.2009ms 65.7857 Ops/s 65.6922 Ops/s $\color{#35bf28}+0.14\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 22.7366ms 21.6969ms 46.0895 Ops/s 44.6221 Ops/s $\color{#35bf28}+3.29\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 18.2486ms 16.4209ms 60.8980 Ops/s 59.7679 Ops/s $\color{#35bf28}+1.89\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}18$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.7249s 0.7205s 1.3879 Ops/s 1.3498 Ops/s $\color{#35bf28}+2.82\%$
test_transformed 0.9859s 0.9793s 1.0212 Ops/s 1.0257 Ops/s $\color{#d91a1a}-0.44\%$
test_serial 2.2527s 2.1786s 0.4590 Ops/s 0.4639 Ops/s $\color{#d91a1a}-1.05\%$
test_parallel 1.9254s 1.8615s 0.5372 Ops/s 0.5242 Ops/s $\color{#35bf28}+2.49\%$
test_step_mdp_speed[True-True-True-True-True] 0.2553ms 39.9444μs 25.0348 KOps/s 24.6034 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-True-True-True-False] 0.1400ms 23.7588μs 42.0897 KOps/s 42.3776 KOps/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[True-True-True-False-True] 0.1066ms 22.1950μs 45.0551 KOps/s 44.4467 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-True-True-False-False] 39.7100μs 13.1525μs 76.0312 KOps/s 75.3482 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-True-False-True-True] 79.1210μs 43.0174μs 23.2464 KOps/s 22.9773 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-True-False-True-False] 64.4100μs 25.8082μs 38.7474 KOps/s 37.5937 KOps/s $\color{#35bf28}+3.07\%$
test_step_mdp_speed[True-True-False-False-True] 54.1510μs 25.3102μs 39.5098 KOps/s 41.1834 KOps/s $\color{#d91a1a}-4.06\%$
test_step_mdp_speed[True-True-False-False-False] 0.1823ms 15.3949μs 64.9567 KOps/s 64.8432 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-False-True-True-True] 0.1361ms 45.7100μs 21.8770 KOps/s 22.2564 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[True-False-True-True-False] 67.6410μs 28.1330μs 35.5454 KOps/s 35.5172 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[True-False-True-False-True] 54.7400μs 24.9405μs 40.0955 KOps/s 40.5729 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-False-True-False-False] 41.7000μs 15.6217μs 64.0134 KOps/s 65.0978 KOps/s $\color{#d91a1a}-1.67\%$
test_step_mdp_speed[True-False-False-True-True] 82.2600μs 48.0418μs 20.8152 KOps/s 22.0952 KOps/s $\textbf{\color{#d91a1a}-5.79\%}$
test_step_mdp_speed[True-False-False-True-False] 0.2097ms 30.3446μs 32.9548 KOps/s 32.6662 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[True-False-False-False-True] 0.2024ms 26.8340μs 37.2662 KOps/s 37.2293 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[True-False-False-False-False] 44.9300μs 17.5199μs 57.0780 KOps/s 56.9910 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-True-True-True-True] 0.2689ms 44.9399μs 22.2519 KOps/s 22.1357 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-True-True-True-False] 56.9910μs 28.0039μs 35.7093 KOps/s 34.9768 KOps/s $\color{#35bf28}+2.09\%$
test_step_mdp_speed[False-True-True-False-True] 0.2075ms 28.6626μs 34.8887 KOps/s 34.1482 KOps/s $\color{#35bf28}+2.17\%$
test_step_mdp_speed[False-True-True-False-False] 0.1977ms 16.9418μs 59.0257 KOps/s 58.1392 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[False-True-False-True-True] 0.2480ms 47.5752μs 21.0194 KOps/s 20.9855 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[False-True-False-True-False] 0.2242ms 30.2513μs 33.0564 KOps/s 32.4430 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-True-False-False-True] 3.2098ms 31.3128μs 31.9359 KOps/s 31.7679 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-True-False-False-False] 46.3610μs 19.5189μs 51.2324 KOps/s 50.6957 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-False-True-True-True] 86.6210μs 51.2496μs 19.5123 KOps/s 19.9001 KOps/s $\color{#d91a1a}-1.95\%$
test_step_mdp_speed[False-False-True-True-False] 0.1265ms 33.2825μs 30.0458 KOps/s 30.1545 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-False-True-False-True] 0.1469ms 30.8293μs 32.4367 KOps/s 32.1059 KOps/s $\color{#35bf28}+1.03\%$
test_step_mdp_speed[False-False-True-False-False] 44.7000μs 19.4371μs 51.4479 KOps/s 50.8294 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-False-False-True-True] 79.4400μs 51.1481μs 19.5511 KOps/s 19.1589 KOps/s $\color{#35bf28}+2.05\%$
test_step_mdp_speed[False-False-False-True-False] 62.8110μs 34.8899μs 28.6616 KOps/s 28.2046 KOps/s $\color{#35bf28}+1.62\%$
test_step_mdp_speed[False-False-False-False-True] 58.6100μs 32.4205μs 30.8447 KOps/s 30.6528 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-False-False-False-False] 53.2000μs 21.3745μs 46.7847 KOps/s 46.1931 KOps/s $\color{#35bf28}+1.28\%$
test_values[generalized_advantage_estimate-True-True] 24.9945ms 24.6955ms 40.4932 Ops/s 39.5544 Ops/s $\color{#35bf28}+2.37\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1036s 2.9742ms 336.2232 Ops/s 351.6169 Ops/s $\color{#d91a1a}-4.38\%$
test_values[td0_return_estimate-False-False] 0.1047ms 80.8457μs 12.3692 KOps/s 12.4113 KOps/s $\color{#d91a1a}-0.34\%$
test_values[td1_return_estimate-False-False] 55.9394ms 55.5317ms 18.0077 Ops/s 17.8463 Ops/s $\color{#35bf28}+0.90\%$
test_values[vec_td1_return_estimate-False-False] 1.3261ms 1.0914ms 916.2788 Ops/s 912.8493 Ops/s $\color{#35bf28}+0.38\%$
test_values[td_lambda_return_estimate-True-False] 88.4492ms 88.0106ms 11.3623 Ops/s 11.2410 Ops/s $\color{#35bf28}+1.08\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2440ms 1.0816ms 924.5321 Ops/s 918.8060 Ops/s $\color{#35bf28}+0.62\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.2358ms 24.9940ms 40.0096 Ops/s 39.7915 Ops/s $\color{#35bf28}+0.55\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0332ms 0.7537ms 1.3268 KOps/s 1.3052 KOps/s $\color{#35bf28}+1.66\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8868ms 0.6750ms 1.4815 KOps/s 1.4706 KOps/s $\color{#35bf28}+0.74\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6309ms 1.4817ms 674.8856 Ops/s 673.6682 Ops/s $\color{#35bf28}+0.18\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8612ms 0.6903ms 1.4486 KOps/s 1.4408 KOps/s $\color{#35bf28}+0.54\%$
test_dqn_speed[False-None] 6.8633ms 1.5276ms 654.6393 Ops/s 654.0371 Ops/s $\color{#35bf28}+0.09\%$
test_dqn_speed[False-backward] 2.2774ms 2.1222ms 471.2106 Ops/s 471.6977 Ops/s $\color{#d91a1a}-0.10\%$
test_dqn_speed[True-None] 0.7171ms 0.5654ms 1.7686 KOps/s 1.6923 KOps/s $\color{#35bf28}+4.51\%$
test_dqn_speed[True-backward] 1.3999ms 1.2348ms 809.8486 Ops/s 878.5680 Ops/s $\textbf{\color{#d91a1a}-7.82\%}$
test_dqn_speed[reduce-overhead-None] 0.7387ms 0.5769ms 1.7333 KOps/s 1.7238 KOps/s $\color{#35bf28}+0.55\%$
test_dqn_speed[reduce-overhead-backward] 1.2486ms 1.1033ms 906.3784 Ops/s 901.9516 Ops/s $\color{#35bf28}+0.49\%$
test_ddpg_speed[False-None] 3.1805ms 2.8610ms 349.5336 Ops/s 346.0909 Ops/s $\color{#35bf28}+0.99\%$
test_ddpg_speed[False-backward] 4.6332ms 4.2493ms 235.3320 Ops/s 233.9235 Ops/s $\color{#35bf28}+0.60\%$
test_ddpg_speed[True-None] 1.2755ms 1.1136ms 897.9575 Ops/s 887.5502 Ops/s $\color{#35bf28}+1.17\%$
test_ddpg_speed[True-backward] 2.5262ms 2.3566ms 424.3350 Ops/s 445.7838 Ops/s $\color{#d91a1a}-4.81\%$
test_ddpg_speed[reduce-overhead-None] 1.3132ms 1.1275ms 886.9141 Ops/s 884.6706 Ops/s $\color{#35bf28}+0.25\%$
test_ddpg_speed[reduce-overhead-backward] 1.9445ms 1.8123ms 551.7852 Ops/s 588.8418 Ops/s $\textbf{\color{#d91a1a}-6.29\%}$
test_sac_speed[False-None] 8.4041ms 8.0305ms 124.5245 Ops/s 123.5019 Ops/s $\color{#35bf28}+0.83\%$
test_sac_speed[False-backward] 11.7653ms 11.1575ms 89.6256 Ops/s 91.0375 Ops/s $\color{#d91a1a}-1.55\%$
test_sac_speed[True-None] 1.7627ms 1.5739ms 635.3660 Ops/s 627.6821 Ops/s $\color{#35bf28}+1.22\%$
test_sac_speed[True-backward] 3.6595ms 3.4704ms 288.1551 Ops/s 300.2985 Ops/s $\color{#d91a1a}-4.04\%$
test_sac_speed[reduce-overhead-None] 23.8234ms 13.0189ms 76.8116 Ops/s 78.2193 Ops/s $\color{#d91a1a}-1.80\%$
test_sac_speed[reduce-overhead-backward] 1.4869ms 1.3764ms 726.5180 Ops/s 720.8965 Ops/s $\color{#35bf28}+0.78\%$
test_redq_speed[False-None] 8.3835ms 7.5603ms 132.2700 Ops/s 130.3198 Ops/s $\color{#35bf28}+1.50\%$
test_redq_speed[False-backward] 11.9761ms 11.2623ms 88.7916 Ops/s 87.8551 Ops/s $\color{#35bf28}+1.07\%$
test_redq_speed[True-None] 2.3093ms 2.0367ms 490.9861 Ops/s 486.8054 Ops/s $\color{#35bf28}+0.86\%$
test_redq_speed[True-backward] 3.9065ms 3.7344ms 267.7775 Ops/s 253.6415 Ops/s $\textbf{\color{#35bf28}+5.57\%}$
test_redq_speed[reduce-overhead-None] 2.2145ms 2.0448ms 489.0403 Ops/s 483.5715 Ops/s $\color{#35bf28}+1.13\%$
test_redq_speed[reduce-overhead-backward] 3.9026ms 3.7308ms 268.0417 Ops/s 254.5627 Ops/s $\textbf{\color{#35bf28}+5.29\%}$
test_redq_deprec_speed[False-None] 9.7239ms 9.0422ms 110.5922 Ops/s 107.9949 Ops/s $\color{#35bf28}+2.41\%$
test_redq_deprec_speed[False-backward] 12.5537ms 12.0511ms 82.9801 Ops/s 79.9164 Ops/s $\color{#35bf28}+3.83\%$
test_redq_deprec_speed[True-None] 2.7095ms 2.3874ms 418.8733 Ops/s 400.4496 Ops/s $\color{#35bf28}+4.60\%$
test_redq_deprec_speed[True-backward] 4.2470ms 4.0870ms 244.6764 Ops/s 233.6064 Ops/s $\color{#35bf28}+4.74\%$
test_redq_deprec_speed[reduce-overhead-None] 2.6483ms 2.3769ms 420.7249 Ops/s 419.3878 Ops/s $\color{#35bf28}+0.32\%$
test_redq_deprec_speed[reduce-overhead-backward] 4.6320ms 4.1107ms 243.2673 Ops/s 234.9423 Ops/s $\color{#35bf28}+3.54\%$
test_td3_speed[False-None] 8.2404ms 7.9342ms 126.0370 Ops/s 125.7044 Ops/s $\color{#35bf28}+0.26\%$
test_td3_speed[False-backward] 10.7445ms 10.2564ms 97.4996 Ops/s 95.9651 Ops/s $\color{#35bf28}+1.60\%$
test_td3_speed[True-None] 1.6607ms 1.6344ms 611.8376 Ops/s 612.8823 Ops/s $\color{#d91a1a}-0.17\%$
test_td3_speed[True-backward] 3.3900ms 3.2190ms 310.6574 Ops/s 294.6069 Ops/s $\textbf{\color{#35bf28}+5.45\%}$
test_td3_speed[reduce-overhead-None] 82.7817ms 27.0962ms 36.9056 Ops/s 35.4400 Ops/s $\color{#35bf28}+4.14\%$
test_td3_speed[reduce-overhead-backward] 1.3997ms 1.3316ms 750.9947 Ops/s 666.0501 Ops/s $\textbf{\color{#35bf28}+12.75\%}$
test_cql_speed[False-None] 17.3210ms 16.8399ms 59.3828 Ops/s 58.6357 Ops/s $\color{#35bf28}+1.27\%$
test_cql_speed[False-backward] 22.6473ms 21.9494ms 45.5593 Ops/s 44.5768 Ops/s $\color{#35bf28}+2.20\%$
test_cql_speed[True-None] 3.2294ms 3.0369ms 329.2778 Ops/s 329.5425 Ops/s $\color{#d91a1a}-0.08\%$
test_cql_speed[True-backward] 5.6806ms 5.2225ms 191.4792 Ops/s 189.3547 Ops/s $\color{#35bf28}+1.12\%$
test_cql_speed[reduce-overhead-None] 22.2565ms 13.5572ms 73.7616 Ops/s 74.5721 Ops/s $\color{#d91a1a}-1.09\%$
test_cql_speed[reduce-overhead-backward] 1.6792ms 1.5536ms 643.6495 Ops/s 601.5148 Ops/s $\textbf{\color{#35bf28}+7.00\%}$
test_a2c_speed[False-None] 3.4023ms 3.2215ms 310.4148 Ops/s 305.3483 Ops/s $\color{#35bf28}+1.66\%$
test_a2c_speed[False-backward] 6.5305ms 6.1121ms 163.6105 Ops/s 156.1406 Ops/s $\color{#35bf28}+4.78\%$
test_a2c_speed[True-None] 1.1901ms 1.0407ms 960.8469 Ops/s 958.1301 Ops/s $\color{#35bf28}+0.28\%$
test_a2c_speed[True-backward] 2.8037ms 2.6809ms 373.0082 Ops/s 374.2836 Ops/s $\color{#d91a1a}-0.34\%$
test_a2c_speed[reduce-overhead-None] 22.2708ms 11.9095ms 83.9669 Ops/s 84.8156 Ops/s $\color{#d91a1a}-1.00\%$
test_a2c_speed[reduce-overhead-backward] 1.1335ms 0.9969ms 1.0031 KOps/s 982.7112 Ops/s $\color{#35bf28}+2.07\%$
test_ppo_speed[False-None] 3.8528ms 3.6925ms 270.8193 Ops/s 269.8134 Ops/s $\color{#35bf28}+0.37\%$
test_ppo_speed[False-backward] 7.3423ms 6.8344ms 146.3182 Ops/s 147.7720 Ops/s $\color{#d91a1a}-0.98\%$
test_ppo_speed[True-None] 1.2477ms 0.9982ms 1.0018 KOps/s 1.0290 KOps/s $\color{#d91a1a}-2.64\%$
test_ppo_speed[True-backward] 2.7491ms 2.6226ms 381.3038 Ops/s 360.3526 Ops/s $\textbf{\color{#35bf28}+5.81\%}$
test_ppo_speed[reduce-overhead-None] 0.7432ms 0.5336ms 1.8742 KOps/s 1.8162 KOps/s $\color{#35bf28}+3.20\%$
test_ppo_speed[reduce-overhead-backward] 1.0577ms 0.9798ms 1.0206 KOps/s 843.1512 Ops/s $\textbf{\color{#35bf28}+21.04\%}$
test_reinforce_speed[False-None] 2.5692ms 2.3244ms 430.2147 Ops/s 438.2441 Ops/s $\color{#d91a1a}-1.83\%$
test_reinforce_speed[False-backward] 3.4292ms 3.2806ms 304.8231 Ops/s 296.2496 Ops/s $\color{#35bf28}+2.89\%$
test_reinforce_speed[True-None] 1.0220ms 0.8571ms 1.1667 KOps/s 1.1516 KOps/s $\color{#35bf28}+1.31\%$
test_reinforce_speed[True-backward] 2.6478ms 2.4773ms 403.6677 Ops/s 376.8296 Ops/s $\textbf{\color{#35bf28}+7.12\%}$
test_reinforce_speed[reduce-overhead-None] 22.5460ms 12.0015ms 83.3229 Ops/s 85.6496 Ops/s $\color{#d91a1a}-2.72\%$
test_reinforce_speed[reduce-overhead-backward] 1.1966ms 1.0655ms 938.5488 Ops/s 808.0372 Ops/s $\textbf{\color{#35bf28}+16.15\%}$
test_iql_speed[False-None] 9.8503ms 9.3506ms 106.9448 Ops/s 107.5547 Ops/s $\color{#d91a1a}-0.57\%$
test_iql_speed[False-backward] 13.5900ms 13.0316ms 76.7366 Ops/s 75.6963 Ops/s $\color{#35bf28}+1.37\%$
test_iql_speed[True-None] 2.1099ms 1.8838ms 530.8427 Ops/s 559.6995 Ops/s $\textbf{\color{#d91a1a}-5.16\%}$
test_iql_speed[True-backward] 4.4631ms 4.3307ms 230.9078 Ops/s 228.2887 Ops/s $\color{#35bf28}+1.15\%$
test_iql_speed[reduce-overhead-None] 20.5837ms 11.7778ms 84.9058 Ops/s 86.3358 Ops/s $\color{#d91a1a}-1.66\%$
test_iql_speed[reduce-overhead-backward] 1.5878ms 1.4580ms 685.8756 Ops/s 691.0396 Ops/s $\color{#d91a1a}-0.75\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.9732ms 6.4695ms 154.5707 Ops/s 152.3418 Ops/s $\color{#35bf28}+1.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5079ms 0.3046ms 3.2826 KOps/s 3.4398 KOps/s $\color{#d91a1a}-4.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6251ms 0.2939ms 3.4029 KOps/s 3.3756 KOps/s $\color{#35bf28}+0.81\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.5355ms 6.2030ms 161.2122 Ops/s 158.3142 Ops/s $\color{#35bf28}+1.83\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.3318ms 0.3307ms 3.0238 KOps/s 3.0592 KOps/s $\color{#d91a1a}-1.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5582ms 0.3308ms 3.0229 KOps/s 3.1072 KOps/s $\color{#d91a1a}-2.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5947ms 1.3560ms 737.4733 Ops/s 710.4735 Ops/s $\color{#35bf28}+3.80\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5577ms 1.1688ms 855.5424 Ops/s 868.1166 Ops/s $\color{#d91a1a}-1.45\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.6335ms 6.3586ms 157.2671 Ops/s 155.9964 Ops/s $\color{#35bf28}+0.81\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9269ms 0.4391ms 2.2774 KOps/s 2.1703 KOps/s $\color{#35bf28}+4.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6798ms 0.4281ms 2.3360 KOps/s 2.5044 KOps/s $\textbf{\color{#d91a1a}-6.73\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.5696ms 6.3030ms 158.6552 Ops/s 157.8651 Ops/s $\color{#35bf28}+0.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0450ms 0.3072ms 3.2555 KOps/s 3.4792 KOps/s $\textbf{\color{#d91a1a}-6.43\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5123ms 0.2894ms 3.4556 KOps/s 3.2520 KOps/s $\textbf{\color{#35bf28}+6.26\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.6828ms 6.1802ms 161.8068 Ops/s 159.3597 Ops/s $\color{#35bf28}+1.54\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7080ms 0.2717ms 3.6803 KOps/s 2.9238 KOps/s $\textbf{\color{#35bf28}+25.87\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5676ms 0.2840ms 3.5209 KOps/s 3.0881 KOps/s $\textbf{\color{#35bf28}+14.01\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.6434ms 6.3688ms 157.0145 Ops/s 156.0420 Ops/s $\color{#35bf28}+0.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0737ms 0.4041ms 2.4749 KOps/s 2.1821 KOps/s $\textbf{\color{#35bf28}+13.42\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6126ms 0.3830ms 2.6108 KOps/s 2.2923 KOps/s $\textbf{\color{#35bf28}+13.89\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 7.0578ms 5.4475ms 183.5697 Ops/s 180.4914 Ops/s $\color{#35bf28}+1.71\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 6.1572ms 2.0176ms 495.6417 Ops/s 423.2452 Ops/s $\textbf{\color{#35bf28}+17.11\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.7820ms 1.2745ms 784.6402 Ops/s 786.6496 Ops/s $\color{#d91a1a}-0.26\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 8.7156ms 5.5830ms 179.1144 Ops/s 183.0107 Ops/s $\color{#d91a1a}-2.13\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.4385ms 2.0659ms 484.0529 Ops/s 418.2050 Ops/s $\textbf{\color{#35bf28}+15.75\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 8.1699ms 1.2162ms 822.2441 Ops/s 810.5529 Ops/s $\color{#35bf28}+1.44\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.5178s 15.9888ms 62.5436 Ops/s 32.8995 Ops/s $\textbf{\color{#35bf28}+90.11\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 10.1816ms 2.1913ms 456.3440 Ops/s 447.6101 Ops/s $\color{#35bf28}+1.95\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.2180ms 1.3759ms 726.8211 Ops/s 690.7922 Ops/s $\textbf{\color{#35bf28}+5.22\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 15.7099ms 15.4791ms 64.6031 Ops/s 63.6178 Ops/s $\color{#35bf28}+1.55\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 22.1464ms 18.2913ms 54.6707 Ops/s 56.5921 Ops/s $\color{#d91a1a}-3.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 20.2709ms 19.8828ms 50.2946 Ops/s 48.3218 Ops/s $\color{#35bf28}+4.08\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.0327ms 17.8824ms 55.9208 Ops/s 55.5777 Ops/s $\color{#35bf28}+0.62\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 20.8734ms 20.3228ms 49.2059 Ops/s 48.4139 Ops/s $\color{#35bf28}+1.64\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.5730ms 19.7410ms 50.6561 Ops/s 50.7050 Ops/s $\color{#d91a1a}-0.10\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot