Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix collector length with non-empty batch size #2575

Merged
merged 1 commit into from
Nov 18, 2024
Merged

[BugFix] Fix collector length with non-empty batch size #2575

merged 1 commit into from
Nov 18, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Nov 18, 2024 •
edited
Loading

@vmoens
Update
7caf365 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Nov 18, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2575

Note: Links to docs will display an error until the docs builds have been completed.

❗ 2 Active SEVs

There are 2 currently active SEVs. If your PR is affected, please view them below:

❌ 2 New Failures, 10 Unrelated Failures

As of commit 7caf365 with merge base 83a7a57 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Nov 18, 2024
@vmoens
[BugFix] Fix collector length with non-empty batch size
c848285 
ghstack-source-id: 0c6a7a49f0570fad083340a64dd89c0f4c220c06
Pull Request resolved: #2575
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 18, 2024
This was referenced Nov 18, 2024
Merged
Merged
Merged
@vmoens vmoens merged commit 7caf365 into gh/vmoens/44/base Nov 18, 2024
55 of 64 checks passed
vmoens added a commit that referenced this pull request Nov 18, 2024
@vmoens
[BugFix] Fix collector length with non-empty batch size
b875979 
ghstack-source-id: 0c6a7a49f0570fad083340a64dd89c0f4c220c06
Pull Request resolved: #2575
@vmoens vmoens deleted the gh/vmoens/44/head branch November 18, 2024 15:18
@vmoens vmoens added the bug Something isn't working label Nov 18, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}22$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.4264s 0.4244s 2.3563 Ops/s 2.2501 Ops/s $\color{#35bf28}+4.72\%$
test_transformed 0.6011s 0.5980s 1.6722 Ops/s 1.6123 Ops/s $\color{#35bf28}+3.71\%$
test_serial 1.3323s 1.3302s 0.7518 Ops/s 0.7296 Ops/s $\color{#35bf28}+3.03\%$
test_parallel 1.2965s 1.2833s 0.7792 Ops/s 0.7617 Ops/s $\color{#35bf28}+2.30\%$
test_step_mdp_speed[True-True-True-True-True] 0.2153ms 27.2798μs 36.6572 KOps/s 38.1757 KOps/s $\color{#d91a1a}-3.98\%$
test_step_mdp_speed[True-True-True-True-False] 42.7700μs 16.0765μs 62.2026 KOps/s 65.0034 KOps/s $\color{#d91a1a}-4.31\%$
test_step_mdp_speed[True-True-True-False-True] 46.2660μs 15.5300μs 64.3916 KOps/s 65.3349 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[True-True-True-False-False] 32.3000μs 9.2384μs 108.2433 KOps/s 112.7021 KOps/s $\color{#d91a1a}-3.96\%$
test_step_mdp_speed[True-True-False-True-True] 91.4410μs 29.1410μs 34.3159 KOps/s 35.1285 KOps/s $\color{#d91a1a}-2.31\%$
test_step_mdp_speed[True-True-False-True-False] 40.3550μs 17.5725μs 56.9071 KOps/s 58.4919 KOps/s $\color{#d91a1a}-2.71\%$
test_step_mdp_speed[True-True-False-False-True] 0.1215ms 17.1308μs 58.3744 KOps/s 59.2606 KOps/s $\color{#d91a1a}-1.50\%$
test_step_mdp_speed[True-True-False-False-False] 0.1211ms 11.2700μs 88.7311 KOps/s 97.6818 KOps/s $\textbf{\color{#d91a1a}-9.16\%}$
test_step_mdp_speed[True-False-True-True-True] 0.6241ms 30.4911μs 32.7964 KOps/s 33.4992 KOps/s $\color{#d91a1a}-2.10\%$
test_step_mdp_speed[True-False-True-True-False] 48.6810μs 19.3836μs 51.5900 KOps/s 52.8034 KOps/s $\color{#d91a1a}-2.30\%$
test_step_mdp_speed[True-False-True-False-True] 53.3600μs 17.1681μs 58.2476 KOps/s 59.3312 KOps/s $\color{#d91a1a}-1.83\%$
test_step_mdp_speed[True-False-True-False-False] 41.2570μs 10.7524μs 93.0022 KOps/s 95.6556 KOps/s $\color{#d91a1a}-2.77\%$
test_step_mdp_speed[True-False-False-True-True] 70.7020μs 32.4061μs 30.8584 KOps/s 31.2644 KOps/s $\color{#d91a1a}-1.30\%$
test_step_mdp_speed[True-False-False-True-False] 91.4320μs 19.9954μs 50.0116 KOps/s 48.0775 KOps/s $\color{#35bf28}+4.02\%$
test_step_mdp_speed[True-False-False-False-True] 0.1634ms 18.6068μs 53.7438 KOps/s 54.0586 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[True-False-False-False-False] 0.1098ms 12.3090μs 81.2411 KOps/s 78.8823 KOps/s $\color{#35bf28}+2.99\%$
test_step_mdp_speed[False-True-True-True-True] 0.1801ms 31.4506μs 31.7959 KOps/s 32.9324 KOps/s $\color{#d91a1a}-3.45\%$
test_step_mdp_speed[False-True-True-True-False] 50.5440μs 19.1565μs 52.2017 KOps/s 52.1642 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-True-True-False-True] 77.3040μs 19.7478μs 50.6385 KOps/s 51.7747 KOps/s $\color{#d91a1a}-2.19\%$
test_step_mdp_speed[False-True-True-False-False] 45.5350μs 11.8465μs 84.4130 KOps/s 84.9686 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[False-True-False-True-True] 83.6660μs 31.8473μs 31.3998 KOps/s 31.3484 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[False-True-False-True-False] 68.1380μs 20.8628μs 47.9322 KOps/s 48.1221 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[False-True-False-False-True] 2.8204ms 21.2636μs 47.0287 KOps/s 47.4450 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[False-True-False-False-False] 79.6920μs 13.5493μs 73.8044 KOps/s 74.9706 KOps/s $\color{#d91a1a}-1.56\%$
test_step_mdp_speed[False-False-True-True-True] 78.4570μs 34.0391μs 29.3780 KOps/s 29.9544 KOps/s $\color{#d91a1a}-1.92\%$
test_step_mdp_speed[False-False-True-True-False] 66.0040μs 22.6682μs 44.1146 KOps/s 45.3066 KOps/s $\color{#d91a1a}-2.63\%$
test_step_mdp_speed[False-False-True-False-True] 54.8330μs 21.4347μs 46.6533 KOps/s 47.4210 KOps/s $\color{#d91a1a}-1.62\%$
test_step_mdp_speed[False-False-True-False-False] 0.5229ms 13.7219μs 72.8763 KOps/s 74.7655 KOps/s $\color{#d91a1a}-2.53\%$
test_step_mdp_speed[False-False-False-True-True] 0.1804ms 35.3632μs 28.2780 KOps/s 28.9995 KOps/s $\color{#d91a1a}-2.49\%$
test_step_mdp_speed[False-False-False-True-False] 88.3150μs 23.7613μs 42.0852 KOps/s 42.4252 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-False-False-False-True] 74.5090μs 22.4770μs 44.4900 KOps/s 45.0807 KOps/s $\color{#d91a1a}-1.31\%$
test_step_mdp_speed[False-False-False-False-False] 41.3570μs 15.1121μs 66.1721 KOps/s 66.9222 KOps/s $\color{#d91a1a}-1.12\%$
test_values[generalized_advantage_estimate-True-True] 10.0659ms 9.5008ms 105.2544 Ops/s 101.1693 Ops/s $\color{#35bf28}+4.04\%$
test_values[vec_generalized_advantage_estimate-True-True] 39.1672ms 35.7682ms 27.9578 Ops/s 29.6964 Ops/s $\textbf{\color{#d91a1a}-5.85\%}$
test_values[td0_return_estimate-False-False] 0.2257ms 0.1652ms 6.0523 KOps/s 5.7986 KOps/s $\color{#35bf28}+4.38\%$
test_values[td1_return_estimate-False-False] 23.7828ms 23.4538ms 42.6371 Ops/s 41.5641 Ops/s $\color{#35bf28}+2.58\%$
test_values[vec_td1_return_estimate-False-False] 38.5329ms 35.8844ms 27.8672 Ops/s 29.6680 Ops/s $\textbf{\color{#d91a1a}-6.07\%}$
test_values[td_lambda_return_estimate-True-False] 36.9259ms 33.8772ms 29.5184 Ops/s 28.4451 Ops/s $\color{#35bf28}+3.77\%$
test_values[vec_td_lambda_return_estimate-True-False] 45.5237ms 36.5377ms 27.3690 Ops/s 29.5229 Ops/s $\textbf{\color{#d91a1a}-7.30\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 13.6187ms 8.5742ms 116.6285 Ops/s 117.7144 Ops/s $\color{#d91a1a}-0.92\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.1825ms 1.8356ms 544.7725 Ops/s 582.6464 Ops/s $\textbf{\color{#d91a1a}-6.50\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5934ms 0.3580ms 2.7930 KOps/s 2.8214 KOps/s $\color{#d91a1a}-1.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 43.2329ms 41.0168ms 24.3802 Ops/s 24.5689 Ops/s $\color{#d91a1a}-0.77\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 5.4429ms 3.0951ms 323.0908 Ops/s 327.7503 Ops/s $\color{#d91a1a}-1.42\%$
test_dqn_speed[False-None] 1.8784ms 1.3163ms 759.6984 Ops/s 756.8642 Ops/s $\color{#35bf28}+0.37\%$
test_dqn_speed[False-backward] 1.8659ms 1.7791ms 562.0889 Ops/s 563.2611 Ops/s $\color{#d91a1a}-0.21\%$
test_dqn_speed[True-None] 0.9892ms 0.4581ms 2.1828 KOps/s 2.0002 KOps/s $\textbf{\color{#35bf28}+9.13\%}$
test_dqn_speed[True-backward] 0.9515ms 0.8753ms 1.1425 KOps/s 1.1095 KOps/s $\color{#35bf28}+2.97\%$
test_dqn_speed[reduce-overhead-None] 0.6369ms 0.4634ms 2.1580 KOps/s 2.0752 KOps/s $\color{#35bf28}+3.99\%$
test_dqn_speed[reduce-overhead-backward] 0.9340ms 0.8768ms 1.1404 KOps/s 1.0962 KOps/s $\color{#35bf28}+4.04\%$
test_ddpg_speed[False-None] 3.5032ms 2.7327ms 365.9388 Ops/s 358.4068 Ops/s $\color{#35bf28}+2.10\%$
test_ddpg_speed[False-backward] 5.0254ms 3.9844ms 250.9781 Ops/s 249.6096 Ops/s $\color{#35bf28}+0.55\%$
test_ddpg_speed[True-None] 1.3408ms 1.0006ms 999.4123 Ops/s 969.8685 Ops/s $\color{#35bf28}+3.05\%$
test_ddpg_speed[True-backward] 2.1105ms 1.9097ms 523.6496 Ops/s 502.3493 Ops/s $\color{#35bf28}+4.24\%$
test_ddpg_speed[reduce-overhead-None] 1.1664ms 0.9909ms 1.0092 KOps/s 976.3969 Ops/s $\color{#35bf28}+3.36\%$
test_ddpg_speed[reduce-overhead-backward] 1.9175ms 1.8766ms 532.8717 Ops/s 498.6534 Ops/s $\textbf{\color{#35bf28}+6.86\%}$
test_sac_speed[False-None] 9.1706ms 7.7050ms 129.7854 Ops/s 120.4244 Ops/s $\textbf{\color{#35bf28}+7.77\%}$
test_sac_speed[False-backward] 10.9009ms 10.4572ms 95.6276 Ops/s 89.4108 Ops/s $\textbf{\color{#35bf28}+6.95\%}$
test_sac_speed[True-None] 2.5001ms 1.8371ms 544.3441 Ops/s 537.4531 Ops/s $\color{#35bf28}+1.28\%$
test_sac_speed[True-backward] 3.6656ms 3.4955ms 286.0828 Ops/s 284.0826 Ops/s $\color{#35bf28}+0.70\%$
test_sac_speed[reduce-overhead-None] 2.5267ms 1.8142ms 551.2110 Ops/s 532.0218 Ops/s $\color{#35bf28}+3.61\%$
test_sac_speed[reduce-overhead-backward] 3.9840ms 3.4905ms 286.4898 Ops/s 283.7138 Ops/s $\color{#35bf28}+0.98\%$
test_redq_speed[False-None] 14.0484ms 12.8020ms 78.1125 Ops/s 73.1471 Ops/s $\textbf{\color{#35bf28}+6.79\%}$
test_redq_speed[False-backward] 23.6563ms 21.9646ms 45.5277 Ops/s 43.6967 Ops/s $\color{#35bf28}+4.19\%$
test_redq_speed[True-None] 5.6977ms 4.4523ms 224.6051 Ops/s 197.8334 Ops/s $\textbf{\color{#35bf28}+13.53\%}$
test_redq_speed[True-backward] 12.4588ms 11.8994ms 84.0378 Ops/s 80.9862 Ops/s $\color{#35bf28}+3.77\%$
test_redq_speed[reduce-overhead-None] 5.3091ms 4.4667ms 223.8792 Ops/s 216.9997 Ops/s $\color{#35bf28}+3.17\%$
test_redq_speed[reduce-overhead-backward] 12.6845ms 11.8515ms 84.3778 Ops/s 79.3546 Ops/s $\textbf{\color{#35bf28}+6.33\%}$
test_redq_deprec_speed[False-None] 13.6938ms 12.2905ms 81.3638 Ops/s 72.0266 Ops/s $\textbf{\color{#35bf28}+12.96\%}$
test_redq_deprec_speed[False-backward] 19.5817ms 18.0268ms 55.4730 Ops/s 33.5197 Ops/s $\textbf{\color{#35bf28}+65.49\%}$
test_redq_deprec_speed[True-None] 4.2895ms 3.5319ms 283.1341 Ops/s 261.8723 Ops/s $\textbf{\color{#35bf28}+8.12\%}$
test_redq_deprec_speed[True-backward] 9.3419ms 8.2030ms 121.9067 Ops/s 113.3467 Ops/s $\textbf{\color{#35bf28}+7.55\%}$
test_redq_deprec_speed[reduce-overhead-None] 4.0981ms 3.5566ms 281.1675 Ops/s 262.4234 Ops/s $\textbf{\color{#35bf28}+7.14\%}$
test_redq_deprec_speed[reduce-overhead-backward] 8.2354ms 8.0093ms 124.8541 Ops/s 114.1270 Ops/s $\textbf{\color{#35bf28}+9.40\%}$
test_td3_speed[False-None] 9.0238ms 7.6605ms 130.5404 Ops/s 123.5776 Ops/s $\textbf{\color{#35bf28}+5.63\%}$
test_td3_speed[False-backward] 11.2837ms 10.3666ms 96.4636 Ops/s 92.9730 Ops/s $\color{#35bf28}+3.75\%$
test_td3_speed[True-None] 1.9944ms 1.7140ms 583.4418 Ops/s 575.9153 Ops/s $\color{#35bf28}+1.31\%$
test_td3_speed[True-backward] 3.6214ms 3.4364ms 290.9985 Ops/s 290.0314 Ops/s $\color{#35bf28}+0.33\%$
test_td3_speed[reduce-overhead-None] 2.4801ms 1.7278ms 578.7816 Ops/s 581.0294 Ops/s $\color{#d91a1a}-0.39\%$
test_td3_speed[reduce-overhead-backward] 3.5674ms 3.3954ms 294.5187 Ops/s 302.0682 Ops/s $\color{#d91a1a}-2.50\%$
test_cql_speed[False-None] 38.0172ms 35.6059ms 28.0852 Ops/s 27.1297 Ops/s $\color{#35bf28}+3.52\%$
test_cql_speed[False-backward] 51.6206ms 47.0298ms 21.2631 Ops/s 20.9970 Ops/s $\color{#35bf28}+1.27\%$
test_cql_speed[True-None] 16.0998ms 15.6596ms 63.8587 Ops/s 62.1472 Ops/s $\color{#35bf28}+2.75\%$
test_cql_speed[True-backward] 23.4853ms 22.6105ms 44.2272 Ops/s 44.8973 Ops/s $\color{#d91a1a}-1.49\%$
test_cql_speed[reduce-overhead-None] 16.5206ms 15.9234ms 62.8008 Ops/s 64.6264 Ops/s $\color{#d91a1a}-2.82\%$
test_cql_speed[reduce-overhead-backward] 23.6621ms 21.9839ms 45.4879 Ops/s 44.6742 Ops/s $\color{#35bf28}+1.82\%$
test_a2c_speed[False-None] 8.6847ms 7.0241ms 142.3678 Ops/s 137.7766 Ops/s $\color{#35bf28}+3.33\%$
test_a2c_speed[False-backward] 14.1641ms 13.7924ms 72.5035 Ops/s 68.5391 Ops/s $\textbf{\color{#35bf28}+5.78\%}$
test_a2c_speed[True-None] 3.7321ms 3.2725ms 305.5740 Ops/s 298.1811 Ops/s $\color{#35bf28}+2.48\%$
test_a2c_speed[True-backward] 10.0179ms 9.6211ms 103.9380 Ops/s 104.1118 Ops/s $\color{#d91a1a}-0.17\%$
test_a2c_speed[reduce-overhead-None] 3.9535ms 3.2935ms 303.6300 Ops/s 302.3233 Ops/s $\color{#35bf28}+0.43\%$
test_a2c_speed[reduce-overhead-backward] 9.9901ms 9.5874ms 104.3038 Ops/s 103.2332 Ops/s $\color{#35bf28}+1.04\%$
test_ppo_speed[False-None] 8.3255ms 7.2476ms 137.9762 Ops/s 132.7471 Ops/s $\color{#35bf28}+3.94\%$
test_ppo_speed[False-backward] 15.0037ms 14.3476ms 69.6980 Ops/s 67.1631 Ops/s $\color{#35bf28}+3.77\%$
test_ppo_speed[True-None] 4.9210ms 3.7731ms 265.0360 Ops/s 271.4195 Ops/s $\color{#d91a1a}-2.35\%$
test_ppo_speed[True-backward] 9.8375ms 9.4581ms 105.7293 Ops/s 101.3593 Ops/s $\color{#35bf28}+4.31\%$
test_ppo_speed[reduce-overhead-None] 4.0197ms 3.6790ms 271.8162 Ops/s 271.8251 Ops/s $-0.00\%$
test_ppo_speed[reduce-overhead-backward] 10.6288ms 9.5249ms 104.9881 Ops/s 104.2432 Ops/s $\color{#35bf28}+0.71\%$
test_reinforce_speed[False-None] 8.3829ms 6.3782ms 156.7849 Ops/s 149.8261 Ops/s $\color{#35bf28}+4.64\%$
test_reinforce_speed[False-backward] 9.8610ms 9.5529ms 104.6799 Ops/s 100.1252 Ops/s $\color{#35bf28}+4.55\%$
test_reinforce_speed[True-None] 3.2040ms 2.6216ms 381.4443 Ops/s 370.9008 Ops/s $\color{#35bf28}+2.84\%$
test_reinforce_speed[True-backward] 9.3414ms 8.5673ms 116.7234 Ops/s 117.7040 Ops/s $\color{#d91a1a}-0.83\%$
test_reinforce_speed[reduce-overhead-None] 3.1744ms 2.6245ms 381.0237 Ops/s 372.3176 Ops/s $\color{#35bf28}+2.34\%$
test_reinforce_speed[reduce-overhead-backward] 10.1969ms 8.5396ms 117.1012 Ops/s 117.1697 Ops/s $\color{#d91a1a}-0.06\%$
test_iql_speed[False-None] 32.6268ms 31.4644ms 31.7819 Ops/s 30.0202 Ops/s $\textbf{\color{#35bf28}+5.87\%}$
test_iql_speed[False-backward] 63.7422ms 44.7654ms 22.3387 Ops/s 21.5803 Ops/s $\color{#35bf28}+3.51\%$
test_iql_speed[True-None] 11.8743ms 10.7651ms 92.8932 Ops/s 94.0586 Ops/s $\color{#d91a1a}-1.24\%$
test_iql_speed[True-backward] 22.7359ms 21.4346ms 46.6536 Ops/s 47.0052 Ops/s $\color{#d91a1a}-0.75\%$
test_iql_speed[reduce-overhead-None] 11.6146ms 10.9010ms 91.7348 Ops/s 94.6882 Ops/s $\color{#d91a1a}-3.12\%$
test_iql_speed[reduce-overhead-backward] 22.4247ms 21.2820ms 46.9881 Ops/s 46.6266 Ops/s $\color{#35bf28}+0.78\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8724ms 4.6988ms 212.8192 Ops/s 203.0183 Ops/s $\color{#35bf28}+4.83\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4128ms 0.5028ms 1.9889 KOps/s 1.9671 KOps/s $\color{#35bf28}+1.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7017ms 0.4757ms 2.1020 KOps/s 2.0722 KOps/s $\color{#35bf28}+1.44\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1206ms 4.4776ms 223.3337 Ops/s 210.1375 Ops/s $\textbf{\color{#35bf28}+6.28\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0028ms 0.4903ms 2.0396 KOps/s 1.9887 KOps/s $\color{#35bf28}+2.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6929ms 0.4658ms 2.1470 KOps/s 2.1018 KOps/s $\color{#35bf28}+2.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.5116ms 1.6089ms 621.5436 Ops/s 614.0936 Ops/s $\color{#35bf28}+1.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.8535ms 1.5582ms 641.7809 Ops/s 636.3987 Ops/s $\color{#35bf28}+0.85\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 8.0648ms 4.7239ms 211.6879 Ops/s 205.6832 Ops/s $\color{#35bf28}+2.92\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9607ms 0.6286ms 1.5909 KOps/s 1.5724 KOps/s $\color{#35bf28}+1.18\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8420ms 0.6041ms 1.6554 KOps/s 1.6205 KOps/s $\color{#35bf28}+2.16\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.9771ms 4.5480ms 219.8792 Ops/s 208.8207 Ops/s $\textbf{\color{#35bf28}+5.30\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.9072ms 0.5066ms 1.9740 KOps/s 1.8469 KOps/s $\textbf{\color{#35bf28}+6.88\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.1575ms 0.4890ms 2.0449 KOps/s 2.0123 KOps/s $\color{#35bf28}+1.62\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.6784ms 4.6667ms 214.2830 Ops/s 210.8632 Ops/s $\color{#35bf28}+1.62\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6602ms 0.4923ms 2.0314 KOps/s 1.9930 KOps/s $\color{#35bf28}+1.93\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 8.1364ms 0.4900ms 2.0409 KOps/s 2.0788 KOps/s $\color{#d91a1a}-1.83\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9709ms 4.6990ms 212.8099 Ops/s 201.5931 Ops/s $\textbf{\color{#35bf28}+5.56\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4369ms 0.6314ms 1.5837 KOps/s 1.5377 KOps/s $\color{#35bf28}+2.99\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7653ms 0.6098ms 1.6400 KOps/s 1.6371 KOps/s $\color{#35bf28}+0.17\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.3916s 11.9245ms 83.8610 Ops/s 255.4353 Ops/s $\textbf{\color{#d91a1a}-67.17\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 3.4249ms 2.0639ms 484.5235 Ops/s 399.9153 Ops/s $\textbf{\color{#35bf28}+21.16\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.8778ms 1.2224ms 818.0635 Ops/s 807.0662 Ops/s $\color{#35bf28}+1.36\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 5.6519ms 4.2149ms 237.2524 Ops/s 38.0709 Ops/s $\textbf{\color{#35bf28}+523.19\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 6.9622ms 2.2820ms 438.2080 Ops/s 442.8209 Ops/s $\color{#d91a1a}-1.04\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.3298ms 1.3783ms 725.5417 Ops/s 833.1732 Ops/s $\textbf{\color{#d91a1a}-12.92\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.3870s 12.0595ms 82.9224 Ops/s 226.0223 Ops/s $\textbf{\color{#d91a1a}-63.31\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.9384ms 2.4904ms 401.5388 Ops/s 397.0900 Ops/s $\color{#35bf28}+1.12\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.5859ms 1.4538ms 687.8438 Ops/s 682.2630 Ops/s $\color{#35bf28}+0.82\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 11.0410ms 10.6754ms 93.6730 Ops/s 92.2579 Ops/s $\color{#35bf28}+1.53\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 16.6144ms 14.4410ms 69.2474 Ops/s 69.4968 Ops/s $\color{#d91a1a}-0.36\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 20.4324ms 19.5923ms 51.0406 Ops/s 49.7507 Ops/s $\color{#35bf28}+2.59\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 16.0837ms 14.4746ms 69.0866 Ops/s 68.6867 Ops/s $\color{#35bf28}+0.58\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 21.0686ms 19.5916ms 51.0424 Ops/s 50.0448 Ops/s $\color{#35bf28}+1.99\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 16.1049ms 15.6403ms 63.9375 Ops/s 64.1845 Ops/s $\color{#d91a1a}-0.38\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}21$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.7327s 0.7320s 1.3662 Ops/s 1.3260 Ops/s $\color{#35bf28}+3.03\%$
test_transformed 0.9763s 0.9754s 1.0252 Ops/s 1.0296 Ops/s $\color{#d91a1a}-0.43\%$
test_serial 2.1088s 2.1028s 0.4755 Ops/s 0.4754 Ops/s $\color{#35bf28}+0.03\%$
test_parallel 1.9618s 1.9113s 0.5232 Ops/s 0.5080 Ops/s $\color{#35bf28}+3.00\%$
test_step_mdp_speed[True-True-True-True-True] 0.2682ms 35.8459μs 27.8972 KOps/s 28.2904 KOps/s $\color{#d91a1a}-1.39\%$
test_step_mdp_speed[True-True-True-True-False] 0.1357ms 19.9947μs 50.0132 KOps/s 49.7025 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-True-True-False-True] 49.8900μs 19.6658μs 50.8497 KOps/s 50.7571 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-True-True-False-False] 37.7500μs 11.4799μs 87.1085 KOps/s 86.7392 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[True-True-False-True-True] 0.1164ms 36.2571μs 27.5808 KOps/s 26.4743 KOps/s $\color{#35bf28}+4.18\%$
test_step_mdp_speed[True-True-False-True-False] 99.2110μs 21.8224μs 45.8245 KOps/s 45.2631 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-True-False-False-True] 52.1800μs 21.7703μs 45.9342 KOps/s 46.2809 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-False-False-False] 98.9410μs 13.5447μs 73.8298 KOps/s 74.5402 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[True-False-True-True-True] 0.1727ms 39.9599μs 25.0251 KOps/s 25.1042 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-False-True-True-False] 56.7000μs 24.3136μs 41.1292 KOps/s 41.7239 KOps/s $\color{#d91a1a}-1.43\%$
test_step_mdp_speed[True-False-True-False-True] 56.8410μs 21.7694μs 45.9360 KOps/s 46.3802 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-False-True-False-False] 0.2173ms 13.5823μs 73.6254 KOps/s 74.4272 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[True-False-False-True-True] 77.4710μs 41.6959μs 23.9832 KOps/s 24.0993 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-False-False-True-False] 61.2900μs 26.5232μs 37.7029 KOps/s 38.1833 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[True-False-False-False-True] 58.4610μs 23.7291μs 42.1423 KOps/s 42.8580 KOps/s $\color{#d91a1a}-1.67\%$
test_step_mdp_speed[True-False-False-False-False] 0.1060ms 15.4055μs 64.9120 KOps/s 66.2834 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[False-True-True-True-True] 73.1210μs 40.2348μs 24.8541 KOps/s 25.1854 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[False-True-True-True-False] 0.1112ms 24.3881μs 41.0036 KOps/s 41.3466 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[False-True-True-False-True] 57.2310μs 25.1736μs 39.7242 KOps/s 38.9875 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-True-True-False-False] 0.1867ms 15.0741μs 66.3389 KOps/s 66.8013 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-True-False-True-True] 81.0500μs 41.3688μs 24.1728 KOps/s 23.8050 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[False-True-False-True-False] 67.4210μs 26.2423μs 38.1065 KOps/s 38.1704 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[False-True-False-False-True] 4.0432ms 26.8782μs 37.2048 KOps/s 36.6535 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-True-False-False-False] 48.7510μs 17.0493μs 58.6536 KOps/s 59.2791 KOps/s $\color{#d91a1a}-1.06\%$
test_step_mdp_speed[False-False-True-True-True] 0.1213ms 43.6096μs 22.9307 KOps/s 23.0208 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[False-False-True-True-False] 72.6000μs 28.3597μs 35.2613 KOps/s 35.5451 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-False-True-False-True] 0.1027ms 26.9227μs 37.1434 KOps/s 37.1934 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-False-True-False-False] 52.7210μs 17.0969μs 58.4900 KOps/s 59.2568 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[False-False-False-True-True] 0.1024ms 45.4508μs 22.0018 KOps/s 22.1983 KOps/s $\color{#d91a1a}-0.89\%$
test_step_mdp_speed[False-False-False-True-False] 71.2010μs 29.6283μs 33.7515 KOps/s 33.1518 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[False-False-False-False-True] 70.9300μs 28.3104μs 35.3228 KOps/s 35.0434 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-False-False-False-False] 62.4510μs 19.0161μs 52.5870 KOps/s 54.0989 KOps/s $\color{#d91a1a}-2.79\%$
test_values[generalized_advantage_estimate-True-True] 26.2707ms 24.9058ms 40.1513 Ops/s 39.5302 Ops/s $\color{#35bf28}+1.57\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1098s 3.0774ms 324.9495 Ops/s 352.0926 Ops/s $\textbf{\color{#d91a1a}-7.71\%}$
test_values[td0_return_estimate-False-False] 86.1210μs 64.6077μs 15.4780 KOps/s 14.8264 KOps/s $\color{#35bf28}+4.40\%$
test_values[td1_return_estimate-False-False] 59.1184ms 56.4077ms 17.7281 Ops/s 17.6742 Ops/s $\color{#35bf28}+0.30\%$
test_values[vec_td1_return_estimate-False-False] 1.3556ms 1.0712ms 933.5056 Ops/s 942.0297 Ops/s $\color{#d91a1a}-0.90\%$
test_values[td_lambda_return_estimate-True-False] 92.9949ms 88.5336ms 11.2951 Ops/s 10.9167 Ops/s $\color{#35bf28}+3.47\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3900ms 1.0733ms 931.7176 Ops/s 935.6270 Ops/s $\color{#d91a1a}-0.42\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.3801ms 24.8320ms 40.2707 Ops/s 38.2642 Ops/s $\textbf{\color{#35bf28}+5.24\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0404ms 0.7316ms 1.3669 KOps/s 1.3732 KOps/s $\color{#d91a1a}-0.46\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8020ms 0.6494ms 1.5398 KOps/s 1.5442 KOps/s $\color{#d91a1a}-0.28\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6766ms 1.4614ms 684.2745 Ops/s 679.7948 Ops/s $\color{#35bf28}+0.66\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8116ms 0.6631ms 1.5080 KOps/s 1.5109 KOps/s $\color{#d91a1a}-0.19\%$
test_dqn_speed[False-None] 0.1039s 1.4721ms 679.3067 Ops/s 765.6689 Ops/s $\textbf{\color{#d91a1a}-11.28\%}$
test_dqn_speed[False-backward] 2.0382ms 1.8665ms 535.7660 Ops/s 552.4190 Ops/s $\color{#d91a1a}-3.01\%$
test_dqn_speed[True-None] 1.2274ms 0.5635ms 1.7747 KOps/s 1.7790 KOps/s $\color{#d91a1a}-0.25\%$
test_dqn_speed[True-backward] 1.2196ms 1.0325ms 968.5396 Ops/s 809.6098 Ops/s $\textbf{\color{#35bf28}+19.63\%}$
test_dqn_speed[reduce-overhead-None] 0.7583ms 0.5714ms 1.7502 KOps/s 1.7319 KOps/s $\color{#35bf28}+1.06\%$
test_dqn_speed[reduce-overhead-backward] 1.0782ms 1.0194ms 980.9975 Ops/s 980.9609 Ops/s $+0.00\%$
test_ddpg_speed[False-None] 3.3132ms 2.7083ms 369.2347 Ops/s 378.4663 Ops/s $\color{#d91a1a}-2.44\%$
test_ddpg_speed[False-backward] 4.3729ms 3.9824ms 251.1060 Ops/s 255.2006 Ops/s $\color{#d91a1a}-1.60\%$
test_ddpg_speed[True-None] 1.5447ms 1.2445ms 803.5285 Ops/s 770.7451 Ops/s $\color{#35bf28}+4.25\%$
test_ddpg_speed[True-backward] 2.4110ms 2.2264ms 449.1559 Ops/s 355.6672 Ops/s $\textbf{\color{#35bf28}+26.29\%}$
test_ddpg_speed[reduce-overhead-None] 1.4184ms 1.2407ms 806.0131 Ops/s 778.9319 Ops/s $\color{#35bf28}+3.48\%$
test_ddpg_speed[reduce-overhead-backward] 2.3755ms 2.2641ms 441.6807 Ops/s 441.8391 Ops/s $\color{#d91a1a}-0.04\%$
test_sac_speed[False-None] 8.4612ms 7.4796ms 133.6968 Ops/s 134.5384 Ops/s $\color{#d91a1a}-0.63\%$
test_sac_speed[False-backward] 11.5320ms 10.7720ms 92.8332 Ops/s 93.2553 Ops/s $\color{#d91a1a}-0.45\%$
test_sac_speed[True-None] 2.3030ms 2.0093ms 497.6759 Ops/s 494.9984 Ops/s $\color{#35bf28}+0.54\%$
test_sac_speed[True-backward] 4.2555ms 4.0165ms 248.9740 Ops/s 234.9333 Ops/s $\textbf{\color{#35bf28}+5.98\%}$
test_sac_speed[reduce-overhead-None] 2.3975ms 2.0242ms 494.0115 Ops/s 486.8935 Ops/s $\color{#35bf28}+1.46\%$
test_sac_speed[reduce-overhead-backward] 4.1107ms 3.9833ms 251.0490 Ops/s 250.5450 Ops/s $\color{#35bf28}+0.20\%$
test_redq_speed[False-None] 16.0472ms 11.2374ms 88.9888 Ops/s 93.2904 Ops/s $\color{#d91a1a}-4.61\%$
test_redq_speed[False-backward] 18.6638ms 17.8454ms 56.0369 Ops/s 55.6833 Ops/s $\color{#35bf28}+0.64\%$
test_redq_speed[True-None] 3.9998ms 3.5583ms 281.0291 Ops/s 288.1193 Ops/s $\color{#d91a1a}-2.46\%$
test_redq_speed[True-backward] 10.9099ms 9.0313ms 110.7264 Ops/s 105.3811 Ops/s $\textbf{\color{#35bf28}+5.07\%}$
test_redq_speed[reduce-overhead-None] 3.9979ms 3.5938ms 278.2539 Ops/s 286.9470 Ops/s $\color{#d91a1a}-3.03\%$
test_redq_speed[reduce-overhead-backward] 9.0493ms 8.7088ms 114.8260 Ops/s 120.3420 Ops/s $\color{#d91a1a}-4.58\%$
test_redq_deprec_speed[False-None] 11.0160ms 10.6091ms 94.2586 Ops/s 96.3825 Ops/s $\color{#d91a1a}-2.20\%$
test_redq_deprec_speed[False-backward] 17.4670ms 15.5793ms 64.1878 Ops/s 65.9279 Ops/s $\color{#d91a1a}-2.64\%$
test_redq_deprec_speed[True-None] 3.5414ms 3.2334ms 309.2686 Ops/s 295.5404 Ops/s $\color{#35bf28}+4.65\%$
test_redq_deprec_speed[True-backward] 7.6914ms 7.3192ms 136.6267 Ops/s 137.7810 Ops/s $\color{#d91a1a}-0.84\%$
test_redq_deprec_speed[reduce-overhead-None] 3.4618ms 3.2019ms 312.3141 Ops/s 307.0311 Ops/s $\color{#35bf28}+1.72\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.4498ms 7.2340ms 138.2359 Ops/s 137.9725 Ops/s $\color{#35bf28}+0.19\%$
test_td3_speed[False-None] 7.5693ms 7.4139ms 134.8826 Ops/s 134.9404 Ops/s $\color{#d91a1a}-0.04\%$
test_td3_speed[False-backward] 10.8700ms 10.3467ms 96.6496 Ops/s 96.0874 Ops/s $\color{#35bf28}+0.59\%$
test_td3_speed[True-None] 1.9630ms 1.9015ms 525.8945 Ops/s 526.6383 Ops/s $\color{#d91a1a}-0.14\%$
test_td3_speed[True-backward] 3.8950ms 3.7271ms 268.3046 Ops/s 218.7328 Ops/s $\textbf{\color{#35bf28}+22.66\%}$
test_td3_speed[reduce-overhead-None] 1.9170ms 1.8847ms 530.5823 Ops/s 524.0289 Ops/s $\color{#35bf28}+1.25\%$
test_td3_speed[reduce-overhead-backward] 4.0523ms 3.7374ms 267.5651 Ops/s 268.9602 Ops/s $\color{#d91a1a}-0.52\%$
test_cql_speed[False-None] 28.4436ms 25.4000ms 39.3701 Ops/s 40.7988 Ops/s $\color{#d91a1a}-3.50\%$
test_cql_speed[False-backward] 39.3751ms 34.8978ms 28.6551 Ops/s 29.1941 Ops/s $\color{#d91a1a}-1.85\%$
test_cql_speed[True-None] 11.4149ms 11.0218ms 90.7289 Ops/s 92.9819 Ops/s $\color{#d91a1a}-2.42\%$
test_cql_speed[True-backward] 17.4962ms 16.9704ms 58.9260 Ops/s 58.7871 Ops/s $\color{#35bf28}+0.24\%$
test_cql_speed[reduce-overhead-None] 11.3786ms 11.0201ms 90.7435 Ops/s 91.6544 Ops/s $\color{#d91a1a}-0.99\%$
test_cql_speed[reduce-overhead-backward] 17.7078ms 16.9770ms 58.9034 Ops/s 59.9305 Ops/s $\color{#d91a1a}-1.71\%$
test_a2c_speed[False-None] 5.4652ms 5.1972ms 192.4115 Ops/s 186.5233 Ops/s $\color{#35bf28}+3.16\%$
test_a2c_speed[False-backward] 11.9327ms 11.5618ms 86.4918 Ops/s 83.1345 Ops/s $\color{#35bf28}+4.04\%$
test_a2c_speed[True-None] 3.2570ms 3.0505ms 327.8105 Ops/s 328.1988 Ops/s $\color{#d91a1a}-0.12\%$
test_a2c_speed[True-backward] 8.7856ms 8.4848ms 117.8574 Ops/s 114.6702 Ops/s $\color{#35bf28}+2.78\%$
test_a2c_speed[reduce-overhead-None] 3.2624ms 3.0475ms 328.1326 Ops/s 327.4056 Ops/s $\color{#35bf28}+0.22\%$
test_a2c_speed[reduce-overhead-backward] 8.9718ms 8.4961ms 117.7010 Ops/s 118.0758 Ops/s $\color{#d91a1a}-0.32\%$
test_ppo_speed[False-None] 6.0041ms 5.7106ms 175.1144 Ops/s 177.9529 Ops/s $\color{#d91a1a}-1.60\%$
test_ppo_speed[False-backward] 13.4975ms 12.6800ms 78.8644 Ops/s 81.9216 Ops/s $\color{#d91a1a}-3.73\%$
test_ppo_speed[True-None] 3.7671ms 3.4096ms 293.2918 Ops/s 293.9078 Ops/s $\color{#d91a1a}-0.21\%$
test_ppo_speed[True-backward] 8.5129ms 8.2594ms 121.0739 Ops/s 118.1746 Ops/s $\color{#35bf28}+2.45\%$
test_ppo_speed[reduce-overhead-None] 3.5923ms 3.4106ms 293.2057 Ops/s 294.0571 Ops/s $\color{#d91a1a}-0.29\%$
test_ppo_speed[reduce-overhead-backward] 8.5240ms 8.2042ms 121.8889 Ops/s 122.4039 Ops/s $\color{#d91a1a}-0.42\%$
test_reinforce_speed[False-None] 4.8987ms 4.4879ms 222.8229 Ops/s 228.8848 Ops/s $\color{#d91a1a}-2.65\%$
test_reinforce_speed[False-backward] 7.8409ms 7.4722ms 133.8300 Ops/s 137.1636 Ops/s $\color{#d91a1a}-2.43\%$
test_reinforce_speed[True-None] 2.6193ms 2.2602ms 442.4318 Ops/s 436.3206 Ops/s $\color{#35bf28}+1.40\%$
test_reinforce_speed[True-backward] 7.6131ms 7.0827ms 141.1898 Ops/s 129.4191 Ops/s $\textbf{\color{#35bf28}+9.09\%}$
test_reinforce_speed[reduce-overhead-None] 2.6233ms 2.2259ms 449.2471 Ops/s 451.6236 Ops/s $\color{#d91a1a}-0.53\%$
test_reinforce_speed[reduce-overhead-backward] 7.4616ms 7.1208ms 140.4345 Ops/s 139.7487 Ops/s $\color{#35bf28}+0.49\%$
test_iql_speed[False-None] 20.8079ms 19.8303ms 50.4279 Ops/s 52.5693 Ops/s $\color{#d91a1a}-4.07\%$
test_iql_speed[False-backward] 31.6531ms 30.9721ms 32.2871 Ops/s 33.0416 Ops/s $\color{#d91a1a}-2.28\%$
test_iql_speed[True-None] 7.3670ms 6.8441ms 146.1104 Ops/s 146.6230 Ops/s $\color{#d91a1a}-0.35\%$
test_iql_speed[True-backward] 16.2432ms 15.5912ms 64.1387 Ops/s 59.8953 Ops/s $\textbf{\color{#35bf28}+7.08\%}$
test_iql_speed[reduce-overhead-None] 7.2814ms 6.8212ms 146.6022 Ops/s 140.9293 Ops/s $\color{#35bf28}+4.03\%$
test_iql_speed[reduce-overhead-backward] 16.5013ms 15.6386ms 63.9443 Ops/s 63.7623 Ops/s $\color{#35bf28}+0.29\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.4588ms 6.2573ms 159.8145 Ops/s 158.0652 Ops/s $\color{#35bf28}+1.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.9643ms 0.3728ms 2.6827 KOps/s 3.1417 KOps/s $\textbf{\color{#d91a1a}-14.61\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6417ms 0.3112ms 3.2132 KOps/s 2.9749 KOps/s $\textbf{\color{#35bf28}+8.01\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3470ms 6.0207ms 166.0949 Ops/s 164.1604 Ops/s $\color{#35bf28}+1.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.3113ms 0.2805ms 3.5647 KOps/s 3.0264 KOps/s $\textbf{\color{#35bf28}+17.79\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5558ms 0.2666ms 3.7509 KOps/s 3.0645 KOps/s $\textbf{\color{#35bf28}+22.40\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.4797ms 1.2617ms 792.5551 Ops/s 790.4323 Ops/s $\color{#35bf28}+0.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5414ms 1.2071ms 828.4278 Ops/s 826.7366 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4442ms 6.2402ms 160.2507 Ops/s 157.0344 Ops/s $\color{#35bf28}+2.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3619ms 0.4136ms 2.4176 KOps/s 2.1405 KOps/s $\textbf{\color{#35bf28}+12.95\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6843ms 0.4252ms 2.3516 KOps/s 2.1078 KOps/s $\textbf{\color{#35bf28}+11.57\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.3888ms 6.1658ms 162.1841 Ops/s 162.5993 Ops/s $\color{#d91a1a}-0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.3300ms 0.2885ms 3.4659 KOps/s 2.7785 KOps/s $\textbf{\color{#35bf28}+24.74\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5191ms 0.2982ms 3.3531 KOps/s 2.9292 KOps/s $\textbf{\color{#35bf28}+14.47\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.4489ms 6.0930ms 164.1224 Ops/s 163.3268 Ops/s $\color{#35bf28}+0.49\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7358ms 0.2867ms 3.4874 KOps/s 3.4750 KOps/s $\color{#35bf28}+0.36\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6595ms 0.2423ms 4.1271 KOps/s 3.8717 KOps/s $\textbf{\color{#35bf28}+6.60\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4372ms 6.2305ms 160.5010 Ops/s 159.0520 Ops/s $\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.3495ms 0.4147ms 2.4114 KOps/s 2.1775 KOps/s $\textbf{\color{#35bf28}+10.74\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6470ms 0.3932ms 2.5435 KOps/s 2.1913 KOps/s $\textbf{\color{#35bf28}+16.07\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4710s 14.6928ms 68.0607 Ops/s 187.9832 Ops/s $\textbf{\color{#d91a1a}-63.79\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.6169ms 1.9813ms 504.7270 Ops/s 504.2442 Ops/s $\color{#35bf28}+0.10\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.6558ms 1.2175ms 821.3531 Ops/s 794.4146 Ops/s $\color{#35bf28}+3.39\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 7.3918ms 5.4290ms 184.1946 Ops/s 190.4268 Ops/s $\color{#d91a1a}-3.27\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.4305ms 2.0963ms 477.0386 Ops/s 440.1184 Ops/s $\textbf{\color{#35bf28}+8.39\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.1432ms 1.2390ms 807.0948 Ops/s 819.4202 Ops/s $\color{#d91a1a}-1.50\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4203s 13.9455ms 71.7079 Ops/s 183.3078 Ops/s $\textbf{\color{#d91a1a}-60.88\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 12.7004ms 2.1199ms 471.7271 Ops/s 419.3559 Ops/s $\textbf{\color{#35bf28}+12.49\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.3733ms 1.2684ms 788.3743 Ops/s 728.5743 Ops/s $\textbf{\color{#35bf28}+8.21\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 13.2633ms 12.7328ms 78.5376 Ops/s 77.7734 Ops/s $\color{#35bf28}+0.98\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 17.4472ms 16.5826ms 60.3042 Ops/s 60.4781 Ops/s $\color{#d91a1a}-0.29\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 18.0709ms 17.3346ms 57.6880 Ops/s 56.2781 Ops/s $\color{#35bf28}+2.51\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 17.7746ms 16.8301ms 59.4174 Ops/s 58.9306 Ops/s $\color{#35bf28}+0.83\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 18.2026ms 17.2799ms 57.8705 Ops/s 56.1330 Ops/s $\color{#35bf28}+3.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 19.4586ms 18.6788ms 53.5366 Ops/s 55.7098 Ops/s $\color{#d91a1a}-3.90\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot