Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] python 3.12 nightlies #2281

Merged
merged 2 commits into from
Jul 9, 2024
Merged

[CI] python 3.12 nightlies #2281

merged 2 commits into from
Jul 9, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 9, 2024

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 9, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2281

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 4 Unrelated Failures

As of commit 3903933 with merge base b81eff3 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 9, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 9, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1101s 59.8983ms 16.6950 Ops/s 17.4108 Ops/s $\color{#d91a1a}-4.11\%$
test_sync 36.8056ms 31.2293ms 32.0212 Ops/s 32.6897 Ops/s $\color{#d91a1a}-2.04\%$
test_async 59.3582ms 29.3755ms 34.0420 Ops/s 34.3271 Ops/s $\color{#d91a1a}-0.83\%$
test_simple 0.3829s 0.3806s 2.6271 Ops/s 2.6402 Ops/s $\color{#d91a1a}-0.50\%$
test_transformed 0.5484s 0.5415s 1.8466 Ops/s 1.8101 Ops/s $\color{#35bf28}+2.02\%$
test_serial 1.3310s 1.2749s 0.7844 Ops/s 0.7833 Ops/s $\color{#35bf28}+0.14\%$
test_parallel 1.1343s 1.0878s 0.9193 Ops/s 0.9093 Ops/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[True-True-True-True-True] 87.6030μs 22.6244μs 44.2000 KOps/s 44.8737 KOps/s $\color{#d91a1a}-1.50\%$
test_step_mdp_speed[True-True-True-True-False] 49.6820μs 13.2160μs 75.6659 KOps/s 75.4294 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-True-True-False-True] 55.1620μs 12.9326μs 77.3240 KOps/s 76.8604 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-True-True-False-False] 34.1530μs 7.7108μs 129.6882 KOps/s 130.0788 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-True-False-True-True] 73.5570μs 23.6553μs 42.2737 KOps/s 41.9816 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[True-True-False-True-False] 80.0690μs 14.3712μs 69.5837 KOps/s 69.0592 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[True-True-False-False-True] 59.6890μs 14.0500μs 71.1744 KOps/s 70.3723 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-True-False-False-False] 34.9560μs 8.9660μs 111.5327 KOps/s 111.5486 KOps/s $\color{#d91a1a}-0.01\%$
test_step_mdp_speed[True-False-True-True-True] 55.5940μs 25.0009μs 39.9985 KOps/s 39.6525 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[True-False-True-True-False] 58.3800μs 15.7101μs 63.6533 KOps/s 63.3028 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[True-False-True-False-True] 44.0420μs 14.1130μs 70.8565 KOps/s 71.0509 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[True-False-True-False-False] 49.5730μs 8.9822μs 111.3314 KOps/s 112.9049 KOps/s $\color{#d91a1a}-1.39\%$
test_step_mdp_speed[True-False-False-True-True] 50.5840μs 26.1777μs 38.2005 KOps/s 38.3635 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-False-False-True-False] 64.9520μs 16.9564μs 58.9747 KOps/s 59.3964 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[True-False-False-False-True] 44.1830μs 15.3455μs 65.1657 KOps/s 66.1195 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[True-False-False-False-False] 49.2020μs 10.0376μs 99.6250 KOps/s 99.1073 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-True-True-True-True] 62.0760μs 24.9339μs 40.1061 KOps/s 40.1817 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[False-True-True-True-False] 63.9090μs 15.7722μs 63.4027 KOps/s 63.2893 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[False-True-True-False-True] 51.2460μs 16.3182μs 61.2814 KOps/s 60.6389 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-True-True-False-False] 0.1449ms 10.1861μs 98.1734 KOps/s 98.2145 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[False-True-False-True-True] 0.1069ms 26.0709μs 38.3569 KOps/s 38.4458 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-True-False-True-False] 39.9040μs 17.0370μs 58.6957 KOps/s 59.3147 KOps/s $\color{#d91a1a}-1.04\%$
test_step_mdp_speed[False-True-False-False-True] 60.0630μs 17.5051μs 57.1264 KOps/s 56.7469 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-True-False-False-False] 35.1760μs 11.3048μs 88.4577 KOps/s 87.8804 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[False-False-True-True-True] 85.3100μs 27.4659μs 36.4088 KOps/s 36.5772 KOps/s $\color{#d91a1a}-0.46\%$
test_step_mdp_speed[False-False-True-True-False] 75.9310μs 18.3176μs 54.5922 KOps/s 54.9215 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[False-False-True-False-True] 53.3190μs 17.5756μs 56.8970 KOps/s 56.7856 KOps/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[False-False-True-False-False] 52.2680μs 11.3308μs 88.2548 KOps/s 89.2899 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[False-False-False-True-True] 44.5830μs 28.9099μs 34.5902 KOps/s 34.8717 KOps/s $\color{#d91a1a}-0.81\%$
test_step_mdp_speed[False-False-False-True-False] 46.6060μs 19.4148μs 51.5072 KOps/s 51.6897 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[False-False-False-False-True] 60.6430μs 18.5300μs 53.9666 KOps/s 53.7374 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-False-False-False-False] 32.8610μs 12.3340μs 81.0764 KOps/s 81.3697 KOps/s $\color{#d91a1a}-0.36\%$
test_values[generalized_advantage_estimate-True-True] 10.2126ms 9.5158ms 105.0883 Ops/s 104.7925 Ops/s $\color{#35bf28}+0.28\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.5169ms 35.0514ms 28.5295 Ops/s 28.3377 Ops/s $\color{#35bf28}+0.68\%$
test_values[td0_return_estimate-False-False] 0.2024ms 0.1716ms 5.8275 KOps/s 5.8678 KOps/s $\color{#d91a1a}-0.69\%$
test_values[td1_return_estimate-False-False] 26.9620ms 23.6468ms 42.2890 Ops/s 42.3350 Ops/s $\color{#d91a1a}-0.11\%$
test_values[vec_td1_return_estimate-False-False] 50.4070ms 35.9697ms 27.8012 Ops/s 28.3181 Ops/s $\color{#d91a1a}-1.83\%$
test_values[td_lambda_return_estimate-True-False] 37.1754ms 34.0102ms 29.4029 Ops/s 29.1814 Ops/s $\color{#35bf28}+0.76\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.4163ms 35.3064ms 28.3235 Ops/s 28.2924 Ops/s $\color{#35bf28}+0.11\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.0865ms 8.3576ms 119.6522 Ops/s 119.5173 Ops/s $\color{#35bf28}+0.11\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5433ms 2.0393ms 490.3563 Ops/s 498.9646 Ops/s $\color{#d91a1a}-1.73\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5075ms 0.3578ms 2.7947 KOps/s 2.7989 KOps/s $\color{#d91a1a}-0.15\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 50.3018ms 47.2926ms 21.1450 Ops/s 20.8994 Ops/s $\color{#35bf28}+1.18\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.5591ms 3.0433ms 328.5893 Ops/s 329.2429 Ops/s $\color{#d91a1a}-0.20\%$
test_dqn_speed 1.9333ms 1.3397ms 746.4125 Ops/s 738.0526 Ops/s $\color{#35bf28}+1.13\%$
test_ddpg_speed 3.1879ms 2.8367ms 352.5238 Ops/s 351.1657 Ops/s $\color{#35bf28}+0.39\%$
test_sac_speed 9.0993ms 8.5388ms 117.1125 Ops/s 116.7836 Ops/s $\color{#35bf28}+0.28\%$
test_redq_speed 20.3832ms 13.6164ms 73.4409 Ops/s 66.4767 Ops/s $\textbf{\color{#35bf28}+10.48\%}$
test_redq_deprec_speed 14.2384ms 13.4274ms 74.4748 Ops/s 71.7763 Ops/s $\color{#35bf28}+3.76\%$
test_td3_speed 17.4440ms 8.5386ms 117.1154 Ops/s 118.3211 Ops/s $\color{#d91a1a}-1.02\%$
test_cql_speed 38.1395ms 36.5297ms 27.3750 Ops/s 27.2297 Ops/s $\color{#35bf28}+0.53\%$
test_a2c_speed 8.5223ms 7.4840ms 133.6189 Ops/s 133.4280 Ops/s $\color{#35bf28}+0.14\%$
test_ppo_speed 9.3192ms 7.7886ms 128.3921 Ops/s 128.4425 Ops/s $\color{#d91a1a}-0.04\%$
test_reinforce_speed 7.7272ms 6.7079ms 149.0780 Ops/s 149.7939 Ops/s $\color{#d91a1a}-0.48\%$
test_iql_speed 33.4568ms 32.6148ms 30.6610 Ops/s 30.6789 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.3127ms 3.5798ms 279.3447 Ops/s 278.4484 Ops/s $\color{#35bf28}+0.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.0002ms 0.5189ms 1.9271 KOps/s 1.7625 KOps/s $\textbf{\color{#35bf28}+9.34\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8078ms 0.4761ms 2.1005 KOps/s 2.0725 KOps/s $\color{#35bf28}+1.35\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8768ms 3.5571ms 281.1274 Ops/s 278.7551 Ops/s $\color{#35bf28}+0.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0012ms 0.4946ms 2.0217 KOps/s 2.0226 KOps/s $\color{#d91a1a}-0.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7602ms 0.4731ms 2.1138 KOps/s 2.0898 KOps/s $\color{#35bf28}+1.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8442ms 1.7253ms 579.6213 Ops/s 573.7052 Ops/s $\color{#35bf28}+1.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3259ms 1.6408ms 609.4631 Ops/s 605.1720 Ops/s $\color{#35bf28}+0.71\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.8553ms 3.6740ms 272.1824 Ops/s 272.0283 Ops/s $\color{#35bf28}+0.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1162s 0.7467ms 1.3392 KOps/s 1.5723 KOps/s $\textbf{\color{#d91a1a}-14.83\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9587ms 0.6105ms 1.6379 KOps/s 1.6150 KOps/s $\color{#35bf28}+1.42\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.2970ms 3.6089ms 277.0957 Ops/s 281.5999 Ops/s $\color{#d91a1a}-1.60\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0164ms 0.4991ms 2.0037 KOps/s 1.9744 KOps/s $\color{#35bf28}+1.48\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7416ms 0.4760ms 2.1010 KOps/s 2.0757 KOps/s $\color{#35bf28}+1.22\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.5289ms 3.5786ms 279.4402 Ops/s 282.3861 Ops/s $\color{#d91a1a}-1.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7071ms 0.4930ms 2.0286 KOps/s 2.0084 KOps/s $\color{#35bf28}+1.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.6980ms 0.4798ms 2.0843 KOps/s 2.0673 KOps/s $\color{#35bf28}+0.82\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.7949ms 3.7220ms 268.6734 Ops/s 273.2480 Ops/s $\color{#d91a1a}-1.67\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6504ms 0.6424ms 1.5567 KOps/s 1.5578 KOps/s $\color{#d91a1a}-0.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7546ms 0.6084ms 1.6436 KOps/s 1.5741 KOps/s $\color{#35bf28}+4.42\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1300s 6.2420ms 160.2045 Ops/s 123.3437 Ops/s $\textbf{\color{#35bf28}+29.88\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.4125ms 12.4966ms 80.0215 Ops/s 78.4048 Ops/s $\color{#35bf28}+2.06\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.1101ms 1.0369ms 964.4537 Ops/s 943.9826 Ops/s $\color{#35bf28}+2.17\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1130s 8.0060ms 124.9059 Ops/s 173.5311 Ops/s $\textbf{\color{#d91a1a}-28.02\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.6903ms 12.3961ms 80.6704 Ops/s 78.9530 Ops/s $\color{#35bf28}+2.18\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.7760ms 1.1177ms 894.6992 Ops/s 943.4740 Ops/s $\textbf{\color{#d91a1a}-5.17\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1075s 5.9193ms 168.9396 Ops/s 167.4683 Ops/s $\color{#35bf28}+0.88\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 14.9566ms 12.5125ms 79.9199 Ops/s 66.8248 Ops/s $\textbf{\color{#35bf28}+19.60\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.7398ms 1.2393ms 806.8874 Ops/s 830.1970 Ops/s $\color{#d91a1a}-2.81\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 9, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1601s 0.1081s 9.2505 Ops/s 9.3281 Ops/s $\color{#d91a1a}-0.83\%$
test_sync 94.6911ms 92.4485ms 10.8168 Ops/s 10.7473 Ops/s $\color{#35bf28}+0.65\%$
test_async 0.1779s 89.7120ms 11.1468 Ops/s 11.1738 Ops/s $\color{#d91a1a}-0.24\%$
test_single_pixels 0.1157s 0.1142s 8.7586 Ops/s 8.5796 Ops/s $\color{#35bf28}+2.09\%$
test_sync_pixels 70.6573ms 69.2984ms 14.4304 Ops/s 14.3350 Ops/s $\color{#35bf28}+0.67\%$
test_async_pixels 0.1190s 57.7230ms 17.3241 Ops/s 15.1323 Ops/s $\textbf{\color{#35bf28}+14.48\%}$
test_simple 0.7264s 0.7260s 1.3774 Ops/s 1.3053 Ops/s $\textbf{\color{#35bf28}+5.52\%}$
test_transformed 1.0235s 0.9650s 1.0362 Ops/s 0.9947 Ops/s $\color{#35bf28}+4.18\%$
test_serial 2.2113s 2.1491s 0.4653 Ops/s 0.4487 Ops/s $\color{#35bf28}+3.71\%$
test_parallel 1.9100s 1.8581s 0.5382 Ops/s 0.5225 Ops/s $\color{#35bf28}+3.00\%$
test_step_mdp_speed[True-True-True-True-True] 86.7220μs 33.2826μs 30.0457 KOps/s 29.1895 KOps/s $\color{#35bf28}+2.93\%$
test_step_mdp_speed[True-True-True-True-False] 44.3710μs 18.9251μs 52.8399 KOps/s 51.3084 KOps/s $\color{#35bf28}+2.99\%$
test_step_mdp_speed[True-True-True-False-True] 42.6410μs 18.7782μs 53.2532 KOps/s 52.6040 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-True-True-False-False] 41.7000μs 10.9532μs 91.2977 KOps/s 89.5756 KOps/s $\color{#35bf28}+1.92\%$
test_step_mdp_speed[True-True-False-True-True] 0.1016ms 35.0828μs 28.5040 KOps/s 27.9460 KOps/s $\color{#35bf28}+2.00\%$
test_step_mdp_speed[True-True-False-True-False] 50.3310μs 20.9538μs 47.7240 KOps/s 46.5744 KOps/s $\color{#35bf28}+2.47\%$
test_step_mdp_speed[True-True-False-False-True] 46.0410μs 20.5604μs 48.6372 KOps/s 47.5381 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[True-True-False-False-False] 32.0610μs 13.0240μs 76.7811 KOps/s 75.9516 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[True-False-True-True-True] 69.7710μs 37.2221μs 26.8658 KOps/s 26.5907 KOps/s $\color{#35bf28}+1.03\%$
test_step_mdp_speed[True-False-True-True-False] 50.9110μs 23.1294μs 43.2349 KOps/s 42.4925 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-False-True-False-True] 46.4510μs 20.9213μs 47.7982 KOps/s 48.3625 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[True-False-True-False-False] 31.4810μs 12.9745μs 77.0745 KOps/s 76.6608 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-False-False-True-True] 69.5310μs 38.9923μs 25.6461 KOps/s 25.1032 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-False-False-True-False] 45.7810μs 24.7592μs 40.3890 KOps/s 39.3688 KOps/s $\color{#35bf28}+2.59\%$
test_step_mdp_speed[True-False-False-False-True] 48.6810μs 22.4955μs 44.4533 KOps/s 43.8840 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-False-False-False-False] 36.5610μs 14.6598μs 68.2136 KOps/s 67.1226 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[False-True-True-True-True] 67.8210μs 37.3145μs 26.7992 KOps/s 26.0470 KOps/s $\color{#35bf28}+2.89\%$
test_step_mdp_speed[False-True-True-True-False] 45.8810μs 23.1359μs 43.2229 KOps/s 42.8096 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-True-True-False-True] 50.7510μs 25.2394μs 39.6205 KOps/s 40.0880 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[False-True-True-False-False] 35.7310μs 14.5570μs 68.6954 KOps/s 67.7773 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-True-False-True-True] 72.1810μs 39.0912μs 25.5812 KOps/s 25.1444 KOps/s $\color{#35bf28}+1.74\%$
test_step_mdp_speed[False-True-False-True-False] 43.7110μs 24.9914μs 40.0137 KOps/s 39.5521 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-True-False-False-True] 52.0410μs 26.4309μs 37.8345 KOps/s 37.5877 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[False-True-False-False-False] 36.3510μs 16.4513μs 60.7854 KOps/s 59.9329 KOps/s $\color{#35bf28}+1.42\%$
test_step_mdp_speed[False-False-True-True-True] 77.6610μs 40.8765μs 24.4639 KOps/s 24.2224 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-False-True-True-False] 56.3010μs 27.3707μs 36.5355 KOps/s 36.4503 KOps/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[False-False-True-False-True] 45.8510μs 27.3584μs 36.5519 KOps/s 37.4567 KOps/s $\color{#d91a1a}-2.42\%$
test_step_mdp_speed[False-False-True-False-False] 39.0710μs 16.5122μs 60.5614 KOps/s 59.5062 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[False-False-False-True-True] 80.4520μs 43.1260μs 23.1879 KOps/s 23.0535 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-False-True-False] 53.5120μs 28.9043μs 34.5969 KOps/s 34.2278 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[False-False-False-False-True] 55.7210μs 28.0116μs 35.6995 KOps/s 35.1757 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[False-False-False-False-False] 44.1210μs 18.4031μs 54.3386 KOps/s 54.0881 KOps/s $\color{#35bf28}+0.46\%$
test_values[generalized_advantage_estimate-True-True] 25.6071ms 25.2007ms 39.6815 Ops/s 38.6818 Ops/s $\color{#35bf28}+2.58\%$
test_values[vec_generalized_advantage_estimate-True-True] 88.5152ms 2.6756ms 373.7541 Ops/s 351.7076 Ops/s $\textbf{\color{#35bf28}+6.27\%}$
test_values[td0_return_estimate-False-False] 90.3120μs 66.6360μs 15.0069 KOps/s 14.5095 KOps/s $\color{#35bf28}+3.43\%$
test_values[td1_return_estimate-False-False] 56.3473ms 56.0016ms 17.8566 Ops/s 17.3316 Ops/s $\color{#35bf28}+3.03\%$
test_values[vec_td1_return_estimate-False-False] 1.4608ms 1.0951ms 913.1822 Ops/s 904.5162 Ops/s $\color{#35bf28}+0.96\%$
test_values[td_lambda_return_estimate-True-False] 88.8783ms 88.5632ms 11.2914 Ops/s 10.8506 Ops/s $\color{#35bf28}+4.06\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4183ms 1.0939ms 914.1412 Ops/s 908.3145 Ops/s $\color{#35bf28}+0.64\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.2071ms 25.0625ms 39.9003 Ops/s 38.8078 Ops/s $\color{#35bf28}+2.81\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9440ms 0.7252ms 1.3789 KOps/s 1.3437 KOps/s $\color{#35bf28}+2.62\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7569ms 0.6777ms 1.4755 KOps/s 1.4559 KOps/s $\color{#35bf28}+1.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5020ms 1.4741ms 678.3624 Ops/s 671.4365 Ops/s $\color{#35bf28}+1.03\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7344ms 0.6928ms 1.4434 KOps/s 1.4240 KOps/s $\color{#35bf28}+1.36\%$
test_dqn_speed 7.9200ms 1.4425ms 693.2434 Ops/s 667.7106 Ops/s $\color{#35bf28}+3.82\%$
test_ddpg_speed 3.1945ms 2.9304ms 341.2495 Ops/s 325.1661 Ops/s $\color{#35bf28}+4.95\%$
test_sac_speed 8.8596ms 8.3447ms 119.8363 Ops/s 112.7652 Ops/s $\textbf{\color{#35bf28}+6.27\%}$
test_redq_speed 0.1023s 11.7812ms 84.8807 Ops/s 91.5376 Ops/s $\textbf{\color{#d91a1a}-7.27\%}$
test_redq_deprec_speed 12.2347ms 11.6190ms 86.0660 Ops/s 85.5798 Ops/s $\color{#35bf28}+0.57\%$
test_td3_speed 8.4928ms 8.3110ms 120.3220 Ops/s 115.0213 Ops/s $\color{#35bf28}+4.61\%$
test_cql_speed 26.7360ms 25.8693ms 38.6559 Ops/s 37.9633 Ops/s $\color{#35bf28}+1.82\%$
test_a2c_speed 5.8502ms 5.6434ms 177.1987 Ops/s 175.9639 Ops/s $\color{#35bf28}+0.70\%$
test_ppo_speed 6.2167ms 5.9913ms 166.9086 Ops/s 166.6792 Ops/s $\color{#35bf28}+0.14\%$
test_reinforce_speed 4.8223ms 4.6276ms 216.0929 Ops/s 213.7327 Ops/s $\color{#35bf28}+1.10\%$
test_iql_speed 20.3350ms 19.5956ms 51.0318 Ops/s 49.5524 Ops/s $\color{#35bf28}+2.99\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.7071ms 4.5568ms 219.4504 Ops/s 214.8315 Ops/s $\color{#35bf28}+2.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7861ms 0.5398ms 1.8526 KOps/s 1.8201 KOps/s $\color{#35bf28}+1.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.2758ms 0.5208ms 1.9200 KOps/s 1.8959 KOps/s $\color{#35bf28}+1.27\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.7639ms 4.5505ms 219.7573 Ops/s 217.4428 Ops/s $\color{#35bf28}+1.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7544ms 0.5312ms 1.8827 KOps/s 1.8415 KOps/s $\color{#35bf28}+2.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.4362ms 0.5178ms 1.9311 KOps/s 1.9175 KOps/s $\color{#35bf28}+0.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.0938ms 1.9645ms 509.0263 Ops/s 491.3966 Ops/s $\color{#35bf28}+3.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.5828ms 1.8814ms 531.5227 Ops/s 517.6901 Ops/s $\color{#35bf28}+2.67\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8084ms 4.6651ms 214.3590 Ops/s 209.6549 Ops/s $\color{#35bf28}+2.24\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9237ms 0.6820ms 1.4662 KOps/s 1.4130 KOps/s $\color{#35bf28}+3.77\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8609ms 0.6654ms 1.5028 KOps/s 1.3896 KOps/s $\textbf{\color{#35bf28}+8.14\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.6696ms 4.5534ms 219.6145 Ops/s 216.3720 Ops/s $\color{#35bf28}+1.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6532ms 0.5373ms 1.8610 KOps/s 1.8216 KOps/s $\color{#35bf28}+2.16\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7398ms 0.5208ms 1.9200 KOps/s 1.8703 KOps/s $\color{#35bf28}+2.66\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.7059ms 4.4882ms 222.8049 Ops/s 216.8036 Ops/s $\color{#35bf28}+2.77\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6790ms 0.5338ms 1.8734 KOps/s 1.8336 KOps/s $\color{#35bf28}+2.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.6427ms 0.5178ms 1.9313 KOps/s 1.9092 KOps/s $\color{#35bf28}+1.16\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8117ms 4.6939ms 213.0408 Ops/s 207.6900 Ops/s $\color{#35bf28}+2.58\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8383ms 0.6822ms 1.4658 KOps/s 1.4252 KOps/s $\color{#35bf28}+2.84\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8091ms 0.6619ms 1.5109 KOps/s 1.4661 KOps/s $\color{#35bf28}+3.05\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1411s 7.5861ms 131.8208 Ops/s 137.8742 Ops/s $\color{#d91a1a}-4.39\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 22.7868ms 15.2563ms 65.5467 Ops/s 63.9334 Ops/s $\color{#35bf28}+2.52\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2147ms 1.1289ms 885.8100 Ops/s 860.7986 Ops/s $\color{#35bf28}+2.91\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1244s 9.5692ms 104.5021 Ops/s 139.2606 Ops/s $\textbf{\color{#d91a1a}-24.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.6287ms 15.2122ms 65.7366 Ops/s 55.0920 Ops/s $\textbf{\color{#35bf28}+19.32\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1534ms 1.1131ms 898.3517 Ops/s 843.8766 Ops/s $\textbf{\color{#35bf28}+6.46\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1248s 7.3750ms 135.5940 Ops/s 134.1871 Ops/s $\color{#35bf28}+1.05\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.4001ms 15.0861ms 66.2860 Ops/s 63.8982 Ops/s $\color{#35bf28}+3.74\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.2636ms 1.3935ms 717.6017 Ops/s 741.2946 Ops/s $\color{#d91a1a}-3.20\%$

@vmoens vmoens added the CI Has to do with CI setup (e.g. wheels & builds, tests...) label Jul 9, 2024
@vmoens vmoens merged commit 2555437 into main Jul 9, 2024
70 of 79 checks passed
@vmoens vmoens deleted the nightly-v3.12 branch July 9, 2024 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot