Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] pin-mem all rb outputs #2411

Merged
merged 1 commit into from
Sep 2, 2024
Merged

[BugFix] pin-mem all rb outputs #2411

merged 1 commit into from
Sep 2, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Sep 2, 2024 •
edited
Loading

Stack from ghstack (oldest at bottom):

@vmoens
Update
d509f86 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Sep 2, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2411

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 14 Unrelated Failures

As of commit d509f86 with merge base e82a69f (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Sep 2, 2024
@vmoens
[BugFix] pin-mem all rb outputs
8b1a1a9 
ghstack-source-id: 1706fec493c3a2467c0a8ad19e41cca0c8e71a8f
Pull Request resolved: #2411
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 2, 2024
@vmoens
Copy link
Contributor Author

vmoens commented Sep 2, 2024

closes #2404

@vmoens vmoens added the bug Something isn't working label Sep 2, 2024
@vmoens vmoens merged commit fc5836f into gh/vmoens/23/base Sep 2, 2024
40 of 55 checks passed
@vmoens vmoens deleted the gh/vmoens/23/head branch September 2, 2024 15:06
@vmoens vmoens linked an issue Sep 2, 2024 that may be closed by this pull request
[BUG] TensorDictReplayBuffer's pin_memory=True does not work with custom transforms #2404
Closed
3 tasks
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 2, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 58.3466ms 57.9305ms 17.2621 Ops/s 17.2619 Ops/s $+0.00\%$
test_sync 32.9578ms 31.6348ms 31.6107 Ops/s 30.5206 Ops/s $\color{#35bf28}+3.57\%$
test_async 72.1445ms 30.7789ms 32.4898 Ops/s 31.6600 Ops/s $\color{#35bf28}+2.62\%$
test_simple 0.4876s 0.4106s 2.4355 Ops/s 2.3993 Ops/s $\color{#35bf28}+1.51\%$
test_transformed 0.6219s 0.5596s 1.7869 Ops/s 1.8077 Ops/s $\color{#d91a1a}-1.15\%$
test_serial 1.3071s 1.2412s 0.8056 Ops/s 0.7934 Ops/s $\color{#35bf28}+1.54\%$
test_parallel 1.1850s 1.1129s 0.8985 Ops/s 0.8930 Ops/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-True-True-True-True] 0.1633ms 26.6357μs 37.5437 KOps/s 36.9417 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-True-True-False] 42.5300μs 15.8863μs 62.9472 KOps/s 63.1341 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-True-True-False-True] 54.4120μs 15.1355μs 66.0697 KOps/s 65.5066 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[True-True-True-False-False] 41.2380μs 8.9395μs 111.8630 KOps/s 112.9099 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-True-False-True-True] 0.1113ms 28.4390μs 35.1629 KOps/s 34.7253 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[True-True-False-True-False] 65.8030μs 17.4204μs 57.4039 KOps/s 57.4037 KOps/s $+0.00\%$
test_step_mdp_speed[True-True-False-False-True] 50.5150μs 16.8360μs 59.3966 KOps/s 59.1465 KOps/s $\color{#35bf28}+0.42\%$
test_step_mdp_speed[True-True-False-False-False] 61.6290μs 10.6531μs 93.8694 KOps/s 95.3153 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-False-True-True-True] 0.1131ms 30.6984μs 32.5750 KOps/s 32.7144 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-False-True-True-False] 57.7590μs 19.4751μs 51.3476 KOps/s 52.0479 KOps/s $\color{#d91a1a}-1.35\%$
test_step_mdp_speed[True-False-True-False-True] 61.7720μs 16.9136μs 59.1239 KOps/s 59.1152 KOps/s $\color{#35bf28}+0.01\%$
test_step_mdp_speed[True-False-True-False-False] 35.1560μs 10.7571μs 92.9617 KOps/s 94.8286 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[True-False-False-True-True] 91.3020μs 32.1309μs 31.1227 KOps/s 31.3376 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[True-False-False-True-False] 69.4410μs 21.2122μs 47.1426 KOps/s 47.8226 KOps/s $\color{#d91a1a}-1.42\%$
test_step_mdp_speed[True-False-False-False-True] 51.9480μs 18.6107μs 53.7326 KOps/s 54.2231 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-False-False-False-False] 49.3830μs 12.2378μs 81.7143 KOps/s 82.5136 KOps/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[False-True-True-True-True] 67.7970μs 30.6919μs 32.5819 KOps/s 32.5457 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[False-True-True-True-False] 54.5230μs 19.3884μs 51.5772 KOps/s 51.6745 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[False-True-True-False-True] 53.9510μs 19.5955μs 51.0322 KOps/s 50.7340 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-True-True-False-False] 71.0630μs 11.9730μs 83.5214 KOps/s 83.5223 KOps/s $-0.00\%$
test_step_mdp_speed[False-True-False-True-True] 84.1070μs 32.3525μs 30.9095 KOps/s 31.0797 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-True-False-True-False] 73.1900μs 20.8450μs 47.9732 KOps/s 48.3358 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[False-True-False-False-True] 65.0380μs 20.8358μs 47.9942 KOps/s 47.3191 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[False-True-False-False-False] 45.4850μs 13.4923μs 74.1164 KOps/s 74.6103 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-False-True-True-True] 3.2650ms 33.4127μs 29.9287 KOps/s 29.4944 KOps/s $\color{#35bf28}+1.47\%$
test_step_mdp_speed[False-False-True-True-False] 56.4450μs 22.3565μs 44.7297 KOps/s 44.5808 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[False-False-True-False-True] 68.3080μs 21.1878μs 47.1969 KOps/s 47.1374 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-False-True-False-False] 64.0440μs 13.4484μs 74.3583 KOps/s 74.0572 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-False-False-True-True] 81.7200μs 34.8886μs 28.6626 KOps/s 28.1103 KOps/s $\color{#35bf28}+1.96\%$
test_step_mdp_speed[False-False-False-True-False] 60.4340μs 23.6637μs 42.2588 KOps/s 41.7829 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[False-False-False-False-True] 62.6980μs 22.2141μs 45.0164 KOps/s 44.3272 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[False-False-False-False-False] 42.0790μs 14.8524μs 67.3293 KOps/s 67.1849 KOps/s $\color{#35bf28}+0.21\%$
test_values[generalized_advantage_estimate-True-True] 10.4470ms 9.3900ms 106.4966 Ops/s 107.0088 Ops/s $\color{#d91a1a}-0.48\%$
test_values[vec_generalized_advantage_estimate-True-True] 38.6965ms 36.3779ms 27.4892 Ops/s 27.8258 Ops/s $\color{#d91a1a}-1.21\%$
test_values[td0_return_estimate-False-False] 0.2392ms 0.1676ms 5.9683 KOps/s 5.8922 KOps/s $\color{#35bf28}+1.29\%$
test_values[td1_return_estimate-False-False] 26.5849ms 23.3179ms 42.8854 Ops/s 41.8576 Ops/s $\color{#35bf28}+2.46\%$
test_values[vec_td1_return_estimate-False-False] 45.1483ms 37.0800ms 26.9687 Ops/s 27.9032 Ops/s $\color{#d91a1a}-3.35\%$
test_values[td_lambda_return_estimate-True-False] 37.0293ms 33.5958ms 29.7656 Ops/s 29.6543 Ops/s $\color{#35bf28}+0.38\%$
test_values[vec_td_lambda_return_estimate-True-False] 37.4806ms 36.4579ms 27.4289 Ops/s 27.7648 Ops/s $\color{#d91a1a}-1.21\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.4677ms 8.2222ms 121.6221 Ops/s 121.3889 Ops/s $\color{#35bf28}+0.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5928ms 2.0164ms 495.9346 Ops/s 473.5739 Ops/s $\color{#35bf28}+4.72\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5114ms 0.3652ms 2.7382 KOps/s 2.6906 KOps/s $\color{#35bf28}+1.77\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.1735ms 46.4443ms 21.5312 Ops/s 21.0680 Ops/s $\color{#35bf28}+2.20\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.0404ms 3.0295ms 330.0879 Ops/s 326.3936 Ops/s $\color{#35bf28}+1.13\%$
test_dqn_speed 6.3166ms 1.2977ms 770.5890 Ops/s 762.7094 Ops/s $\color{#35bf28}+1.03\%$
test_ddpg_speed 3.7192ms 2.7070ms 369.4189 Ops/s 365.6985 Ops/s $\color{#35bf28}+1.02\%$
test_sac_speed 8.2904ms 7.9375ms 125.9837 Ops/s 123.8957 Ops/s $\color{#35bf28}+1.69\%$
test_redq_speed 14.4014ms 12.7506ms 78.4280 Ops/s 78.3069 Ops/s $\color{#35bf28}+0.15\%$
test_redq_deprec_speed 13.8823ms 12.5923ms 79.4138 Ops/s 79.0827 Ops/s $\color{#35bf28}+0.42\%$
test_td3_speed 8.3246ms 8.0366ms 124.4308 Ops/s 122.3413 Ops/s $\color{#35bf28}+1.71\%$
test_cql_speed 40.3929ms 35.8333ms 27.9070 Ops/s 28.3617 Ops/s $\color{#d91a1a}-1.60\%$
test_a2c_speed 9.0725ms 7.3126ms 136.7498 Ops/s 137.0468 Ops/s $\color{#d91a1a}-0.22\%$
test_ppo_speed 9.6533ms 7.6980ms 129.9034 Ops/s 132.8491 Ops/s $\color{#d91a1a}-2.22\%$
test_reinforce_speed 7.5967ms 6.6340ms 150.7375 Ops/s 150.1804 Ops/s $\color{#35bf28}+0.37\%$
test_iql_speed 33.6331ms 32.2801ms 30.9789 Ops/s 30.3998 Ops/s $\color{#35bf28}+1.90\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 9.3511ms 5.0572ms 197.7361 Ops/s 198.2582 Ops/s $\color{#d91a1a}-0.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8566ms 0.4785ms 2.0899 KOps/s 2.0913 KOps/s $\color{#d91a1a}-0.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6423ms 0.4580ms 2.1833 KOps/s 2.2201 KOps/s $\color{#d91a1a}-1.66\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.2438ms 4.9212ms 203.2010 Ops/s 208.6248 Ops/s $\color{#d91a1a}-2.60\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7824ms 0.4791ms 2.0872 KOps/s 2.1193 KOps/s $\color{#d91a1a}-1.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7178ms 0.4719ms 2.1192 KOps/s 2.2433 KOps/s $\textbf{\color{#d91a1a}-5.53\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8517ms 1.6847ms 593.5652 Ops/s 594.2404 Ops/s $\color{#d91a1a}-0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1468ms 1.6132ms 619.8726 Ops/s 626.4922 Ops/s $\color{#d91a1a}-1.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.8854ms 5.2010ms 192.2706 Ops/s 203.9519 Ops/s $\textbf{\color{#d91a1a}-5.73\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.0670ms 0.6316ms 1.5832 KOps/s 1.6531 KOps/s $\color{#d91a1a}-4.23\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8300ms 0.5916ms 1.6903 KOps/s 1.7292 KOps/s $\color{#d91a1a}-2.25\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.3364ms 4.8633ms 205.6204 Ops/s 208.4052 Ops/s $\color{#d91a1a}-1.34\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4160ms 0.4787ms 2.0891 KOps/s 2.1179 KOps/s $\color{#d91a1a}-1.36\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6312ms 0.4631ms 2.1594 KOps/s 2.2453 KOps/s $\color{#d91a1a}-3.82\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8661ms 5.0038ms 199.8466 Ops/s 213.2604 Ops/s $\textbf{\color{#d91a1a}-6.29\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7578ms 0.4953ms 2.0190 KOps/s 2.1142 KOps/s $\color{#d91a1a}-4.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6499ms 0.4652ms 2.1497 KOps/s 2.2535 KOps/s $\color{#d91a1a}-4.61\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.4841ms 5.1377ms 194.6397 Ops/s 205.5081 Ops/s $\textbf{\color{#d91a1a}-5.29\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1327s 0.8402ms 1.1902 KOps/s 1.6452 KOps/s $\textbf{\color{#d91a1a}-27.65\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7975ms 0.6024ms 1.6600 KOps/s 1.7375 KOps/s $\color{#d91a1a}-4.46\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1283s 6.4118ms 155.9629 Ops/s 159.1836 Ops/s $\color{#d91a1a}-2.02\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.6253ms 13.2531ms 75.4542 Ops/s 76.2875 Ops/s $\color{#d91a1a}-1.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.8381ms 1.2492ms 800.4863 Ops/s 744.9546 Ops/s $\textbf{\color{#35bf28}+7.45\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1333s 6.5194ms 153.3890 Ops/s 165.3246 Ops/s $\textbf{\color{#d91a1a}-7.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.5578ms 14.2298ms 70.2751 Ops/s 64.6328 Ops/s $\textbf{\color{#35bf28}+8.73\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1931ms 1.3387ms 746.9675 Ops/s 720.7546 Ops/s $\color{#35bf28}+3.64\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1376s 9.5022ms 105.2388 Ops/s 160.2720 Ops/s $\textbf{\color{#d91a1a}-34.34\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.5281ms 13.6136ms 73.4562 Ops/s 74.9640 Ops/s $\color{#d91a1a}-2.01\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.2010ms 1.4874ms 672.3219 Ops/s 620.5017 Ops/s $\textbf{\color{#35bf28}+8.35\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 2, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1017s 0.1014s 9.8601 Ops/s 9.7322 Ops/s $\color{#35bf28}+1.31\%$
test_sync 97.5671ms 89.2701ms 11.2020 Ops/s 11.0939 Ops/s $\color{#35bf28}+0.97\%$
test_async 0.1526s 84.9627ms 11.7699 Ops/s 11.9261 Ops/s $\color{#d91a1a}-1.31\%$
test_single_pixels 0.1099s 0.1078s 9.2790 Ops/s 9.1807 Ops/s $\color{#35bf28}+1.07\%$
test_sync_pixels 72.4251ms 71.2309ms 14.0389 Ops/s 14.0691 Ops/s $\color{#d91a1a}-0.21\%$
test_async_pixels 0.1343s 67.9692ms 14.7125 Ops/s 14.8636 Ops/s $\color{#d91a1a}-1.02\%$
test_simple 0.8343s 0.7551s 1.3244 Ops/s 1.3258 Ops/s $\color{#d91a1a}-0.11\%$
test_transformed 1.0382s 0.9685s 1.0325 Ops/s 1.0421 Ops/s $\color{#d91a1a}-0.92\%$
test_serial 2.1445s 2.0777s 0.4813 Ops/s 0.4847 Ops/s $\color{#d91a1a}-0.69\%$
test_parallel 1.9011s 1.8576s 0.5383 Ops/s 0.5366 Ops/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-True-True-True-True] 0.2306ms 35.9397μs 27.8244 KOps/s 27.5076 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[True-True-True-True-False] 50.6610μs 20.7338μs 48.2305 KOps/s 47.3918 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[True-True-True-False-True] 45.7910μs 20.3397μs 49.1650 KOps/s 49.1451 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-True-True-False-False] 40.6310μs 11.8375μs 84.4771 KOps/s 84.2067 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-True-False-True-True] 0.4664ms 38.3065μs 26.1053 KOps/s 25.7804 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[True-True-False-True-False] 48.6210μs 22.7791μs 43.9000 KOps/s 43.5103 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[True-True-False-False-True] 51.1110μs 22.4195μs 44.6040 KOps/s 44.2693 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[True-True-False-False-False] 38.5000μs 13.8937μs 71.9751 KOps/s 71.6401 KOps/s $\color{#35bf28}+0.47\%$
test_step_mdp_speed[True-False-True-True-True] 73.9910μs 40.5433μs 24.6650 KOps/s 24.7277 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-False-True-True-False] 0.1347ms 25.1380μs 39.7804 KOps/s 39.7150 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-False-True-False-True] 0.2109ms 22.6603μs 44.1301 KOps/s 44.6218 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[True-False-True-False-False] 43.7410μs 13.9785μs 71.5382 KOps/s 72.7694 KOps/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[True-False-False-True-True] 72.6910μs 42.0998μs 23.7531 KOps/s 23.4587 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-False-False-True-False] 67.8310μs 26.8742μs 37.2104 KOps/s 36.7776 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-False-False-False-True] 62.5710μs 24.1581μs 41.3940 KOps/s 40.6623 KOps/s $\color{#35bf28}+1.80\%$
test_step_mdp_speed[True-False-False-False-False] 47.6610μs 15.7996μs 63.2928 KOps/s 62.8636 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[False-True-True-True-True] 69.4710μs 40.5190μs 24.6798 KOps/s 24.4941 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-True-True-True-False] 65.0710μs 25.2242μs 39.6445 KOps/s 39.6051 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-True-True-False-True] 54.5210μs 25.8977μs 38.6135 KOps/s 37.6745 KOps/s $\color{#35bf28}+2.49\%$
test_step_mdp_speed[False-True-True-False-False] 43.2610μs 15.6331μs 63.9667 KOps/s 63.9125 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-True-False-True-True] 73.7410μs 42.6560μs 23.4434 KOps/s 23.3357 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-True-False-True-False] 78.2120μs 27.3173μs 36.6069 KOps/s 36.7567 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[False-True-False-False-True] 60.5710μs 27.8189μs 35.9468 KOps/s 35.0904 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-True-False-False-False] 48.7710μs 17.6611μs 56.6216 KOps/s 54.9093 KOps/s $\color{#35bf28}+3.12\%$
test_step_mdp_speed[False-False-True-True-True] 4.2734ms 44.9511μs 22.2464 KOps/s 22.2250 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-False-True-True-False] 62.6010μs 29.1926μs 34.2552 KOps/s 34.1122 KOps/s $\color{#35bf28}+0.42\%$
test_step_mdp_speed[False-False-True-False-True] 54.9610μs 28.1728μs 35.4952 KOps/s 35.2313 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-False-True-False-False] 48.3210μs 17.6252μs 56.7369 KOps/s 56.0666 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-False-False-True-True] 79.7410μs 46.0794μs 21.7017 KOps/s 21.4963 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-False-False-True-False] 59.1710μs 31.1134μs 32.1405 KOps/s 32.3187 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-False-False-False-True] 60.9510μs 29.6048μs 33.7784 KOps/s 33.4906 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-False-False-False-False] 45.2710μs 19.4346μs 51.4546 KOps/s 51.7889 KOps/s $\color{#d91a1a}-0.65\%$
test_values[generalized_advantage_estimate-True-True] 24.4918ms 23.9362ms 41.7778 Ops/s 41.5906 Ops/s $\color{#35bf28}+0.45\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1068s 3.0221ms 330.8934 Ops/s 329.0958 Ops/s $\color{#35bf28}+0.55\%$
test_values[td0_return_estimate-False-False] 93.9120μs 65.3255μs 15.3080 KOps/s 15.4754 KOps/s $\color{#d91a1a}-1.08\%$
test_values[td1_return_estimate-False-False] 54.2658ms 53.9646ms 18.5307 Ops/s 17.6693 Ops/s $\color{#35bf28}+4.88\%$
test_values[vec_td1_return_estimate-False-False] 1.3275ms 1.0692ms 935.2780 Ops/s 928.8982 Ops/s $\color{#35bf28}+0.69\%$
test_values[td_lambda_return_estimate-True-False] 85.6359ms 85.2304ms 11.7329 Ops/s 11.3989 Ops/s $\color{#35bf28}+2.93\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3970ms 1.0611ms 942.3858 Ops/s 941.3308 Ops/s $\color{#35bf28}+0.11\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.1877ms 23.9385ms 41.7738 Ops/s 41.7394 Ops/s $\color{#35bf28}+0.08\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9273ms 0.7094ms 1.4097 KOps/s 1.3972 KOps/s $\color{#35bf28}+0.90\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7700ms 0.6532ms 1.5310 KOps/s 1.5338 KOps/s $\color{#d91a1a}-0.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4869ms 1.4527ms 688.3613 Ops/s 688.8052 Ops/s $\color{#d91a1a}-0.06\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8163ms 0.6681ms 1.4967 KOps/s 1.4760 KOps/s $\color{#35bf28}+1.41\%$
test_dqn_speed 1.6247ms 1.2647ms 790.6736 Ops/s 762.0374 Ops/s $\color{#35bf28}+3.76\%$
test_ddpg_speed 2.9487ms 2.6003ms 384.5747 Ops/s 379.5417 Ops/s $\color{#35bf28}+1.33\%$
test_sac_speed 7.7733ms 7.5374ms 132.6723 Ops/s 131.2651 Ops/s $\color{#35bf28}+1.07\%$
test_redq_speed 13.2118ms 10.1557ms 98.4672 Ops/s 98.1139 Ops/s $\color{#35bf28}+0.36\%$
test_redq_deprec_speed 10.8505ms 10.5899ms 94.4298 Ops/s 93.6567 Ops/s $\color{#35bf28}+0.83\%$
test_td3_speed 7.6198ms 7.5524ms 132.4085 Ops/s 129.9707 Ops/s $\color{#35bf28}+1.88\%$
test_cql_speed 27.1227ms 24.4432ms 40.9112 Ops/s 40.2477 Ops/s $\color{#35bf28}+1.65\%$
test_a2c_speed 5.6486ms 5.4347ms 184.0022 Ops/s 185.3680 Ops/s $\color{#d91a1a}-0.74\%$
test_ppo_speed 5.9503ms 5.7533ms 173.8142 Ops/s 176.6415 Ops/s $\color{#d91a1a}-1.60\%$
test_reinforce_speed 5.4566ms 4.5252ms 220.9824 Ops/s 227.2081 Ops/s $\color{#d91a1a}-2.74\%$
test_iql_speed 20.0343ms 19.1560ms 52.2029 Ops/s 52.7181 Ops/s $\color{#d91a1a}-0.98\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.4828ms 6.3270ms 158.0520 Ops/s 158.1665 Ops/s $\color{#d91a1a}-0.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5784ms 0.2365ms 4.2291 KOps/s 3.1326 KOps/s $\textbf{\color{#35bf28}+35.00\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.3669ms 0.2135ms 4.6844 KOps/s 3.3109 KOps/s $\textbf{\color{#35bf28}+41.49\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.5805ms 6.2713ms 159.4575 Ops/s 159.4273 Ops/s $\color{#35bf28}+0.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.4797ms 0.2347ms 4.2607 KOps/s 4.2509 KOps/s $\color{#35bf28}+0.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.3635ms 0.2129ms 4.6976 KOps/s 4.7577 KOps/s $\color{#d91a1a}-1.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5373ms 1.3468ms 742.5073 Ops/s 739.2289 Ops/s $\color{#35bf28}+0.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4223ms 1.2549ms 796.8958 Ops/s 696.3002 Ops/s $\textbf{\color{#35bf28}+14.45\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.5560ms 6.4072ms 156.0749 Ops/s 156.6722 Ops/s $\color{#d91a1a}-0.38\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.6608ms 0.4372ms 2.2874 KOps/s 2.2281 KOps/s $\color{#35bf28}+2.66\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.5341ms 0.4035ms 2.4785 KOps/s 2.3055 KOps/s $\textbf{\color{#35bf28}+7.50\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 9.8297ms 6.4882ms 154.1263 Ops/s 157.7598 Ops/s $\color{#d91a1a}-2.30\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5629ms 0.2434ms 4.1088 KOps/s 4.1678 KOps/s $\color{#d91a1a}-1.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.3685ms 0.2121ms 4.7158 KOps/s 4.6257 KOps/s $\color{#35bf28}+1.95\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.5683ms 6.2320ms 160.4627 Ops/s 159.0515 Ops/s $\color{#35bf28}+0.89\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8416ms 0.3390ms 2.9494 KOps/s 2.8139 KOps/s $\color{#35bf28}+4.82\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5578ms 0.3159ms 3.1658 KOps/s 2.9920 KOps/s $\textbf{\color{#35bf28}+5.81\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.6306ms 6.4630ms 154.7272 Ops/s 155.6853 Ops/s $\color{#d91a1a}-0.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7209ms 0.4841ms 2.0658 KOps/s 2.0109 KOps/s $\color{#35bf28}+2.73\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6729ms 0.4629ms 2.1601 KOps/s 2.1600 KOps/s $+0.00\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1678s 8.3598ms 119.6203 Ops/s 120.8338 Ops/s $\color{#d91a1a}-1.00\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 21.0792ms 16.0080ms 62.4688 Ops/s 54.1760 Ops/s $\textbf{\color{#35bf28}+15.31\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.0337ms 1.1729ms 852.5667 Ops/s 969.4413 Ops/s $\textbf{\color{#d91a1a}-12.06\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1253s 9.8910ms 101.1024 Ops/s 134.1793 Ops/s $\textbf{\color{#d91a1a}-24.65\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 21.0691ms 16.1441ms 61.9422 Ops/s 61.9474 Ops/s $-0.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.2935ms 1.0659ms 938.1489 Ops/s 927.2381 Ops/s $\color{#35bf28}+1.18\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1246s 7.6944ms 129.9652 Ops/s 130.0530 Ops/s $\color{#d91a1a}-0.07\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 21.3013ms 16.2621ms 61.4925 Ops/s 61.9227 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 9.0804ms 1.3454ms 743.2927 Ops/s 923.1199 Ops/s $\textbf{\color{#d91a1a}-19.48\%}$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] TensorDictReplayBuffer's pin_memory=True does not work with custom transforms
2 participants
@vmoens @facebook-github-bot