[Versioning] v0.4.0

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1860

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (18 Unrelated Failures)

As of commit be82fa0 with merge base d930f5c ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Continuous Benchmark / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t b9bebe9d03a19258b794198c436616e0c622fbac6accf91b8658de9b6365a927 /exec failed with exit code 139
• Libs Tests on Linux / unittests-brax (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-jumanji (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-vmas (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-gpu (3.8, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-optdeps (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-stable-gpu (3.8, 11.8) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
  The process '/usr/local/bin/git' failed with exit code 128
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)
  The process '/usr/local/bin/git' failed with exit code 128
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Windows / unittests-gpu / windows-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1215s	0.1209s	8.2731 Ops/s	8.4429 Ops/s	$\color{#d91a1a}-2.01\%$
test_sync	0.1375s	0.1359s	7.3577 Ops/s	7.4426 Ops/s	$\color{#d91a1a}-1.14\%$
test_async	0.3371s	0.1347s	7.4231 Ops/s	7.3488 Ops/s	$\color{#35bf28}+1.01\%$
test_single_pixels	0.1438s	0.1423s	7.0266 Ops/s	7.6692 Ops/s	$\textbf{\color{#d91a1a}-8.38\%}$
test_sync_pixels	0.1017s	99.2439ms	10.0762 Ops/s	9.9173 Ops/s	$\color{#35bf28}+1.60\%$
test_async_pixels	0.2553s	95.0242ms	10.5236 Ops/s	10.9300 Ops/s	$\color{#d91a1a}-3.72\%$
test_simple	0.9266s	0.8595s	1.1635 Ops/s	1.1921 Ops/s	$\color{#d91a1a}-2.40\%$
test_transformed	1.1725s	1.1058s	0.9043 Ops/s	0.9132 Ops/s	$\color{#d91a1a}-0.97\%$
test_serial	2.4410s	2.3750s	0.4210 Ops/s	0.4184 Ops/s	$\color{#35bf28}+0.63\%$
test_parallel	3.3102s	3.2028s	0.3122 Ops/s	0.3136 Ops/s	$\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-True-True-True-True]	0.1300ms	33.0587μs	30.2492 KOps/s	31.0315 KOps/s	$\color{#d91a1a}-2.52\%$
test_step_mdp_speed[True-True-True-True-False]	53.8510μs	19.8681μs	50.3320 KOps/s	50.4993 KOps/s	$\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-True-True-False-True]	0.1345ms	19.1382μs	52.2514 KOps/s	53.5004 KOps/s	$\color{#d91a1a}-2.33\%$
test_step_mdp_speed[True-True-True-False-False]	41.3910μs	11.3225μs	88.3197 KOps/s	88.8713 KOps/s	$\color{#d91a1a}-0.62\%$
test_step_mdp_speed[True-True-False-True-True]	63.6110μs	35.5420μs	28.1357 KOps/s	29.5932 KOps/s	$\color{#d91a1a}-4.92\%$
test_step_mdp_speed[True-True-False-True-False]	0.1739ms	21.8679μs	45.7292 KOps/s	46.9471 KOps/s	$\color{#d91a1a}-2.59\%$
test_step_mdp_speed[True-True-False-False-True]	0.1399ms	21.3749μs	46.7839 KOps/s	49.4144 KOps/s	$\textbf{\color{#d91a1a}-5.32\%}$
test_step_mdp_speed[True-True-False-False-False]	48.3510μs	13.2618μs	75.4045 KOps/s	76.7940 KOps/s	$\color{#d91a1a}-1.81\%$
test_step_mdp_speed[True-False-True-True-True]	95.2120μs	37.8543μs	26.4171 KOps/s	27.2920 KOps/s	$\color{#d91a1a}-3.21\%$
test_step_mdp_speed[True-False-True-True-False]	57.9810μs	23.8042μs	42.0094 KOps/s	42.1831 KOps/s	$\color{#d91a1a}-0.41\%$
test_step_mdp_speed[True-False-True-False-True]	89.2710μs	20.9553μs	47.7207 KOps/s	48.2082 KOps/s	$\color{#d91a1a}-1.01\%$
test_step_mdp_speed[True-False-True-False-False]	45.3700μs	13.2641μs	75.3916 KOps/s	76.1257 KOps/s	$\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-False-False-True-True]	0.1205ms	38.9821μs	25.6528 KOps/s	26.0993 KOps/s	$\color{#d91a1a}-1.71\%$
test_step_mdp_speed[True-False-False-True-False]	49.7110μs	25.4394μs	39.3092 KOps/s	40.4722 KOps/s	$\color{#d91a1a}-2.87\%$
test_step_mdp_speed[True-False-False-False-True]	64.6610μs	22.6690μs	44.1131 KOps/s	45.6385 KOps/s	$\color{#d91a1a}-3.34\%$
test_step_mdp_speed[True-False-False-False-False]	40.0400μs	15.1533μs	65.9921 KOps/s	66.5265 KOps/s	$\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-True-True-True-True]	0.1167ms	37.0581μs	26.9847 KOps/s	27.4639 KOps/s	$\color{#d91a1a}-1.75\%$
test_step_mdp_speed[False-True-True-True-False]	0.1425ms	23.9031μs	41.8356 KOps/s	43.0233 KOps/s	$\color{#d91a1a}-2.76\%$
test_step_mdp_speed[False-True-True-False-True]	48.0810μs	24.7005μs	40.4850 KOps/s	41.3211 KOps/s	$\color{#d91a1a}-2.02\%$
test_step_mdp_speed[False-True-True-False-False]	33.3900μs	15.1530μs	65.9935 KOps/s	67.3618 KOps/s	$\color{#d91a1a}-2.03\%$
test_step_mdp_speed[False-True-False-True-True]	74.1320μs	39.1169μs	25.5644 KOps/s	26.0025 KOps/s	$\color{#d91a1a}-1.68\%$
test_step_mdp_speed[False-True-False-True-False]	54.0210μs	26.1641μs	38.2204 KOps/s	39.4893 KOps/s	$\color{#d91a1a}-3.21\%$
test_step_mdp_speed[False-True-False-False-True]	52.5810μs	26.8315μs	37.2696 KOps/s	38.3678 KOps/s	$\color{#d91a1a}-2.86\%$
test_step_mdp_speed[False-True-False-False-False]	38.3910μs	16.9599μs	58.9626 KOps/s	58.7894 KOps/s	$\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-False-True-True-True]	0.1145ms	41.2204μs	24.2599 KOps/s	25.1916 KOps/s	$\color{#d91a1a}-3.70\%$
test_step_mdp_speed[False-False-True-True-False]	0.1191ms	27.5201μs	36.3371 KOps/s	36.7638 KOps/s	$\color{#d91a1a}-1.16\%$
test_step_mdp_speed[False-False-True-False-True]	0.1879ms	26.7041μs	37.4474 KOps/s	38.8033 KOps/s	$\color{#d91a1a}-3.49\%$
test_step_mdp_speed[False-False-True-False-False]	0.1949ms	16.9128μs	59.1268 KOps/s	59.1471 KOps/s	$\color{#d91a1a}-0.03\%$
test_step_mdp_speed[False-False-False-True-True]	0.2229ms	42.8794μs	23.3212 KOps/s	24.3055 KOps/s	$\color{#d91a1a}-4.05\%$
test_step_mdp_speed[False-False-False-True-False]	0.1185ms	29.6128μs	33.7692 KOps/s	34.8632 KOps/s	$\color{#d91a1a}-3.14\%$
test_step_mdp_speed[False-False-False-False-True]	0.2296ms	28.2655μs	35.3788 KOps/s	36.4884 KOps/s	$\color{#d91a1a}-3.04\%$
test_step_mdp_speed[False-False-False-False-False]	0.2009ms	19.0395μs	52.5224 KOps/s	53.5904 KOps/s	$\color{#d91a1a}-1.99\%$
test_values[generalized_advantage_estimate-True-True]	24.3960ms	23.5597ms	42.4453 Ops/s	42.3902 Ops/s	$\color{#35bf28}+0.13\%$
test_values[vec_generalized_advantage_estimate-True-True]	85.9243ms	3.2770ms	305.1576 Ops/s	308.4929 Ops/s	$\color{#d91a1a}-1.08\%$
test_values[td0_return_estimate-False-False]	97.4320μs	60.6296μs	16.4936 KOps/s	16.8830 KOps/s	$\color{#d91a1a}-2.31\%$
test_values[td1_return_estimate-False-False]	56.1412ms	54.5978ms	18.3158 Ops/s	19.1110 Ops/s	$\color{#d91a1a}-4.16\%$
test_values[vec_td1_return_estimate-False-False]	2.0953ms	1.7528ms	570.5245 Ops/s	563.2861 Ops/s	$\color{#35bf28}+1.29\%$
test_values[td_lambda_return_estimate-True-False]	89.9828ms	87.4796ms	11.4312 Ops/s	11.8054 Ops/s	$\color{#d91a1a}-3.17\%$
test_values[vec_td_lambda_return_estimate-True-False]	4.0172ms	1.7838ms	560.5855 Ops/s	551.4443 Ops/s	$\color{#35bf28}+1.66\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	22.8140ms	22.5673ms	44.3118 Ops/s	43.1863 Ops/s	$\color{#35bf28}+2.61\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.8660ms	0.6876ms	1.4543 KOps/s	1.4621 KOps/s	$\color{#d91a1a}-0.53\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7890ms	0.6398ms	1.5630 KOps/s	1.5064 KOps/s	$\color{#35bf28}+3.76\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5814ms	1.4447ms	692.1838 Ops/s	680.9809 Ops/s	$\color{#35bf28}+1.65\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9311ms	0.6538ms	1.5296 KOps/s	1.5223 KOps/s	$\color{#35bf28}+0.48\%$
test_dqn_speed	7.4127ms	7.0290ms	142.2676 Ops/s	139.0283 Ops/s	$\color{#35bf28}+2.33\%$
test_ddpg_speed	98.3768ms	15.0626ms	66.3894 Ops/s	72.0293 Ops/s	$\textbf{\color{#d91a1a}-7.83\%}$
test_sac_speed	27.6652ms	27.1165ms	36.8779 Ops/s	36.0053 Ops/s	$\color{#35bf28}+2.42\%$
test_redq_speed	12.5592ms	11.2217ms	89.1130 Ops/s	89.2905 Ops/s	$\color{#d91a1a}-0.20\%$
test_redq_deprec_speed	22.7157ms	21.9669ms	45.5231 Ops/s	46.0153 Ops/s	$\color{#d91a1a}-1.07\%$
test_td3_speed	27.7248ms	18.0942ms	55.2663 Ops/s	54.1311 Ops/s	$\color{#35bf28}+2.10\%$
test_cql_speed	73.1445ms	71.9300ms	13.9024 Ops/s	13.5261 Ops/s	$\color{#35bf28}+2.78\%$
test_a2c_speed	24.7525ms	23.9863ms	41.6904 Ops/s	40.5877 Ops/s	$\color{#35bf28}+2.72\%$
test_ppo_speed	24.9701ms	24.2540ms	41.2304 Ops/s	40.3606 Ops/s	$\color{#35bf28}+2.15\%$
test_reinforce_speed	23.9300ms	23.1687ms	43.1616 Ops/s	42.1036 Ops/s	$\color{#35bf28}+2.51\%$
test_iql_speed	49.2317ms	48.0041ms	20.8315 Ops/s	20.3241 Ops/s	$\color{#35bf28}+2.50\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.7547ms	3.5620ms	280.7423 Ops/s	279.3963 Ops/s	$\color{#35bf28}+0.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7238ms	0.5580ms	1.7922 KOps/s	1.7750 KOps/s	$\color{#35bf28}+0.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7408ms	0.5375ms	1.8606 KOps/s	1.8606 KOps/s	$+0.00\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.9510ms	3.6240ms	275.9355 Ops/s	280.8248 Ops/s	$\color{#d91a1a}-1.74\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7700ms	0.5564ms	1.7972 KOps/s	1.7863 KOps/s	$\color{#35bf28}+0.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7488ms	0.5329ms	1.8767 KOps/s	1.8778 KOps/s	$\color{#d91a1a}-0.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.5156ms	3.3350ms	299.8516 Ops/s	309.2651 Ops/s	$\color{#d91a1a}-3.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8702ms	0.6804ms	1.4697 KOps/s	1.4473 KOps/s	$\color{#35bf28}+1.55\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8640ms	0.6530ms	1.5313 KOps/s	1.5038 KOps/s	$\color{#35bf28}+1.83\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.7624ms	3.5954ms	278.1341 Ops/s	284.4115 Ops/s	$\color{#d91a1a}-2.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7589ms	0.5533ms	1.8072 KOps/s	1.8137 KOps/s	$\color{#d91a1a}-0.36\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7580ms	0.5334ms	1.8749 KOps/s	1.8970 KOps/s	$\color{#d91a1a}-1.17\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.8880ms	3.6225ms	276.0528 Ops/s	279.5509 Ops/s	$\color{#d91a1a}-1.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7335ms	0.5509ms	1.8152 KOps/s	1.8279 KOps/s	$\color{#d91a1a}-0.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6920ms	0.5231ms	1.9117 KOps/s	1.9211 KOps/s	$\color{#d91a1a}-0.49\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.4774ms	3.2835ms	304.5521 Ops/s	309.8051 Ops/s	$\color{#d91a1a}-1.70\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9339ms	0.6830ms	1.4642 KOps/s	1.4445 KOps/s	$\color{#35bf28}+1.36\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8334ms	0.6549ms	1.5270 KOps/s	1.5088 KOps/s	$\color{#35bf28}+1.20\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1306s	10.1841ms	98.1918 Ops/s	99.1469 Ops/s	$\color{#d91a1a}-0.96\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	18.3923ms	16.2464ms	61.5520 Ops/s	63.9664 Ops/s	$\color{#d91a1a}-3.77\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	7.8091ms	2.9659ms	337.1693 Ops/s	341.4761 Ops/s	$\color{#d91a1a}-1.26\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1182s	9.8936ms	101.0756 Ops/s	82.2211 Ops/s	$\textbf{\color{#35bf28}+22.93\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.5056ms	16.2965ms	61.3627 Ops/s	63.6873 Ops/s	$\color{#d91a1a}-3.65\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	8.1612ms	3.0103ms	332.1925 Ops/s	338.1030 Ops/s	$\color{#d91a1a}-1.75\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1196s	12.3914ms	80.7010 Ops/s	98.0032 Ops/s	$\textbf{\color{#d91a1a}-17.65\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.7164ms	16.5137ms	60.5559 Ops/s	63.0497 Ops/s	$\color{#d91a1a}-3.96\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	5.3836ms	3.1616ms	316.2949 Ops/s	319.0655 Ops/s	$\color{#d91a1a}-0.87\%$