[Feature] PPO compatibility with compile #1671

Job	Run time
generate-matrix / generate	6s
generate-matrix / generate	6s
pytorch/rl / wheel-py3_9-cpu	4m 28s
pytorch/rl / wheel-py3_9-cpu	4m 28s
pytorch/rl / wheel-py3_9-cuda11_8	14m 25s
pytorch/rl / wheel-py3_9-cuda11_8	14m 25s
pytorch/rl / wheel-py3_9-cuda12_4	10m 31s
pytorch/rl / wheel-py3_9-cuda12_4	10m 31s
pytorch/rl / wheel-py3_9-cuda12_6	8m 35s
pytorch/rl / wheel-py3_9-cuda12_6	8m 35s
pytorch/rl / upload / wheel-py3_9-cpu	9s
pytorch/rl / upload / wheel-py3_9-cpu	10s
pytorch/rl / upload / wheel-py3_9-cuda11_8	11s
pytorch/rl / upload / wheel-py3_9-cuda11_8	10s
pytorch/rl / upload / wheel-py3_9-cuda12_4	12s
pytorch/rl / upload / wheel-py3_9-cuda12_4	19s
pytorch/rl / upload / wheel-py3_9-cuda12_6	9s
pytorch/rl / upload / wheel-py3_9-cuda12_6	10s
	1h 17m 40s

Provide feedback