[Feature] PPO compatibility with compile #1676

Job	Run time
generate-matrix / generate	4s
generate-matrix / generate	4s
pytorch/rl / wheel-py3_9-cpu	8m 19s
pytorch/rl / wheel-py3_9-cpu	8m 19s
pytorch/rl / wheel-py3_9-cuda11_8	11m 27s
pytorch/rl / wheel-py3_9-cuda11_8	11m 27s
pytorch/rl / wheel-py3_9-cuda12_4	9m 52s
pytorch/rl / wheel-py3_9-cuda12_4	9m 52s
pytorch/rl / wheel-py3_9-cuda12_6	11m 6s
pytorch/rl / wheel-py3_9-cuda12_6	11m 6s
pytorch/rl / upload / wheel-py3_9-cpu	11s
pytorch/rl / upload / wheel-py3_9-cpu	9s
pytorch/rl / upload / wheel-py3_9-cuda11_8	11s
pytorch/rl / upload / wheel-py3_9-cuda11_8	14s
pytorch/rl / upload / wheel-py3_9-cuda12_4	15s
pytorch/rl / upload / wheel-py3_9-cuda12_4	14s
pytorch/rl / upload / wheel-py3_9-cuda12_6	10s
pytorch/rl / upload / wheel-py3_9-cuda12_6	14s
	1h 23m 14s

Provide feedback