[Feature] PPO compatibility with compile #1591

Job	Run time
generate-matrix / generate	4s
generate-matrix / generate	4s
pytorch/rl / wheel-py3_9-cpu	10m 8s
pytorch/rl / wheel-py3_9-cpu	10m 8s
pytorch/rl / wheel-py3_9-cuda11_8	9m 35s
pytorch/rl / wheel-py3_9-cuda11_8	9m 35s
pytorch/rl / wheel-py3_9-cuda12_4	11m 6s
pytorch/rl / wheel-py3_9-cuda12_4	11m 6s
pytorch/rl / wheel-py3_9-cuda12_6	9m 41s
pytorch/rl / wheel-py3_9-cuda12_6	9m 41s
pytorch/rl / upload / wheel-py3_9-cpu	15s
pytorch/rl / upload / wheel-py3_9-cpu	9s
pytorch/rl / upload / wheel-py3_9-cuda11_8	15s
pytorch/rl / upload / wheel-py3_9-cuda11_8	14s
pytorch/rl / upload / wheel-py3_9-cuda12_4	15s
pytorch/rl / upload / wheel-py3_9-cuda12_4	12s
pytorch/rl / upload / wheel-py3_9-cuda12_6	10s
pytorch/rl / upload / wheel-py3_9-cuda12_6	18s
	1h 22m 56s

Provide feedback