[benchmark] add option to enable CompiledAutograd #5218

Job	Run time
pytester (ubuntu-22.04, 3.10, latest, core)	12m 10s
pytester (ubuntu-22.04, 3.10, latest, ops)	29m 59s
pytester (ubuntu-22.04, 3.10, nightly, core)	13m 10s
pytester (ubuntu-22.04, 3.10, nightly, ops)	29m 52s
pytester (macOS-14, 3.10, latest, core)	9m 53s
pytester (macOS-14, 3.10, latest, ops)	25m 29s
pytester (macOS-14, 3.10, nightly, core)	9m 33s
pytester (macOS-14, 3.10, nightly, ops)	19m 46s
pytester (windows-2022, 3.10, latest, core)	24m 54s
pytester (windows-2022, 3.10, nightly, core)	26m 2s
pytester (ubuntu-22.04, 3.11, latest)	24m 4s
pytester (ubuntu-22.04, 3.12, latest)	28m 30s
testing-guardian	1s
	4h 13m 23s

Provide feedback