[CI] Fix CI #10468

Job	Run time
build-wheel-linux (3.8, cp38-cp38, --index-url https://download.pytorch.org/whl/cpu, "['cpu', '11...	1m 35s
build-wheel-windows (3.8, 3.8)	3m 21s
build-wheel-windows (3.9, 3.9)	3m 18s
build-wheel-linux (3.9, cp39-cp39, --index-url https://download.pytorch.org/whl/cpu, "['cpu', '11...	1m 40s
build-wheel-windows (3.10, 3.10.3)	3m 48s
build-wheel-windows (3.11, 3.11)	3m 19s
build-wheel-linux (3.10, cp310-cp310, --index-url https://download.pytorch.org/whl/cpu, "['cpu', ...	1m 44s
build-wheel-linux (3.11, cp311-cp311, --index-url https://download.pytorch.org/whl/cpu, "['cpu', ...	1m 37s
test-wheel (linux, ubuntu-20.04, 3.8)	3m 17s
test-wheel (linux, ubuntu-20.04, 3.9)	3m 36s
test-wheel (linux, ubuntu-20.04, 3.10)	2m 24s
test-wheel (linux, ubuntu-20.04, 3.11)	3m 26s
test-wheel-windows (3.8)	4m 9s
test-wheel-windows (3.9)	4m 8s
test-wheel-windows (3.10)	4m 49s
test-wheel-windows (3.11)	3m 44s
	49m 55s

Provide feedback