[Core] Fix wheel creation when multiple versions are installed

[romilbhardwaj]

If a user has multiple Sky versions installed (e.g., in different conda envs), sky launch would fail with:

FileNotFoundError: Could not find built SkyPilot wheels with glob pattern **/skypilot-1.0.0.dev20240823-*.whl under PosixPath('/Users/romilb/.sky/wheels')

This is because we cleanup all except the latest wheels:

skypilot/sky/backends/wheel_utils.py

Lines 51 to 57 in bca709b

	latest_wheel_dir_name = latest_wheel.parent
	# Cleanup older wheels.
	for f in WHEEL_DIR.iterdir():
	if f != latest_wheel_dir_name:
	if f.is_dir() and not f.is_symlink():
	shutil.rmtree(f, ignore_errors=True)
	return latest_wheel

This PR updates our wheel creation behavior to check if a wheel for the version exists and create it if it doesn't already exist.

Tested (run the relevant ones):

• Code formatting: bash format.sh

Manual tests:

• Create 2 conda envs, temporarily update version strs and verify launching from both envs works.

[Michaelvll]

Should we instead check WHEEL_DIR.glob(f'**/{_WHEEL_PATTERN}') instead, similar to https://github.com/skypilot-org/skypilot/blob/master/sky/backends/wheel_utils.py#L43?

Also, since we are checking the whether the current wheel exists, it might be fine to just use the current wheel for the later latest_wheel, i.e. it can become _get_current_wheel_and_remove_older_others?

[romilbhardwaj]

Ah good idea. Refactored _get_latest_wheel_and_remove_all_others into separate methods to get the latest wheel and remove others.

[Michaelvll]

Thanks for the fix @romilbhardwaj ! LGTM

[Michaelvll]

Nit: do we need any here?

[romilbhardwaj]

Yes, glob() returns a generator that we must check with any(). Checking just glob(f'**/{_WHEEL_PATTERN}') would always return true since a generator is always returned.