Add decorator for running functions as SkyPilot tasks

[colinjc]

Experimental decorator for wrapping functions and submitting them as SkyPilot tasks.

UX issues:

• Cloudpickle requires exact matching Python versions to work correctly, but the docker image already includes a Python binary. How can we make matching these as simple as possible?
• No return value support yet. Possible by pickling and writing out the return value to a folder and syncing it back, but would require some extra work.

name: hello-spot

resources:
  accelerators: L4:1
  use_spot: true
  cloud: kubernetes

workdir: .

setup: |
  pip install uv
  uv pip install -r ./requirements.txt
  echo "Running setup."

run: |
  echo "Hello, SkyPilot!"

from sky.decorators import sky_task

capture_me = "captured"

@sky_task(task="hello_spot.yaml", cluster_name="colin-fixed-test")
def test_remote(a, /, b="c"):
    print(f"hello {a} and {b}. capture: {capture_me}")

test_remote(1)

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

[github-actions]

This PR is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[github-actions]

This PR was closed because it has been stalled for 10 days with no activity.