[Spot] Support both managed on-demand and spot instances

[MaoZiming]

From discussion with @Michaelvll

• Allow serverless style job submission for on-demand jobs.
  • Conceptually, the API contains two kinds of jobs, serverful (launch, exec, start, stop), and serverless (spot launch).
  • However, the serverless API is only spot oriented. It could be better to have on-demand support as well, and rename the spot launch to something like serverless launch.
  • This will benefit the TPU users, as the on-demand TPUs can still have quite frequent restart due to stability issue.
  • Small cloud users may benefit from this as well, for improved reliability
• Design a way to select from both on-demand and spot instances.
  • Smaller cloud have quite cheap on-demand with similar price points as spot
  • Good for optimizer to choose among both on-demand instances and spot.

Possibly replace use_spot: True with

managed: SPOT
# managed: ON_DEMAND
# managed: CONSIDER_BOTH

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: bash tests/backward_comaptibility_tests.sh