Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GCP DWS] Fix None issue when no provision timeout is provided #3835

Merged
merged 3 commits into from
Aug 16, 2024

Conversation

Michaelvll
Copy link
Collaborator

@Michaelvll Michaelvll commented Aug 16, 2024

It is possible that the provision_timeout is set to None in our cluster yaml, which will cause a wrong CLI to wait for MIG.

To reproduce:

experimental:
  config_overrides:
    gcp:
      managed_instance_group:
        # Setup the DWS config
        run_duration: 36000
        # Failover to other regions after 30 minutes.
        provision_timeout: 1800
resources:
  accelerators: H100:8
  cpus: 32+

Tested (run the relevant ones):

  • Code formatting: bash format.sh
  • Any manual or new tests for this PR (please specify below)
  • All smoke tests: pytest tests/test_smoke.py
  • Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
  • Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

@Michaelvll Michaelvll enabled auto-merge August 16, 2024 07:22
@Michaelvll Michaelvll added this pull request to the merge queue Aug 16, 2024
Merged via the queue into master with commit 69838f5 Aug 16, 2024
20 checks passed
@Michaelvll Michaelvll deleted the mig-provision-timeout branch August 16, 2024 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants