Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Storage] Make s3 bucket creation log visible to the users #3730

Merged

Conversation

landscapepainter
Copy link
Collaborator

@landscapepainter landscapepainter commented Jul 6, 2024

This resolves #3729 with some refactoring.

Tested (run the relevant ones):

$ sky launch test.yaml --cloud gcp -y
Task from YAML spec: test.yaml
I 07-06 21:59:14 storage.py:1380] Created S3 bucket no-log-s3-creation in us-east-1
I 07-06 21:59:18 storage.py:1824] Created GCS bucket log-gcs-creation in US with storage class STANDARD
I 07-06 21:59:22 optimizer.py:695] == Optimizer ==
I 07-06 21:59:22 optimizer.py:706] Target: minimizing cost
I 07-06 21:59:22 optimizer.py:718] Estimated cost: $0.4 / hour
I 07-06 21:59:22 optimizer.py:718]
I 07-06 21:59:22 optimizer.py:843] Considered resources (1 node):
I 07-06 21:59:22 optimizer.py:913] ----------------------------------------------------------------------------------------------
I 07-06 21:59:22 optimizer.py:913]  CLOUD   INSTANCE        vCPUs   Mem(GB)   ACCELERATORS   REGION/ZONE     COST ($)   CHOSEN
I 07-06 21:59:22 optimizer.py:913] ----------------------------------------------------------------------------------------------
I 07-06 21:59:22 optimizer.py:913]  GCP     n2-standard-8   8       32        -              us-central1-a   0.39          ✔
I 07-06 21:59:22 optimizer.py:913] ----------------------------------------------------------------------------------------------
I 07-06 21:59:22 optimizer.py:913]
Running task on cluster sky-xxxx-gcpuser...
I 07-06 21:59:22 cloud_vm_ray_backend.py:4411] Creating a new cluster: 'sky-xxxx-gcpuser' [1x GCP(n2-standard-8)].
I 07-06 21:59:22 cloud_vm_ray_backend.py:4411] Tip: to reuse an existing cluster, specify --cluster (-c). Run `sky status` to see existing clusters.
I 07-06 21:59:25 cloud_vm_ray_backend.py:1406] To view detailed progress: tail -n100 -f /home/gcpuser/sky_logs/sky-2024-07-06-21-59-20-661208/provision.log
I 07-06 21:59:27 provisioner.py:76] Launching on GCP us-central1 (us-central1-a)
I 07-06 22:00:27 provisioner.py:458] Successfully provisioned or found existing instance.
I 07-06 22:01:14 provisioner.py:560] Successfully provisioned cluster: sky-xxxx-gcpuser

Copy link
Collaborator

@cblmemo cblmemo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! LGTM.

sky/data/storage.py Outdated Show resolved Hide resolved
@landscapepainter landscapepainter merged commit 6acaa75 into skypilot-org:master Jul 7, 2024
20 checks passed
Michaelvll pushed a commit that referenced this pull request Aug 23, 2024
* refactor and make log visible

* nit

* Update sky/data/storage.py

Co-authored-by: Tian Xia <[email protected]>

---------

Co-authored-by: Tian Xia <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Storage] creating s3 bucket fails to display log to user
2 participants