[k8s] GPU Feature discovery label formatter #3493
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Michaelvll
reviewed
May 10, 2024
There was a problem hiding this comment.
This is awesome @asaiacai! It looks very reasonable to me. @romilbhardwaj for another look to make sure it does not break our other formatters : )
Co-authored-by: Zhanghao Wu <[email protected]>
There was a problem hiding this comment.
Thanks @asaiacai. Tested on A100 and H100 from Lambda. Left a comment about documenting that this labelformatter cannot be used with autoscaling, otherwise lgtm!
|
just added the docstring @romilbhardwaj . Thanks for the review! lmk if i this needs anything else. |
romilbhardwaj
approved these changes
Jun 5, 2024
Michaelvll
added a commit
that referenced
this pull request
Aug 23, 2024
* GFDLabel formatter for k8s * update comment * format * substring match against k8s labels instead of strict matching * cleanup * use k8s label * map k8s label value to accelerator instead of accelerator to label value * remove unused get_gke_accelerator_name * remove get acc from value func * pattern match against A100' * pattern match against A100' * format * fix typo * format * re.search * compare strings * add P4000 * format * lower case for check Co-authored-by: Zhanghao Wu <[email protected]> * force upper case * match skypilot labeler logic * format.sh * add docstring * fix class docstring * grammar fix * format --------- Co-authored-by: Zhanghao Wu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #2460
This allows k8s to consume the node label
nvidia.com/gpu.productcreated by GPU feature discovery which is commonly deployed through the NVIDIA GPU operatorTested (run the relevant ones):
bash format.sheks_test_cluster.yamlk3swithgpu-operatorusingdeploy_k3s.shmodified to exclude the skypilot k8s labeler, ensure the following can run