Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding ML platform reference architecture in the folder ml-platform #266

Merged
merged 39 commits into from
Mar 30, 2024
Merged

Adding ML platform reference architecture in the folder ml-platform #266

merged 39 commits into from
Mar 30, 2024

Commits on Mar 29, 2024

  1. Adding ML platform reference architecture in the folder ml-platform (#… 

    …265)

initial push to kick off collaboration
    @gushob21 @arueth
    gushob21 authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    be5f89e View commit details
    Browse the repository at this point in the history

  2. fixing project_id variable defaults (#267) 

    * Fixing project_id defaults
    @gushob21 @arueth
    gushob21 authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    9736e16 View commit details
    Browse the repository at this point in the history

  3. Fixing documentation (#268) 

    * Fixing documentation
    @gushob21 @arueth
    gushob21 authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    6bde1ae View commit details
    Browse the repository at this point in the history

  4. Formatted Terraform files

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    9ab2b07 View commit details
    Browse the repository at this point in the history

  5. Commented out the license in the README files

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    2aba065 View commit details
    Browse the repository at this point in the history

  6. Mlops platform udates (#326) 

    * Adding default single env installation and updated documentation
    @gushob21 @arueth
    gushob21 authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    2aa6017 View commit details
    Browse the repository at this point in the history

  7. MLP restructure (#347) 

    * Updated folder structure

* Added steps to for Terraform apply and destory

* Enabled image streaming

* Moved acm-templates to the correct folder

* Modified git token command to remove new lines

* Cleaned up main.tf and fixed issue with create_cluster_yamls.sh

* Cleaned up create_git_cred.sh

* Cleaned up install_kuberay_operator.sh

* Cleaned up create_namespace.sh and fixed and issue with the templates

* Cleaned up install_ray_cluster.sh

* Removed unnecessary outputs

* Bringing in changes from PR #332
    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    63f6272 View commit details
    Browse the repository at this point in the history

  8. Changed the Configuration flow

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    1b16ad0 View commit details
    Browse the repository at this point in the history

  9. Fixed terraform fmt issues

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    6f7807e View commit details
    Browse the repository at this point in the history

  10. Enabled managed prometheus

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    f6a4245 View commit details
    Browse the repository at this point in the history

  11. Enabled logging and monitoring

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    816ddf6 View commit details
    Browse the repository at this point in the history

  12. Alphabetized and standardized variables.tf

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    86d1b5f View commit details
    Browse the repository at this point in the history

  13. Renamed cluster resource from gke_batch to mlp

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    8bf2696 View commit details
    Browse the repository at this point in the history

  14. Fixed typos in README

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    bccea3e View commit details
    Browse the repository at this point in the history

  15. Fixed guide path

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    3df4a62 View commit details
    Browse the repository at this point in the history

  16. Changed README to reference a single cluster

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    d16c055 View commit details
    Browse the repository at this point in the history

  17. Formatted for Terraform files readability and supportability

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    53cf5bd View commit details
    Browse the repository at this point in the history

  18. Fixed 'set configuration variables' environment variable for project ID

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    44a23b0 View commit details
    Browse the repository at this point in the history

  19. Added serviceaccount.yaml to _namespace_template/app/kustomization.yaml

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    ca9fa68 View commit details
    Browse the repository at this point in the history

  20. Enabled serviceusage.googleapis.com APIs

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    d0861e0 View commit details
    Browse the repository at this point in the history

  21. Fixed terraform fmt issues

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    ba03eb2 View commit details
    Browse the repository at this point in the history

  22. Added shielded VMs

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    30e88b9 View commit details
    Browse the repository at this point in the history

  23. Added a dependency on the GKE cluster for the WI SA IAM binding

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    3396d4f View commit details
    Browse the repository at this point in the history

  24. add shielded config for nap pools as well

    @kenthua @arueth
    kenthua authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    f8e2d8b View commit details
    Browse the repository at this point in the history

  25. mlops platform kh (#363) 

    * remove trailing spaces

* move gpus to worker, map worker resources to node
    @kenthua @arueth
    kenthua authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    0cf408b View commit details
    Browse the repository at this point in the history

  26. Added additional setup and cleanup steps

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    a866414 View commit details
    Browse the repository at this point in the history

  27. Removed code for multiple environment and added additional service ac… 

    …counts
    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    696c94c View commit details
    Browse the repository at this point in the history

  28. Added remove_default_node_pool

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    d48b5be View commit details
    Browse the repository at this point in the history

  29. update to push work to workers

    @kenthua @arueth
    kenthua authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    9d96a92 View commit details
    Browse the repository at this point in the history

  30. adding autoscaling config

    @kenthua @arueth
    kenthua authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    a9f352f View commit details
    Browse the repository at this point in the history

  31. Update values.yaml

    @kenthua @arueth
    kenthua authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    dd7f910 View commit details
    Browse the repository at this point in the history

  32. Project cleanup (#469) 

    * Removed projects module and cleaned up unused variabled

* Added option to have a Terraform managed project
    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    8b59056 View commit details
    Browse the repository at this point in the history

  33. Code Sample for Ray Dataprocessing on GKE (#507) 

    * adding ray data processing src

* adding job details

* externalize var

* update versions

* added more preprocessing steps and

* Adding image for DataProcessing ReadMe

* Create README.md

* Update README.md

* Update README.md

* Rename DataPreprocessing.png to ray-dataprocessing-workflow.png

* Update README.md

* adding final changes

* adding final changes

* removing socket timeout since it didnt help

* Create CONVERSION.md

* Update README.md with links

---------

Co-authored-by: kenthua <[email protected]>
    @karajendran @kenthua @arueth
    2 people authored and arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    702c8da View commit details
    Browse the repository at this point in the history

  34. Restructured folder

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    4951554 View commit details
    Browse the repository at this point in the history

  35. Cleanup from folder restructuring

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    3f0df9a View commit details
    Browse the repository at this point in the history

  36. Updated the CUJs

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    cd069f0 View commit details
    Browse the repository at this point in the history

  37. Leaving default node pool for sanbox to decrease provisioning time

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    bbff3bb View commit details
    Browse the repository at this point in the history

  38. Cleaned up the data processing README

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    5984eac View commit details
    Browse the repository at this point in the history

  39. Cleaned up the initialize feature

    @arueth
    arueth committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    f421af9 View commit details
    Browse the repository at this point in the history