Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ability to scale on tgi custom metrics #263

Merged
merged 9 commits into from
Mar 6, 2024
Merged

Add ability to scale on tgi custom metrics #263

merged 9 commits into from
Mar 6, 2024

Commits on Feb 28, 2024

  1. Misc doc fixes 

    See changes for details. Some extra reasoning behind the changes:
* Prefix all GCS buckets with project_id since GCS buckets are globally
  namespaced.
* gcloud storage buckets add-iam-policy-binding" only accepts bucket
  URLs
    @rsgowman
    rsgowman committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    22f9f3e View commit details
    Browse the repository at this point in the history

Commits on Feb 29, 2024

  1. Add HPA configuration 

    This allows the tgi workload to scale up based on CPU demand (and
eventually other metrics).

NB: CPU is a poor choice for this workload, but acts as a baseline that
we can use to evaluate other metrics.
    @rsgowman
    rsgowman committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    e97ae42 View commit details
    Browse the repository at this point in the history

  2. Add ability to scale on tgi custom metrics

    @rsgowman
    rsgowman committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    666a0f8 View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2024

  1. review feedback + fix to startup probe

    @rsgowman
    rsgowman committed Mar 5, 2024
    Configuration menu
    Copy the full SHA
    d4f8499 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into hpa_custommetrics

    @rsgowman
    rsgowman committed Mar 5, 2024
    Configuration menu
    Copy the full SHA
    fdb506a View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/main' into hpa_custommetrics

    @rsgowman
    rsgowman committed Mar 5, 2024
    Configuration menu
    Copy the full SHA
    e45392a View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2024

  1. Merge remote-tracking branch 'origin/main' into hpa_custommetrics

    @rsgowman
    rsgowman committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    9558d1b View commit details
    Browse the repository at this point in the history

  2. tf fmt

    @rsgowman
    rsgowman committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    5c6cd2d View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/main' into hpa_custommetrics

    @rsgowman
    rsgowman committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    5769207 View commit details
    Browse the repository at this point in the history