fix: Include vertex_ai_beta in vertex_ai param mapping/Do not use google auth project_id #4461
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
t968914
changed the title
|
Realized you had 2 changes here. Since we've addressed the default project issue, can you isolate the PR to
lgtm for merge |
| quota_project_id=project_id, | ||
| scopes=["https://www.googleapis.com/auth/cloud-platform"], | ||
| ) | ||
| if project_id is None: | ||
| project_id = creds_project_id |
There was a problem hiding this comment.
Hi @krrishdholakia, I saw the change in
This part is in the else block and for the application default credential case when a service account isn't being used. The google auth default method will always return None for project_id in this case, and similarly write over the previous value passed into this method.
There was a problem hiding this comment.
And thanks for taking a look at the PR
There was a problem hiding this comment.
@krrishdholakia This is currently blocking us from being able to test vertex_ai_beta in local dev
There was a problem hiding this comment.
Our models in production are also deployed in a different GCP project than our litellm proxy, so we are unable to use vertex_ai_beta in production either
There was a problem hiding this comment.
noted - working on this today
There was a problem hiding this comment.
thanks for the review @andrewmjc @t968914
Title
Vertex_ai_beta chat completion does not work with provided project_id in model info because it is always overwritten by
project_idreturned fromgoogle.auth.default(). However,google.auth.default()returns None in cases where there is no service account. This will also not work when litellm is hosted in a seperate google project from vertex ai models. The project_id passed in as a param should take precedent to support this.Relevant issues
Type
🐛 Bug Fix
Changes