Skip to content

[Platform] Add support for Google vertex AI #297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

junaidbinfarooq
Copy link
Contributor

@junaidbinfarooq junaidbinfarooq commented Aug 10, 2025

Q A
Bug fix? no
New feature? yes
Docs? yes
Issues #16 #272
License MIT

Changes proposed:

  • The PR aims to add support for Google's Vertex AI
  • Support is added for text generation, embeddings, server tools, etc
  • I created a separate directory inside the Bridge directory for the said bridge. Moving forward, we could either create a new namespace named Google and add two namespaces inside it, one each for Gemini and VertexAi, or we could simply keep VertexAi and remove the other, as this api already supports Gemini and more. Either way, Normalize Gemini vs Google #146 would be addressed.
  • I exempted the use of api key from the implementation so far as VertexAi supports two auth mechanisms, Api keys and ADC, ie, Application Default Credentials. The former is recommended for testing while the latter is recommended for production, IIRC. However, it does need to install gCloud on the server and locally too (if verifying the behavior without an api key). The Google auth library is also needed for automatic credential discovery.
  • More information can be found at https://cloud.google.com/vertex-ai/generative-ai/docs

@junaidbinfarooq junaidbinfarooq changed the title feat(platform): Add support for Google vertex AI [Platform] Add support for Google vertex AI Aug 11, 2025
@junaidbinfarooq junaidbinfarooq marked this pull request as ready for review August 11, 2025 14:37
@carsonbot carsonbot added Feature New feature Platform Issues & PRs about the AI Platform component Status: Needs Review labels Aug 11, 2025
@junaidbinfarooq junaidbinfarooq force-pushed the feat/platform-support-for-vertex-ai branch 2 times, most recently from 14ae923 to 35d9a8b Compare August 11, 2025 21:53
@junaidbinfarooq junaidbinfarooq force-pushed the feat/platform-support-for-vertex-ai branch from 35d9a8b to 9124195 Compare August 12, 2025 07:52
Copy link

@faizanakram99 faizanakram99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, ship it

@junaidbinfarooq
Copy link
Contributor Author

junaidbinfarooq commented Aug 12, 2025

lgtm, ship it

Docs and AI bundle integration need to be added first.

@junaidbinfarooq junaidbinfarooq force-pushed the feat/platform-support-for-vertex-ai branch 2 times, most recently from 1346440 to b08d7e7 Compare August 13, 2025 05:16
* }
* }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indention looks wrong

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the PHP doc, the IDE doesn't apparently help with the indentation in this case.
I tried to correct the indentation now.

@chr-hertel
Copy link
Member

Thanks @junaidbinfarooq for your work on this - I think we're getting somewhere.
Please see my comments and review once again yourself with those in mind - not sure I catched everything.

In general it great to have that bridge, I still wonder tho about the amount of duplicated code with the Gemini bridge and we would need to support API Keys as well ... but both topics are fine as follow up

@junaidbinfarooq
Copy link
Contributor Author

junaidbinfarooq commented Aug 13, 2025

Thanks @junaidbinfarooq for your work on this - I think we're getting somewhere. Please see my comments and review once again yourself with those in mind - not sure I catched everything.

In general it great to have that bridge, I still wonder tho about the amount of duplicated code with the Gemini bridge and we would need to support API Keys as well ... but both topics are fine as follow up

@chr-hertel
You're welcome 😄
The duplication is expected as I did not touch the existing Gemini bridge code.
This is what I also pointed out in the PR description about possibly having Vertex AI only as we move forward.
If this is done, there would be no duplication anywhere as far as Google bridges are concerned.
And yes, I do believe this is better done in a follow-up PR after a proper discussion on the topic.

@junaidbinfarooq junaidbinfarooq force-pushed the feat/platform-support-for-vertex-ai branch from b08d7e7 to 605bf5e Compare August 13, 2025 09:34
@junaidbinfarooq junaidbinfarooq force-pushed the feat/platform-support-for-vertex-ai branch from 605bf5e to f986f83 Compare August 13, 2025 21:49
@chr-hertel
Copy link
Member

@junaidbinfarooq are all examples that you added working for you?

i need to apply those changes, any clue why?
image

@junaidbinfarooq
Copy link
Contributor Author

junaidbinfarooq commented Aug 15, 2025

@junaidbinfarooq are all examples that you added working for you?

i need to apply those changes, any clue why? image

@chr-hertel
I just rechecked the vertex ai api and also ran the examples again to verify they are still working as expected. And it is apparent that they are working:
image
Besides, I didn't use versions in model IDs as that is not recommended, IIRC.

- Adds initial support to integrate text generation using vertex AI
- Adds tests to verify the behavior
- Adds examples to interact with vertex ai
- Adds documentation for vertex ai
- Adds support to calculate the token usage
@junaidbinfarooq junaidbinfarooq force-pushed the feat/platform-support-for-vertex-ai branch from 8a402e2 to 153ee1e Compare August 15, 2025 20:51
@junaidbinfarooq junaidbinfarooq force-pushed the feat/platform-support-for-vertex-ai branch from 153ee1e to ce696af Compare August 15, 2025 20:54
@chr-hertel
Copy link
Member

Hmm, super weird 🤔

And where is that $location-aiplatform.googleapi.com/... coming from?
Docs say it's just aiplatform as subdomain:
image
See https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/gemini-2.5-flash-lite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature New feature Platform Issues & PRs about the AI Platform component Status: Needs Review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants