Fixes for Gemini inside Vertex #11511
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tbeamish-benchsci
force-pushed
the
fixes-for-gemini-in-vertex
branch
from
a6f9b27
to
563e4a6
Compare
llama-index-integrations/llms/llama-index-llms-vertex/llama_index/llms/vertex/base.py
Outdated
Show resolved
Hide resolved
tbeamish-benchsci
force-pushed
the
fixes-for-gemini-in-vertex
branch
from
e425f90
to
3d2bc58
Compare
llama-index-integrations/llms/llama-index-llms-gemini/llama_index/llms/gemini/utils.py
Outdated
Show resolved
Hide resolved
tbeamish-benchsci
force-pushed
the
fixes-for-gemini-in-vertex
branch
2 times, most recently
from
295872c
to
2fd9b0e
Compare
tbeamish-benchsci
force-pushed
the
fixes-for-gemini-in-vertex
branch
from
2fd9b0e
to
7ee8e4d
Compare
dosubot
bot
added
the
size:L
logan-markewich
approved these changes
Mar 4, 2024
k8si
added a commit
to Mozilla-Ocho/llama_index
that referenced
this pull request
Mar 5, 2024
* main: (61 commits) bump anthropic versions (run-llama#11654) Fixes for Gemini inside Vertex (run-llama#11511) Update multimodal anthropic docs (run-llama#11643) fix anthropic tokenizer (run-llama#11636) extend claude 3 multimodal cookbook (run-llama#11639) fix pydantic (run-llama#11631) Added Nest Asyncio to Opensearch Client Init to Support ASGI applicat… (run-llama#11622) update readmes (run-llama#11627) typo in chat refine prompt (run-llama#11628) Add Anthropic Multi Modal (run-llama#11623) Add Vertex AI Text Embedding and Multimodal Embedding (run-llama#11561) Update anthropic models (run-llama#11612) Bumped Opensearch Integration Version (run-llama#11618) Solve issue of duplicate Settings classes (run-llama#11606) fix composable retrieval (run-llama#11617) Updated AzureCosmosDBMongoDBVectorSearch to Pydantic Vector Store base class (run-llama#11613) Updated model information for Perplexity.ai (run-llama#11603) Solve issue in custom_agent document , RetryAgentWorker._run_step() got an unexpected keyword argument 'input' (run-llama#11611) Fix empty metadata error for CSV Reader (run-llama#11563) Feature: Improve batch embedding generation throughput for Cohere in Bedrock (run-llama#11572) ...
bdonkey
added a commit
to bdonkey/gpt_index
that referenced
this pull request
Mar 11, 2024
* main: bump anthropic versions (run-llama#11654) Fixes for Gemini inside Vertex (run-llama#11511) Update multimodal anthropic docs (run-llama#11643) fix anthropic tokenizer (run-llama#11636) extend claude 3 multimodal cookbook (run-llama#11639) fix pydantic (run-llama#11631) Added Nest Asyncio to Opensearch Client Init to Support ASGI applicat… (run-llama#11622) update readmes (run-llama#11627) typo in chat refine prompt (run-llama#11628) Add Anthropic Multi Modal (run-llama#11623) Add Vertex AI Text Embedding and Multimodal Embedding (run-llama#11561) Update anthropic models (run-llama#11612) Bumped Opensearch Integration Version (run-llama#11618) Solve issue of duplicate Settings classes (run-llama#11606) fix composable retrieval (run-llama#11617) Updated AzureCosmosDBMongoDBVectorSearch to Pydantic Vector Store base class (run-llama#11613) Updated model information for Perplexity.ai (run-llama#11603) Solve issue in custom_agent document , RetryAgentWorker._run_step() got an unexpected keyword argument 'input' (run-llama#11611) Fix empty metadata error for CSV Reader (run-llama#11563)
Izukimat
pushed a commit
to Izukimat/llama_index
that referenced
this pull request
Mar 29, 2024
This was referenced Apr 5, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Gemini requires an even message count and, in the Gemini LLM, we
merge_neighboring_same_role_messages. This PR does the same thing in the Vertex LLM if we're using Gemini.We also set the default
system_rolein theLLMMetadatato beSystem.USERonly if we're using Gemini, since Gemini only supports "user" and "model" roles.Finally we ensure the role of each message in Vertex is set to the appropriate
MessageRoleNOTE: This change adds/updates NO UNIT TESTs to support it.
Fixes # (#11439)
Type of Change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Suggested Checklist: