Skip to content

Commit

Permalink
feat: Update Notebooks for Google Gen AI SDK
Browse files Browse the repository at this point in the history
  • Loading branch information
@holtskinner
holtskinner committed Jan 6, 2025
1 parent 24c4bfb commit c9aeac8
Show file tree
Hide file tree
Showing 7 changed files with 2,551 additions and 478 deletions.
133 changes: 74 additions & 59 deletions gemini/context-caching/intro_context_caching.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -108,16 +108,19 @@
"\n",
"The Gemini API provides the context caching feature for developers to store frequently used input tokens in a dedicated cache and reference them for subsequent requests, eliminating the need to repeatedly pass the same set of tokens to a model. This feature can help reduce the number of tokens sent to the model, thereby lowering the cost of requests that contain repeat content with high input token counts.\n",
"\n",
"For more information, refer to the [official documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-overview).\n",
"\n",
"### Objectives\n",
"\n",
"In this tutorial, you learn how to use the Gemini API context caching feature in Vertex AI.\n",
"\n",
"You will complete the following tasks:\n",
"\n",
"- Create a context cache\n",
"- Retrieve and use a context cache\n",
"- Use context caching in Chat\n",
"- Update the expire time of a context cache\n",
"- Delete a context cache\n"
"- Delete a context cache"
]
},
{
Expand All @@ -135,7 +138,7 @@
"id": "No17Cw5hgx12"
},
"source": [
"### Install Vertex AI SDK and other required packages\n"
"### Install Google Gen AI SDK for Python"
]
},
{
Expand All @@ -146,7 +149,7 @@
},
"outputs": [],
"source": [
"%pip install --upgrade --user --quiet google-cloud-aiplatform"
"%pip install --upgrade --user --quiet google-genai"
]
},
{
Expand Down Expand Up @@ -220,7 +223,7 @@
"id": "DF4l8DTdWgPY"
},
"source": [
"### Set Google Cloud project information and initialize Vertex AI SDK\n",
"### Set Google Cloud project information and create client\n",
"\n",
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
"\n",
Expand All @@ -235,12 +238,19 @@
},
"outputs": [],
"source": [
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
"import os\n",
"\n",
"from google import genai\n",
"\n",
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
"LOCATION = \"us-central1\" # @param {type:\"string\"}\n",
"\n",
"import vertexai\n",
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
"\n",
"LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
"\n",
"vertexai.init(project=PROJECT_ID, location=LOCATION)"
"client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)"
]
},
{
Expand Down Expand Up @@ -269,12 +279,13 @@
},
"outputs": [],
"source": [
"import datetime\n",
"\n",
"import vertexai\n",
"from vertexai.generative_models import Part\n",
"from vertexai.preview import caching\n",
"from vertexai.preview.generative_models import GenerativeModel"
"from IPython.display import Markdown, display\n",
"from google.genai.types import (\n",
" Content,\n",
" CreateCachedContentConfig,\n",
" GenerateContentConfig,\n",
" Part,\n",
")"
]
},
{
Expand All @@ -285,7 +296,7 @@
"source": [
"### Create a context cache\n",
"\n",
"**Note**: Context caching is only available for stable models with fixed versions (for example, `gemini-1.5-pro-001`). You must include the version postfix (for example, the `-001` in `gemini-1.5-pro-001`).\n",
"**Note**: Context caching is only available for stable models with fixed versions (for example, `gemini-1.5-pro-002`). You must include the version postfix (for example, the `-002` in `gemini-1.5-pro-002`).\n",
"\n",
"For more information, see [Available Gemini stable model versions](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versioning#stable-versions-available).\n"
]
Expand All @@ -298,7 +309,7 @@
},
"outputs": [],
"source": [
"MODEL_ID = \"gemini-1.5-pro-001\" # @param {type:\"string\"}"
"MODEL_ID = \"gemini-1.5-pro-002\" # @param {type:\"string\"}"
]
},
{
Expand Down Expand Up @@ -332,7 +343,7 @@
"Now look at the research paper below, and answer the following questions in 1-2 sentences.\n",
"\"\"\"\n",
"\n",
"contents = [\n",
"pdf_parts = [\n",
" Part.from_uri(\n",
" \"gs://cloud-samples-data/generative-ai/pdf/2312.11805v3.pdf\",\n",
" mime_type=\"application/pdf\",\n",
Expand All @@ -343,11 +354,18 @@
" ),\n",
"]\n",
"\n",
"cached_content = caching.CachedContent.create(\n",
" model_name=MODEL_ID,\n",
" system_instruction=system_instruction,\n",
" contents=contents,\n",
" ttl=datetime.timedelta(minutes=60),\n",
"cached_content = client.caches.create(\n",
" model=MODEL_ID,\n",
" contents=[\n",
" Content(\n",
" role=\"user\",\n",
" parts=pdf_parts,\n",
" )\n",
" ],\n",
" config=CreateCachedContentConfig(\n",
" system_instruction=system_instruction,\n",
" ttl=\"3600s\",\n",
" ),\n",
")"
]
},
Expand All @@ -371,10 +389,10 @@
"outputs": [],
"source": [
"print(cached_content.name)\n",
"print(cached_content.resource_name)\n",
"print(cached_content.model_name)\n",
"print(cached_content.model)\n",
"print(cached_content.create_time)\n",
"print(cached_content.expire_time)"
"print(cached_content.expire_time)\n",
"print(cached_content.usage_metadata)"
]
},
{
Expand All @@ -383,56 +401,59 @@
"id": "d-f5gTEaCPkN"
},
"source": [
"### Retrieve and use a context cache\n",
"### Retrieve a context cache\n",
"\n",
"You can use the property `name` or `resource_name` to reference the contents of the context cache. For example:\n",
"```\n",
"new_cached_content = caching.CachedContent(cached_content_name=cached_content.name)\n",
"```"
"You can use the property `name` to reference the contents of the context cache. For example:"
]
},
{
"cell_type": "markdown",
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RQ1zMmFQ1BNj"
"id": "68e807d418e8"
},
"outputs": [],
"source": [
"To use the context cache, you construct a `GenerativeModel` with the context cache."
"new_cached_content = client.caches.get(name=cached_content.name)"
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"metadata": {
"id": "EPVyJIW1BaVj"
"id": "6f0c98e451f8"
},
"outputs": [],
"source": [
"model = GenerativeModel.from_cached_content(cached_content=cached_content)"
"### Use a context cache"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1kgfyCoGH_w0"
"id": "RQ1zMmFQ1BNj"
},
"source": [
"To use the context cache, you provide the `cached_content` resource name in the `config` parameter of the `generate_content()` method.\n",
"\n",
"Then you can query the model with a prompt, and the cached content will be used as a prefix to the prompt."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5dSDogewDAHB"
"id": "EPVyJIW1BaVj"
},
"outputs": [],
"source": [
"response = model.generate_content(\n",
" \"What is the research goal shared by these research papers?\"\n",
"response = client.models.generate_content(\n",
" model=MODEL_ID,\n",
" contents=\"What is the research goal shared by these research papers?\",\n",
" config=GenerateContentConfig(\n",
" cached_content=cached_content.name,\n",
" ),\n",
")\n",
"\n",
"print(response.text)"
"display(Markdown(response.text))"
]
},
{
Expand All @@ -454,7 +475,7 @@
},
"outputs": [],
"source": [
"chat = model.start_chat()"
"chat = client.chats.create(model=MODEL_ID)"
]
},
{
Expand All @@ -471,7 +492,7 @@
"\n",
"response = chat.send_message(prompt)\n",
"\n",
"print(response.text)"
"display(Markdown(response.text))"
]
},
{
Expand All @@ -492,15 +513,6 @@
"print(response.text)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "h2VhjUmojQjg"
},
"source": [
"You can use `print(chat.history)` to print out the chat session history."
]
},
{
"cell_type": "markdown",
"metadata": {
Expand All @@ -509,10 +521,9 @@
"source": [
"### Update the expiration time of a context cache\n",
"\n",
"\n",
"The default expiration time of a context cache is 60 minutes. To update the expiration time, update one of the following properties:\n",
"\n",
"`ttl` - The number of seconds and nanoseconds that the cache lives after it's created or after the `ttl` is updated before it expires. When you set the `ttl`, the cache `expire_time` is updated.\n",
"`ttl` - The number of seconds that the cache lives after it's created or after the `ttl` is updated before it expires.\n",
"\n",
"`expire_time` - A Timestamp that specifies the absolute date and time when the context cache expires."
]
Expand All @@ -525,9 +536,13 @@
},
"outputs": [],
"source": [
"cached_content.update(ttl=datetime.timedelta(hours=1))\n",
"\n",
"cached_content.refresh()\n",
"cached_content = client.caches.update(\n",
" name=cached_content.name,\n",
" config=CreateCachedContentConfig(\n",
" system_instruction=system_instruction,\n",
" ttl=\"7200s\",\n",
" ),\n",
")\n",
"\n",
"print(cached_content.expire_time)"
]
Expand All @@ -551,7 +566,7 @@
},
"outputs": [],
"source": [
"cached_content.delete()"
"client.caches.delete(name=cached_content.name)"
]
}
],
Expand Down
Loading

0 comments on commit c9aeac8

Please sign in to comment.