Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't cache just the system prompt CachedContent must have at least one content #471

Open
DreamGenX opened this issue Nov 25, 2024 · 3 comments
Labels
api: aiplatform Issues related to the googleapis/nodejs-vertexai API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@DreamGenX
Copy link

When you try to create cache with just system prompt:

    await vertex.preview.cachedContents.create({
      displayName: key,
      model,
      contents: []; // either [] or undefined, does not change the result
      systemInstruction: systemInstructions,
      ttl: `${ttlSeconds}s`
    });

You will get:

[VertexAI.ClientError]: got status: 400 Bad Request. {"error":{"code":400,"message":"CachedContent must have at least one content.","status":"INVALID_ARGUMENT"}}
@DreamGenX DreamGenX added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Nov 25, 2024
@product-auto-label product-auto-label bot added the api: aiplatform Issues related to the googleapis/nodejs-vertexai API. label Nov 25, 2024
@cabljac
Copy link
Contributor

cabljac commented Nov 27, 2024

Could be related?

#469

@DreamGenX
Copy link
Author

@cabljac Don't think so, also happens with Python SDK.

@salopge
Copy link

salopge commented Dec 30, 2024

Are there any further updates?

I'm experiencing the same issue.
I'm using a long system prompt, and I only want to set the system instructions when creating the cache, but it seems impossible with the current structure.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: aiplatform Issues related to the googleapis/nodejs-vertexai API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

3 participants