Constrained output and functionCall don't work together

[Seigneurhol]

Hi,

I’m currently working on developing LLM agents and in the same call, I’m using tools (functionCall) to make external API requests, while also using constrained output to format the response as JSON.

However, I’m encountering the following error:
“For controlled generation of only function calls (forced function calling), please set 'tool_config.function_calling_config.mode' field to ANY instead of populating 'response_mime_type' and 'response_schema' fields. For more details, see: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#tool-config”.

From what I understand, it seems like using tools alongside constrained output in JSON format isn't possible, though this is crucial for building agents.

Is this a bug, or is there another way to achieve this?

Thanks in advance for any help!