Feature Request: Support tools and tool_choice parameter in OpenAI compatible service

[ChanceFlow]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

background

I use docker to deploy xinference and deployed qwen2.5-instruct-llama.cpp on it. And I use openai-swarm to get a response from xinference, but it was a pydantic raise, it looks like a compatibility issue for the latest openai client.

the tool_choice field is often used in openai-swarm. But in this case, I didn't pass any function_call or tool_choice to the llm, I just can't use swarm to complete simple chat.

 File "/usr/local/lib/python3.10/dist-packages/xinference/model/llm/llama_cpp/core.py", line 282, in chat
    generate_config = self._sanitize_generate_config(generate_config)
  File "/usr/local/lib/python3.10/dist-packages/xinference/model/llm/llama_cpp/core.py", line 262, in _sanitize_generate_config
    generate_config = super()._sanitize_generate_config(generate_config)
  File "/usr/local/lib/python3.10/dist-packages/xinference/model/llm/llama_cpp/core.py", line 99, in _sanitize_generate_config
    **CreateCompletionLlamaCpp(**generate_config).dict()
  File "/usr/local/lib/python3.10/dist-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: [address=0.0.0.0:34499, pid=1794] 1 validation error for CreateCompletion
tool_choice
  extra fields not permitted (type=value_error.extra)

Motivation

Swarm has now going popular and own 10k+ stars and it is easy to use, I think it's important for you to deal with it.

Possible Implementation

No response

[ChanceFlow]

same issue for vllm: vllm-project/vllm#1869

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.