diff --git a/docs/source/serving/openai_compatible_server.md b/docs/source/serving/openai_compatible_server.md index 413c87ab28755..114a1c5634757 100644 --- a/docs/source/serving/openai_compatible_server.md +++ b/docs/source/serving/openai_compatible_server.md @@ -254,6 +254,20 @@ AI21's Jamba-1.5 models are supported. Flags: `--tool-call-parser jamba` +#### MiniCPM Models (`minicpm`) +Supported models: +* `openbmb/MiniCPM3-4B` + +The tool calling uses Python's built-in function call format. The tool parsing module is adapted from the implementation in [ShishirPatil/gorilla](https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/bfcl/model_handler/utils.py). + +Known issues: +1. Tool names must be valid Python function names, which can only contain letters (A-Z, a-z), numbers (0-9), and underscores (_). Tool names cannot start with a number. +2. Tool parameter names cannot be Python reserved words, such as `return`, `format`, `class`, etc. +3. Tool parameter descriptions must follow JSON Schema format. Complex formats like allOf, anyOf, etc. are not currently supported. + +The `tool_chat_template_minicpm3.jinja` file contains the MiniCPM chat template. + +Recommended flags: `--tool-call-parser minicpm --chat-template examples/tool_chat_template_minicpm3.jinja` ### How to write a tool parser plugin diff --git a/tests/tool_use/utils.py b/tests/tool_use/utils.py index ce36515a2381c..7b012aa1a19c6 100644 --- a/tests/tool_use/utils.py +++ b/tests/tool_use/utils.py @@ -99,7 +99,17 @@ def ensure_system_prompt(messages: List[Dict[str, Any]], ], "supports_parallel": False, - } + }, + "minicpm3": { + "model": + "openbmb/MiniCPM3-4B", + "arguments": [ + "--tool-call-parser", "minicpm", "--chat-template", + str(VLLM_PATH / "examples/tool_chat_template_minicpm3.jinja") + ], + "supports_parallel": + False, + }, } WEATHER_TOOL: ChatCompletionToolParam = {