Skip to content

Commit

Permalink
add minicpm tool call doc, add minicpm tool call test
Browse files Browse the repository at this point in the history
Signed-off-by: pyx9913 <[email protected]>
  • Loading branch information
Cppowboy committed Nov 1, 2024
1 parent 85fce11 commit 949d654
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 1 deletion.
14 changes: 14 additions & 0 deletions docs/source/serving/openai_compatible_server.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,20 @@ AI21's Jamba-1.5 models are supported.

Flags: `--tool-call-parser jamba`

#### MiniCPM Models (`minicpm`)
Supported models:
* `openbmb/MiniCPM3-4B`

The tool calling uses Python's built-in function call format. The tool parsing module is adapted from the implementation in [ShishirPatil/gorilla](https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/bfcl/model_handler/utils.py).

Known issues:
1. Tool names must be valid Python function names, which can only contain letters (A-Z, a-z), numbers (0-9), and underscores (_). Tool names cannot start with a number.
2. Tool parameter names cannot be Python reserved words, such as `return`, `format`, `class`, etc.
3. Tool parameter descriptions must follow JSON Schema format. Complex formats like allOf, anyOf, etc. are not currently supported.

The `tool_chat_template_minicpm3.jinja` file contains the MiniCPM chat template.

Recommended flags: `--tool-call-parser minicpm --chat-template examples/tool_chat_template_minicpm3.jinja`

### How to write a tool parser plugin

Expand Down
12 changes: 11 additions & 1 deletion tests/tool_use/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,17 @@ def ensure_system_prompt(messages: List[Dict[str, Any]],
],
"supports_parallel":
False,
}
},
"minicpm3": {
"model":
"openbmb/MiniCPM3-4B",
"arguments": [
"--tool-call-parser", "minicpm", "--chat-template",
str(VLLM_PATH / "examples/tool_chat_template_minicpm3.jinja")
],
"supports_parallel":
False,
},
}

WEATHER_TOOL: ChatCompletionToolParam = {
Expand Down

0 comments on commit 949d654

Please sign in to comment.