Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Frontend][Feature] support tool calling for internlm/internlm2_5-7b-chat model #8405

Merged
merged 25 commits into from
Oct 4, 2024
Merged
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
87b6352
[add] add tools call for internlm2
sydnash Sep 12, 2024
5355659
Merge branch 'main' into add-internlm2-for-tool-use
sydnash Sep 12, 2024
68cd89d
[add] add some comments
sydnash Sep 12, 2024
d17f006
[add] add some comments
sydnash Sep 12, 2024
2d7d9d4
[fix] fix internlm2 tool chat template, fix the internlm2 tool call o…
sydnash Sep 13, 2024
12352e7
[add] add tool parser plugin doc
sydnash Sep 13, 2024
11bed0d
[add] add tool parser plugin doc
sydnash Sep 13, 2024
8a8b840
[fix] fix the stream tool call for internlm2
sydnash Sep 13, 2024
00c5da2
[fix] comment
sydnash Sep 13, 2024
882c764
[merge] resolve conflict
sydnash Sep 13, 2024
12b1035
[fix] use metavar to display the help info for --tool-call-parser, ad…
sydnash Sep 14, 2024
ed5b3fd
[add] got valid tool parsers from ToolParserManager
sydnash Sep 14, 2024
ea2c089
[fix] fix build for docs
sydnash Sep 14, 2024
36ad5d0
[fix] internlm's tool call out may arguments or parameters
sydnash Sep 15, 2024
cf981c0
[merge] resolve conflict
sydnash Sep 18, 2024
647db0d
refactor the tool parser to internlm, fix the test case of streamed_args
sydnash Sep 26, 2024
064ca1f
merge main
sydnash Sep 27, 2024
106909c
[fix] fix internlm parallel test, remove vllm/version.py
sydnash Sep 28, 2024
e242501
[format]
sydnash Sep 29, 2024
0a5ddf4
[format]
sydnash Sep 29, 2024
1db530d
[fix] fix the mistral tool call error. recover vllm/version.py and de…
sydnash Sep 29, 2024
dc94a22
[fix] change vocab property to get_vocab method in mistral_tool_parse…
sydnash Sep 29, 2024
3048233
Merge remote-tracking branch 'origin/main' into add-internlm2-for-too…
sydnash Sep 29, 2024
a2f938f
[fix] remove --tokenizer-mode mistral for mistral test. fix the syste…
sydnash Oct 3, 2024
4b619a2
[merge] merge from main
sydnash Oct 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[add] add tool parser plugin doc
sydnash committed Sep 13, 2024
commit 11bed0d8499bb3cc8e6a45a2be748e57ec46bacb
10 changes: 5 additions & 5 deletions docs/source/serving/openai_compatible_server.md
Original file line number Diff line number Diff line change
@@ -259,10 +259,10 @@ class ExampleToolParser(ToolParser):

```

and then you can the flags to specify the plugins:
Then you can use this plugin in the command line like this.
```
--enable-auto-tool-choice \
--tool-parser-plugin <absolute path of the plugin file>
--tool-call-parser example \
--chat-template <your chat template> \
--enable-auto-tool-choice \
--tool-parser-plugin <absolute path of the plugin file>
--tool-call-parser example \
--chat-template <your chat template> \
```