Skip to content

Commit

Permalink
Change granite chat template to keep json list formatting for tool ca…
Browse files Browse the repository at this point in the history
…lls (vllm-project#10452)

Signed-off-by: Max de Bayser <[email protected]>
  • Loading branch information
maxdebayser authored Nov 20, 2024
1 parent b00b33d commit a324d3a
Showing 1 changed file with 1 addition and 5 deletions.
6 changes: 1 addition & 5 deletions examples/tool_chat_template_granite.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,7 @@
{{- '<|start_of_role|>user<|end_of_role|>' + message['content'] + '<|end_of_text|>
' }}
{%- elif message['role'] == 'assistant_tool_call' or (message['role'] == 'assistant' and message.tool_calls is defined) %}
{{- '<|start_of_role|>assistant<|end_of_role|>' }}
{% for tc in message.tool_calls %}
{{- '<|tool_call|> ' + {'name': tc.function.name, 'arguments': tc.function.arguments}|tojson }}
{% endfor %}
{{- '<|end_of_text|>
{{- '<|start_of_role|>assistant<|end_of_role|><|tool_call|>' + message.tool_calls|map(attribute='function')|list|tojson(indent=4) + '<|end_of_text|>
' }}
{%- elif message['role'] == 'assistant' %}
{{- '<|start_of_role|>assistant<|end_of_role|>' + message['content'] + '<|end_of_text|>
Expand Down

0 comments on commit a324d3a

Please sign in to comment.