Skip to content

Commit

Permalink
feat(granite*): Add granite chat template
Browse files Browse the repository at this point in the history
Branch: GraniteThreeSupport

This is a port of the work done in llama.cpp with a slight tweak for the
tool call response:
ggerganov/llama.cpp#10013

Signed-off-by: Gabe Goodhart <[email protected]>
  • Loading branch information
gabe-l-hart committed Nov 5, 2024
1 parent 6a13182 commit bbe64fe
Showing 1 changed file with 13 additions and 0 deletions.
13 changes: 13 additions & 0 deletions llama.cpp/llama.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -19291,6 +19291,19 @@ static int32_t llama_chat_apply_template_internal(
if (add_ass) {
ss << "[|assistant|]";
}
} else if (tmpl == "granite" || tmpl == "granitemoe" || tmpl_contains("<|start_of_role|>")) {
// IBM Granite template
for (const auto & message : chat) {
std::string role(message->role);
ss << "<|start_of_role|>" << role << "<|end_of_role|>";
if (role == "assistant_tool_call") {
ss << "<|tool_call|>";
}
ss << message->content << "<|end_of_text|>\n";
}
if (add_ass) {
ss << "<|start_of_role|>assistant<|end_of_role|>\n";
}
} else {
// template not supported
return -1;
Expand Down

0 comments on commit bbe64fe

Please sign in to comment.