You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The chat format consists in multi-turn conversations between the user and the large language model. Each turn is associated with a role, either system, user or assistant. A chat history can thus be represented as an iterable of tuples:
history= [
("system", "This is a system message"),
("user", "A user asking a question to the model"),
("assistant", "A model responding to the user")
]
We need a function which, given a model, will return the corresponding prompt in the correct format. For instance, for Zephyr the corresponding prompt should be:
<|system|>
This is a system message</s>
<|user|>
A user asking a question to the model</s>
<|assistant|>
A model responding to the user.
The simplest way to go about this is to define a function which, given a model name and a sequence of interactions returns the corresponding prompt.
frompromptsimportto_chat_formathistory= [
("system", "This is a system message"),
("user", "A user asking a question to the model"),
]
prompt=to_chat_format("HuggingFaceH4/zephyr-7b-beta", history)
print(prompt)
# <|system|># This is a system message</s> # <|user|># A user asking a question to the model</s>
However, when we want to pass the prompt to the model for generation we need to append the <|assistant|> token. For this we can define a new function, to_chat_prompt:
frompromptsimportto_chat_prompthistory= [
("system", "This is a system message"),
("user", "A user asking a question to the model"),
]
prompt=to_chat_prompt("HuggingFaceH4/zephyr-7b-beta", history)
print(prompt)
# <|system|># This is a system message</s> # <|user|># A user asking a question to the model</s> # <|assistant|>
This system is still cumbersome, since we need to append </s> to the response, and need to call to_chat_format with the model name before every call to the model. Could we simplify this?
classChat:
history: List[Tuple[str, str]]
def__init__(self, model_name: str, system: str=None, tools=None, documents=None):
raiseNotImplementedErrordef__str__(self):
"""Render the prompt corresponding to the history. If we want to be able to use the prompt with any library this should return the prompt with the "assistant" keyword? As with `Template` we need tobe able to specify the model we are working with. """raiseNotImplementedErrordef__getitem__(self, i):
returnself.history[i]
defuser(self, string: str):
"""Add user input to history"""self.history.append(("user", string))
defassistant(self, string: str):
"""Add user input to history"""self.history.append(("user", string))
So a session would look like:
fromoutlinesimportgenerate, modelsmodel=models.transformers("HuggingFaceH4/zephyr-7b-beta")
session=Chat("HuggingFaceH4/zephyr-7b-beta", "This is a system message")
session.user("A user asking a question to the model")
result=generate.text(model)(session)
session.assistant(result)
The obvious downside of this design is the necessity to specify the model name twice.
str(chat) must return the correctly formatted templates to be able to interface with any library
This issue (and arguably the repository) was triggered by this issue in Outlines and inspired by this repository opened by a member of the Outlines community.
The text was updated successfully, but these errors were encountered:
The chat format consists in multi-turn conversations between the user and the large language model. Each turn is associated with a role, either
system
,user
orassistant
. A chat history can thus be represented as an iterable of tuples:We need a function which, given a model, will return the corresponding prompt in the correct format. For instance, for Zephyr the corresponding prompt should be:
The simplest way to go about this is to define a function which, given a model name and a sequence of interactions returns the corresponding prompt.
However, when we want to pass the prompt to the model for generation we need to append the
<|assistant|>
token. For this we can define a new function,to_chat_prompt
:This system is still cumbersome, since we need to append
</s>
to the response, and need to callto_chat_format
with the model name before every call to the model. Could we simplify this?So a session would look like:
The obvious downside of this design is the necessity to specify the model name twice.
str(chat)
must return the correctly formatted templates to be able to interface with any libraryThis issue (and arguably the repository) was triggered by this issue in Outlines and inspired by this repository opened by a member of the Outlines community.
The text was updated successfully, but these errors were encountered: