You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After I upgraded unsloth from 2024.11.5 to 2024.11.7, it raised the error that the tokenizer does not have a {% if add_generation_prompt %}.
The model is shenzhi-wang/Llama3.1-8B-Chinese-Chat. It has the following chat template:
{% set loop_messages = messages %}
{% for message in loop_messages %}
{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>
'+ message['content'] | trim + '<|eot_id|>' %}
{% if loop.index0 == 0 %}
{% set content = bos_token + content %}
{% endif %}
{{ content }}
{% endfor %}
{{ '<|start_header_id|>assistant<|end_header_id|>
' }}
Could you check and fix it?
The text was updated successfully, but these errors were encountered:
@Galaxy-Husky Ye so the chat template for that model looks incorrect - it should not have {{ '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }} by default, but rather {% if add_generation_prompt %}{ '<|start_header_id|>assistant<|end_header_id|>\n\n'`
@Galaxy-Husky Ye so the chat template for that model looks incorrect - it should not have {{ '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }} by default, but rather {% if add_generation_prompt %}{ '<|start_header_id|>assistant<|end_header_id|>\n\n'`
Yes, I agree.
But 2024.11.5 will help me add {% if add_generation_prompt %} to fix the template while 2024.11.7 will not and I can't use the model.
Is this the expected behavior?
Hi,
After I upgraded unsloth from 2024.11.5 to 2024.11.7, it raised the error that the tokenizer does not have a {% if add_generation_prompt %}.
The model is shenzhi-wang/Llama3.1-8B-Chinese-Chat. It has the following chat template:
Could you check and fix it?
The text was updated successfully, but these errors were encountered: