modellist: automatically replace known chat templates with our versions #3327
+509
−5
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a temporary fix for issues such as #3309, #3282, and #3263 until we can improve Jinja2Cpp itself.
The workaround for users loading a GGUF they downloaded from HuggingFace and then getting an ugly chat template that is in many cases incompatible with Jinja2Cpp, is to make a list of known templates and automatically replace them when found.
This is currently done silently and transparently, aside from a log message that looks like this:
We can make this list less necessary by improving Jinja2Cpp (see the comments in jinja_replacements.cpp for next steps), but for now this is better than what the current release does with these models.
This allows us to be compatible with e.g. any finetune of Llama 3.2 3B Instruct that does not alter the chat template, without having to touch models3.json.
models3.json has been touched in this PR for consistency with the list of substitutions, which now covers all official GPT4All models, even the ones that were working before. This makes them prettier and easier to edit.