[Bug] Support for stop words

[bryanhpchiang]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.

Describe the bug

The ability for the user to specify stop_words does not seem to be exposed. I see that the TurboMind instances has a stop_words attribute but it doesn't seem to be customizable.

Reproduction

.

Error traceback

No response

[lvhan028]

stop_words is hard-coded in model.py
Let's expose it like other fields of the chat template. @AllentDan

[AllentDan]

Added in #352

[bryanhpchiang]

Awesome, thanks for the fast update! Would it be possible to pass in a string / sequence of strings and have those be converted to the tokens under the hood (similar to the OpenAI API interface?)

Or possibly provide documentation on how stop_words should be created from a list of stop sequence strings.

[lvhan028]

I am afraid using string or sequence might not be a good idea for two reasons:

1. There are many codecs of string
2. It'll bring lots of trouble to serve/turbomind/chatbot.py

[lvhan028]

@AllentDan Can you make a document, presenting chat templates and their usage?

[AllentDan]

Yeah, I would add documents later but in other pull requests.

[bryanhpchiang]

I am afraid using string or sequence might not be a good idea for two reasons:

That's understandable. So how does every other library/API offer this? vLLM, OpenAI, etc.

[lvhan028]

My mistake. We'll carefully consider that

[lvhan028]

Merged to main