Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

haystack/components/generators/hugging_face_local.py "stop_words" not behaving as expected #8672

Open
sephcodes opened this issue Dec 24, 2024 · 0 comments
Assignees
Labels
P2 Medium priority, add to the next sprint if no P1 available

Comments

@sephcodes
Copy link

Describe the bug
according to the documentation: "param stop_words: If the model generates a stop word, the generation stops."
however, stop_words is just being used to remove whatever is defined as a stopword from the final output. It is not stopping generation when a stop word is encountered.
If possible, a way to stop generation using stop words would be useful

Error message
N/A

Expected behavior
Expecting the stop_words argument to stop generation of output when a value in stop_words list is encountered

Additional context
N/A

To Reproduce
generator = self.generator(model="HuggingFaceTB/SmolLM-1.7B-Instruct",
task="text-generation",
stop_words=self.stop_words,
generation_kwargs={
"max_new_tokens": 150,
# "do_sample": False,
"do_sample": True,
"temperature": 0.5,
# "top_p": 0.9,
# "eos_token_id": self.tokenizer.eos_token_id,
# "stopping_criteria": self.stopping_criteria
})
whatever is defined in self.stop_words will just be dropped from the output but not stop generation of text as expected

FAQ Check
N/A

System:
All

@julian-risch julian-risch added the P2 Medium priority, add to the next sprint if no P1 available label Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 Medium priority, add to the next sprint if no P1 available
Projects
None yet
Development

No branches or pull requests

3 participants