Skip to content

Commit

Permalink
fix: Revert to non-special tokens tokenizer.
Browse files Browse the repository at this point in the history
  • Loading branch information
Hugoch committed Oct 10, 2024
1 parent 0af45e1 commit a17e35d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/requests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ impl TextGenerationBackend for OpenAITextGenerationBackend {
// we need to count the number of tokens generated as each delta chunk may contain multiple tokens
// that's the case with vLLM chunked prefill or speculative decoding
let num_tokens =
self.tokenizer.encode(content.clone(), true).unwrap().len() as u64;
self.tokenizer.encode(content.clone(), false).unwrap().len() as u64;
if num_tokens > 1 {
warn!(
"Generated more than one token: {num_tokens}",
Expand Down

0 comments on commit a17e35d

Please sign in to comment.