Skip to content

Commit

Permalink
re-enables stream=false for chat passthrough scratchpads
Browse files Browse the repository at this point in the history
  • Loading branch information
valaises authored and olegklimov committed Dec 27, 2024
1 parent 8d5ea06 commit 92eea7b
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion src/scratchpads/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,13 @@ pub async fn create_chat_scratchpad(
tokenizer_arc.clone(), post, messages, allow_at
));
} else if scratchpad_name == "PASSTHROUGH" {
post.stream = Some(true); // this should be passed from the request
// warning! line below added by sergey. He says that it might be a workaround for something
// yet sergey cannot recall for what and I could not find "holes" it might cover, as:
// create_chat_scratchpad is used only in subchat and _chat, and:
// - subchat does not support stream=true at all
// - _chat is used in handlers: v1/chat, v1/chat/completion
// therefore, I deduct, that it is safe to comment line bellow and re-enable streaming
// post.stream = Some(true); // this should be passed from the request
result = Box::new(chat_passthrough::ChatPassthrough::new(
tokenizer_arc.clone(), post, messages, allow_at, supports_tools, supports_clicks
));
Expand Down

0 comments on commit 92eea7b

Please sign in to comment.