Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop button doesn't actually stop chat streaming from the Ollama server #106

Open
iguzu opened this issue Sep 9, 2024 · 1 comment
Open

Comments

@iguzu
Copy link

iguzu commented Sep 9, 2024

If you submit a chat and press the stop button, Ollamac doesn't stop the Ollama from streaming the response, it just stop updating the UI.

This is bad in general, but particularly bad when the model start to loop into garbage. It may continue to use 100% of GPU for several minutes. The Ollama built-in terminal chat, resets the tcp/http connection of the response streaming when ctrl+C is pressed and I assume the Ollama server stop the inference at that point.

Ollamac should reset the chat streaming tcp/http connection when the stop button is pressed.

This can be easily validated using Wireshark to contrast the two behaviours by looking at the traffic in and out of port 11434 on the local adapter.

@kevinhermawan
Copy link
Owner

Oh, sorry about that. I’ll check it out right away.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants