Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trying to use OllamaPromptDriver without Ollama running causes infinite loop #1438

Closed
steamroller-kb opened this issue Dec 12, 2024 · 5 comments
Labels
type:bug Something isn't working

Comments

@steamroller-kb
Copy link

steamroller-kb commented Dec 12, 2024

Describe the bug
If you try to instantiate a chat using an OllamaPromptDriver, but Ollama is not currently running on your machine, it gets stuck in an infinite loop of printing warnings. If stream=True is set, then even CTRL+C can't kill the process, and the entire terminal needs to killed to stop it (or Task Manager). If stream is not set, you can CTRL+C out of it, and it just prints a stack trace and exits.

To Reproduce
With Ollama not running on your machine:

from griptape.drivers import OllamaPromptDriver
from griptape.structures import Agent
from griptape.utils import Chat

agent = Agent(
    prompt_driver=OllamaPromptDriver(temperature=0.7, model="llama3.2"),
    # stream=True # This is optional for the error, but if set, will cause SIGINT not to kill the process, leading to it running forever until terminated by Task Manager or by killing the terminal
)
Chat(agent).start()

Expected behavior
It would maybe throw one error and exit, or at least be able to CTRL+C out of it even with stream enabled.

Screenshots
image

Desktop (please complete the following information):

@steamroller-kb steamroller-kb added the type:bug Something isn't working label Dec 12, 2024
@collindutter
Copy link
Member

Hi @steamroller-kb, this is intended behavior. Prompt Drivers will exponentially retry max_attempts times. You can change this number by passing in max_attempts=N to the Prompt Driver.

Regarding CTRL+C, you should be able to press it a few times to have it break out fully.

@shhlife
Copy link

shhlife commented Dec 12, 2024

I wonder if we should reduce the max_attempts? I think default is 10 - but I find 2 or 3 is usually fine in most casess.. what do you think @collindutter ?

@collindutter
Copy link
Member

Agreed that 10 is probably too many, but would like to get input from others. Technically this would be a breaking change so we'd have to file it as a griptape 2.0 change.

@shhlife
Copy link

shhlife commented Dec 12, 2024

Sweet, I've logged an enhancement request! #1439

@shhlife
Copy link

shhlife commented Dec 17, 2024

Oh, @collindutter looking back at this I mean to ask - would it be possible to check for ollama to be running & return an error about that if it isn't?

I think that's more of the issue here - @steamroller-kb didn't have ollama running, and instead of returning quickly, it tried over and over again.

Seems like a nicer response would be to let the user know they need to start ollama first?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants