Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ollama Local Server service auto-detect with periodic server status #12

Open
repollo opened this issue Oct 6, 2023 · 0 comments
Open

Comments

@repollo
Copy link
Contributor

repollo commented Oct 6, 2023

Description:
Can there be a feature where the application automatically detects a backend service running on Ollama defaults settings and periodically checks the status of the Ollama server using the /api/tags endpoint, as a server status mechanism, so that we can be informed if the server becomes unresponsive or encounters an error. This will enhance the user experience by providing real-time feedback on the server's status.

I dont know TypeScript that well, however see below,

Suggested Implementation:

import axios from 'axios';

const OLLAMA_BASE_URL = 'http://localhost:11434';

const [serverStatus, setServerStatus] = useState<'online' | 'offline' | 'error' | 'unknown'>('unknown');

useEffect(() => {
  const checkServerStatus = async () => {
    try {
      const response = await axios.get(`${OLLAMA_BASE_URL}/api/tags`);
      
      if (response.data && Array.isArray(response.data.models)) {
        setServerStatus('online');
      } else {
        setServerStatus('error');  // Response is not in the expected format
      }
    } catch (error) {
      setServerStatus('offline');  // Network error or server is down
    }
  };

  // Initially check the status when the component mounts
  checkServerStatus();

  // Set up the interval to check server status periodically
  const intervalId = setInterval(checkServerStatus, 10000);  // Check every 10 seconds

  // Clean up the interval when the component unmounts
  return () => clearInterval(intervalId);
}, []);

// The serverStatus state can be used in a status bar to indicate connectivity with the backend

Additional Context:

  • The status check will provide better user feedback and enhance the overall UX of the application.
  • Consider adding more robust error handling and optimizing the polling frequency based on the application's requirements.

EDIT:

After going through the README, I noticed the following claims:

  • Auto check if ollama is running ⏰: This is in line with our original issue regarding checking the status of the Ollama server. Has this been implemented but not released yet, because I dont see it on my latest release, or at least cant figure it out, how is the behavior implemented or what should I be looking at?

  • Detech which models are available to use 📋: Again, this seems to be related to the /api/tags endpoint. It would be great if we could get a brief overview of how it's been implemented.

For both these features, if they are in the pipeline and not released yet, can we expect them in the next release? And if they have been implemented, it would be helpful to understand their behavior.

Thanks in advance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant