Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add API to get a list of all available models #1160

Merged
merged 3 commits into from
Mar 16, 2025

Conversation

arunabhcode
Copy link
Contributor

@arunabhcode arunabhcode commented Mar 8, 2025

Addressing #828

It adds two calls listModels() and listProviders().

@CLAassistant
Copy link

CLAassistant commented Mar 8, 2025

CLA assistant check
All committers have signed the CLA.

@arunabhcode
Copy link
Contributor Author

arunabhcode commented Mar 8, 2025

Test Code:

import AI from './AI.js';

// Create test instance
const ai = new AI({
    authToken: 'test-token',
    APIOrigin: 'test-origin',
    appID: 'test-app'
});

// Test listModels
const models = ai.listModels();
const providers = ai.listProviders();
console.log('Available models:', JSON.stringify(models, null, 2));
console.log('Available providers:', JSON.stringify(providers, null, 2));

Sample output:

Available models: {
  "chat": {
    "openai": [
      "gpt-4o",
      "gpt-4o-mini"
    ],
    "anthropic": [
      "claude-3-haiku-20240307",
      "claude-3-5-sonnet-20240620",
      "claude-3-5-sonnet-20241022",
      "claude-3-5-sonnet-latest",
      "claude-3-7-sonnet-latest"
    ],
    "together": [
      "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
      "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
      "meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo",
      "google/gemma-2-27b-it"
    ],
    "mistral": [
      "mistral-large-latest",
      "codestral-latest"
    ],
    "groq": [
      "distil-whisper-large-v3-en",
      "gemma2-9b-it",
      "gemma-7b-it",
      "llama-3.1-70b-versatile",
      "llama-3.1-8b-instant",
      "llama3-70b-8192",
      "llama3-8b-8192",
      "llama3-groq-70b-8192-tool-use-preview",
      "llama3-groq-8b-8192-tool-use-preview",
      "llama-guard-3-8b",
      "mixtral-8x7b-32768",
      "whisper-large-v3"
    ],
    "xai": [
      "grok-beta"
    ],
    "deepseek": [
      "deepseek-chat",
      "deepseek-reasoner"
    ],
    "gemini": [
      "gemini-1.5-flash",
      "gemini-2.0-flash"
    ]
  }
}
Available providers: {
  "chat": [
    "openai",
    "anthropic",
    "together",
    "mistral",
    "groq",
    "xai",
    "deepseek",
    "gemini"
  ]
}

@KernelDeimos
Copy link
Contributor

Hi, thanks for the contribution. This unfortunately doesn't address the issue properly - that's maybe on us because the issue you linked to doesn't have any description. I'll add further detail on monday. This should be requesting model information from the backend services in the AI module. I'll try to remember to add more information about this by the end of Monday.

@arunabhcode
Copy link
Contributor Author

@KernelDeimos Sounds good, waiting for further clarification will try to work on it in the meantime and guess around what's needed.

@KernelDeimos
Copy link
Contributor

Hi again, this request to backend gets all supported models

await puter.drivers.call('puter-chat-completion', 'ai-chat', 'models');

This endpoint doesn't split it into "models" and "providers", but it does provide enough information for you to derive that.

@arunabhcode
Copy link
Contributor Author

arunabhcode commented Mar 14, 2025

@KernelDeimos so you want to retrieve the dict returned by calling this and then process that dict to create the dict that I hardcoded?

@KernelDeimos
Copy link
Contributor

Yes that's correct

@arunabhcode
Copy link
Contributor Author

@avijh squashed our commits into one since this is a small PR.

providers = new Set(); // Use a Set to store unique providers

models.result.forEach((item) => {
if (item.provider) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (item.provider) {
if (item.provider) {

Comment on lines 85 to 87
if (item.provider) {
providers.add(item.provider);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is supposed to be intended one more level

Suggested change
if (item.provider) {
providers.add(item.provider);
}
if (item.provider) {
providers.add(item.provider);
}

if (item.provider && item.id) {
if (provider) {
if (item.provider === provider) { // Only add models for the specified provider
if (!modelsByProvider[item.provider]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whenever you see if if if if, you can be sure there's a simpler approach. This amount of nesting increases a metric in static code analysis known as "cognitive complexity" which is a measure of how difficult the code is for somebody to understand and maintain later.

We can use guard clauses (also I found this video which is a bit long but you can get the jist of it by just scanning through real quick and looking at how he's changing the code) to simplify (untested):

...forEach(item => {
    if ( ! item.id ) return;
    if ( provider && ( item.provider !== provider ) ) return;
    if ( ! modelsByProvider[item.provider] ) {
      modelsByProvider[item.provider] = [];
    }
    modelsByProvider[item.provider].push(item.id);
}

Technically we could even combine the first two if statements, but I think it's easier to read when separate concerns still belong on separate lines.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I modified the code per your suggestions and also removed redundant {} for readability.

@KernelDeimos
Copy link
Contributor

Manual test passed. I did manual testing early since I know testing this without API keys is difficult. I'll do a quick test again after my review comments are addressed, and then this will be ready to merge.

pr-1160_1.7x.mp4

@KernelDeimos
Copy link
Contributor

Nice work! Thanks for addressing my comments

@KernelDeimos KernelDeimos merged commit 004e47a into HeyPuter:main Mar 16, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants