-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
support Qwen/Qwen2.5-Coder-32B-Instruct
model provided by siliconflow
#2974
Conversation
✅ Deploy Preview for continuedev ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
core/llm/llms/OpenAI.ts
Outdated
@@ -288,7 +288,8 @@ class OpenAI extends BaseLLM { | |||
suffix: string, | |||
options: CompletionOptions, | |||
): AsyncGenerator<string> { | |||
const endpoint = new URL("fim/completions", this.apiBase); | |||
const url = this.apiBase?.includes("api.siliconflow.cn") ? "completions" : "fim/completions" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AnoyiX Can we make a dedicated LLM
class instead of putting the logic in OpenAI.ts
? A good example would be in Deepseek.ts
, where we subclass OpenAI
and adjust the FIM logic
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be better to allow custom endpoint
, there are so many providers, it’s hard to support them all, what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that's fine as long as they have the exact same expected request body. I would just make it a function like getFimEndpoint
so that it can be modified by subclasses, and then make a subclass of the endpoint. I'm not worried about having a large number of subclasses
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good. It's convenient for subclasses to implement getFimEndpoint
, this way we can avoid duplicating a lot of code in *_streamFim
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Description
support some models provided by siliconflow, like
Qwen/Qwen2.5-Coder-32B-Instruct
Qwen/Qwen2.5-Coder-7B-Instruct
Testing
custom config:
AI Edit Example:

Autocomplete Example:
