Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stop generating button #424

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Add stop generating button #424

wants to merge 2 commits into from

Conversation

CYBYOB
Copy link
Contributor

@CYBYOB CYBYOB commented Jul 18, 2023

一、整体思路

  1. 每个大模型的constructor中增加source属性。
  2. 每次发送请求时,将SSE实例赋值到source属性。
  3. 点击【停止生成】按钮时,执行_stopGenerating函数(本质:执行this.source.close()操作)。

二、整体效果

  1. 【停止生成】按钮的逻辑、样式是没问题的。

三、现存问题

  1. 思路需优化:LangChainBot.js的LangChainBot类中,我重写了 _sendPrompt 方法 —— 因为需要拿到 SSE 实例并赋给this.source,方便后续的停止生成操作,大佬们有更好的思路的话欢迎讨论、优化
  2. 繁琐:每个大模型需改写constructor和新增_stopGenerating函数
  3. 待补充:国际化只写了中文的。

@vercel
Copy link

vercel bot commented Jul 18, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
chatall ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 18, 2023 0:12am

@sunner sunner changed the title Fix 350 Add stop button Jul 19, 2023
@sunner sunner changed the title Add stop button Add stop generating button Jul 19, 2023
@sunner
Copy link
Member

sunner commented Jul 19, 2023

多谢多谢,这是个被要求了很久的功能。

不过,有一点我没想好,是应该一个按钮停止所有,还是每个模型有一个独立的停止按钮?也可能两者并存。反正,停止机制很重要。有了这个机制,都好办

@sunner
Copy link
Member

sunner commented Jul 19, 2023

你是把 LangChian 整个取消了啊?这可能有问题。会有太多未来扩展需要依赖 LangChain。

LangChain 有个机制去 abort 正在进行的生成:https://js.langchain.com/docs/api/base_language/interfaces/BaseLanguageModelCallOptions#signal

另外,每个模型的接口都不一样,除了 SSE,还有 WebSocket 和非流式的。需要有套机制都兼容它们。难度不算小

@CYBYOB
Copy link
Contributor Author

CYBYOB commented Jul 20, 2023

你是把 LangChian 整个取消了啊?这可能有问题。会有太多未来扩展需要依赖 LangChain。

LangChain 有个机制去 abort 正在进行的生成:https://js.langchain.com/docs/api/base_language/interfaces/BaseLanguageModelCallOptions#signal

另外,每个模型的接口都不一样,除了 SSE,还有 WebSocket 和非流式的。需要有套机制都兼容它们。难度不算小

感谢,我再研究下~

Copy link
Collaborator

@PeterDaveHello PeterDaveHello left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @CYBYOB! I hope everything's going smoothly. Are you considering moving forward with this PR? It appears there are some areas that might need attention. Let me know your thoughts!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants