feat(model): Support Mixtral-8x7B #959
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fangyinc
changed the title
|
The idea of abstracting an |
Yes, we can also import only the required dependencies. |
|
With the introduction of chat_templating in huggingface, most model adaptations will be simpler, and we do not have to rely on FastChat. |
vshy108
pushed a commit
to vshy108/DB-GPT
that referenced
this pull request
Jan 18, 2024
vshy108
pushed a commit
to vshy108/DB-GPT
that referenced
this pull request
Feb 6, 2024
author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707199703 +0800 parent 3f70da4 author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707198697 +0800 parent 3f70da4 author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707198521 +0800 Add requirements.txt Create only necesasary tables Remove reference info in chat completion result Set disable_alembic_upgrade to True Comment _initialize_awel Comment mount_static_files Fix torch.has_mps deprecated Add API key Comment unused API endpoints Install rocksdict to enable DiskCacheStorage Fix the chat_knowledge missing in chat_mode Update requirements.txt Re-enable awel and add api key check for simple_rag_example DAG Merge main bdf9442 Disable disable_alembic_upgrade Compile bitsandbytes from source and enable verbose Tune the prompt of chat knowledge to only refer to context Add the web static files and uncomment previous unused APIs Add back routers Enable KNOWLEDGE_CHAT_SHOW_RELATIONS Display relation based on CFG.KNOWLEDGE_CHAT_SHOW_RELATIONS Stop reference add to last_output if KNOWLEDGE_CHAT_SHOW_RELATIONS is false Fix always no reference Improve chinese prompts Update requirements.txt Improve prompt Improve prompt Fix prompt variable name Use openhermes-2.5-mistral-7b.Q4_K_M.gguf 1. Fix the delete issue of LlamaCppModel 2. Disable verbose log 3. Update diskcache 4. Remove conda-pack Update chinese prompt and process the model response Extract result from varying tags Add back missing content_matches and put tags regex into variable Update english prompt and decide CANNOT_ANSWER based on language configuration Add 3 new models entries and upgrade bitsandbytes Add few chat templates Update model conversation with fastchat code Revert "Update model conversation with fastchat code" This reverts commit a5dc4b5. Revert "Add few chat templates" This reverts commit e6b6c99. Add OpenHermes-2.5-Mistral-7B chat template Fix missing messages and offset in chat template Update fschat Remove model adapter debugging logs and added conversation template Update chinese chat knowledge prompt Avoid to save the long chat history messages Update chinese chat knowledge prompt Temporary workaround to make the GGUF file use different chat template Use ADD_COLON_SINGLE instead of FALCON_CHAT for separator style Allow no model_name in chat completion request Use starling-lm-7b-alpha.Q5_K_M.gguf Add empty string as system for openchat_3.5 chat template Undo response regex in generate_streaming refactor: Refactor storage and new serve template (eosphoros-ai#947) feat(core): Add API authentication for serve template (eosphoros-ai#950) ci: Add python unit test workflows (eosphoros-ai#954) feat(model): Support Mixtral-8x7B (eosphoros-ai#959) feat(core): Support multi round conversation operator (eosphoros-ai#986) chore(build): Fix typo and new pre-commit config (eosphoros-ai#987) feat(model): Support SOLAR-10.7B-Instruct-v1.0 (eosphoros-ai#1001) refactor: RAG Refactor (eosphoros-ai#985) Co-authored-by: Aralhi <[email protected]> Co-authored-by: csunny <[email protected]> Upgrade english prompt for chat knowledge
vshy108
pushed a commit
to vshy108/DB-GPT
that referenced
this pull request
Feb 13, 2024
author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707199703 +0800 parent 3f70da4 author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707198697 +0800 parent 3f70da4 author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707198521 +0800 Add requirements.txt Create only necesasary tables Remove reference info in chat completion result Set disable_alembic_upgrade to True Comment _initialize_awel Comment mount_static_files Fix torch.has_mps deprecated Add API key Comment unused API endpoints Install rocksdict to enable DiskCacheStorage Fix the chat_knowledge missing in chat_mode Update requirements.txt Re-enable awel and add api key check for simple_rag_example DAG Merge main bdf9442 Disable disable_alembic_upgrade Compile bitsandbytes from source and enable verbose Tune the prompt of chat knowledge to only refer to context Add the web static files and uncomment previous unused APIs Add back routers Enable KNOWLEDGE_CHAT_SHOW_RELATIONS Display relation based on CFG.KNOWLEDGE_CHAT_SHOW_RELATIONS Stop reference add to last_output if KNOWLEDGE_CHAT_SHOW_RELATIONS is false Fix always no reference Improve chinese prompts Update requirements.txt Improve prompt Improve prompt Fix prompt variable name Use openhermes-2.5-mistral-7b.Q4_K_M.gguf 1. Fix the delete issue of LlamaCppModel 2. Disable verbose log 3. Update diskcache 4. Remove conda-pack Update chinese prompt and process the model response Extract result from varying tags Add back missing content_matches and put tags regex into variable Update english prompt and decide CANNOT_ANSWER based on language configuration Add 3 new models entries and upgrade bitsandbytes Add few chat templates Update model conversation with fastchat code Revert "Update model conversation with fastchat code" This reverts commit a5dc4b5. Revert "Add few chat templates" This reverts commit e6b6c99. Add OpenHermes-2.5-Mistral-7B chat template Fix missing messages and offset in chat template Update fschat Remove model adapter debugging logs and added conversation template Update chinese chat knowledge prompt Avoid to save the long chat history messages Update chinese chat knowledge prompt Temporary workaround to make the GGUF file use different chat template Use ADD_COLON_SINGLE instead of FALCON_CHAT for separator style Allow no model_name in chat completion request Use starling-lm-7b-alpha.Q5_K_M.gguf Add empty string as system for openchat_3.5 chat template Undo response regex in generate_streaming refactor: Refactor storage and new serve template (eosphoros-ai#947) feat(core): Add API authentication for serve template (eosphoros-ai#950) ci: Add python unit test workflows (eosphoros-ai#954) feat(model): Support Mixtral-8x7B (eosphoros-ai#959) feat(core): Support multi round conversation operator (eosphoros-ai#986) chore(build): Fix typo and new pre-commit config (eosphoros-ai#987) feat(model): Support SOLAR-10.7B-Instruct-v1.0 (eosphoros-ai#1001) refactor: RAG Refactor (eosphoros-ai#985) Co-authored-by: Aralhi <[email protected]> Co-authored-by: csunny <[email protected]> Upgrade english prompt for chat knowledge
vshy108
pushed a commit
to vshy108/DB-GPT
that referenced
this pull request
Feb 13, 2024
author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707199703 +0800 parent 3f70da4 author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707198697 +0800 parent 3f70da4 author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707198521 +0800 Add requirements.txt Create only necesasary tables Remove reference info in chat completion result Set disable_alembic_upgrade to True Comment _initialize_awel Comment mount_static_files Fix torch.has_mps deprecated Add API key Comment unused API endpoints Install rocksdict to enable DiskCacheStorage Fix the chat_knowledge missing in chat_mode Update requirements.txt Re-enable awel and add api key check for simple_rag_example DAG Merge main bdf9442 Disable disable_alembic_upgrade Compile bitsandbytes from source and enable verbose Tune the prompt of chat knowledge to only refer to context Add the web static files and uncomment previous unused APIs Add back routers Enable KNOWLEDGE_CHAT_SHOW_RELATIONS Display relation based on CFG.KNOWLEDGE_CHAT_SHOW_RELATIONS Stop reference add to last_output if KNOWLEDGE_CHAT_SHOW_RELATIONS is false Fix always no reference Improve chinese prompts Update requirements.txt Improve prompt Improve prompt Fix prompt variable name Use openhermes-2.5-mistral-7b.Q4_K_M.gguf 1. Fix the delete issue of LlamaCppModel 2. Disable verbose log 3. Update diskcache 4. Remove conda-pack Update chinese prompt and process the model response Extract result from varying tags Add back missing content_matches and put tags regex into variable Update english prompt and decide CANNOT_ANSWER based on language configuration Add 3 new models entries and upgrade bitsandbytes Add few chat templates Update model conversation with fastchat code Revert "Update model conversation with fastchat code" This reverts commit a5dc4b5. Revert "Add few chat templates" This reverts commit e6b6c99. Add OpenHermes-2.5-Mistral-7B chat template Fix missing messages and offset in chat template Update fschat Remove model adapter debugging logs and added conversation template Update chinese chat knowledge prompt Avoid to save the long chat history messages Update chinese chat knowledge prompt Temporary workaround to make the GGUF file use different chat template Use ADD_COLON_SINGLE instead of FALCON_CHAT for separator style Allow no model_name in chat completion request Use starling-lm-7b-alpha.Q5_K_M.gguf Add empty string as system for openchat_3.5 chat template Undo response regex in generate_streaming refactor: Refactor storage and new serve template (eosphoros-ai#947) feat(core): Add API authentication for serve template (eosphoros-ai#950) ci: Add python unit test workflows (eosphoros-ai#954) feat(model): Support Mixtral-8x7B (eosphoros-ai#959) feat(core): Support multi round conversation operator (eosphoros-ai#986) chore(build): Fix typo and new pre-commit config (eosphoros-ai#987) feat(model): Support SOLAR-10.7B-Instruct-v1.0 (eosphoros-ai#1001) refactor: RAG Refactor (eosphoros-ai#985) Co-authored-by: Aralhi <[email protected]> Co-authored-by: csunny <[email protected]> Upgrade english prompt for chat knowledge
vshy108
pushed a commit
to vshy108/DB-GPT
that referenced
this pull request
Feb 13, 2024
author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707199703 +0800 parent 3f70da4 author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707198697 +0800 parent 3f70da4 author penghou.ho <[email protected]> 1701341533 +0800 committer penghou.ho <[email protected]> 1707198521 +0800 Add requirements.txt Create only necesasary tables Remove reference info in chat completion result Set disable_alembic_upgrade to True Comment _initialize_awel Comment mount_static_files Fix torch.has_mps deprecated Add API key Comment unused API endpoints Install rocksdict to enable DiskCacheStorage Fix the chat_knowledge missing in chat_mode Update requirements.txt Re-enable awel and add api key check for simple_rag_example DAG Merge main bdf9442 Disable disable_alembic_upgrade Compile bitsandbytes from source and enable verbose Tune the prompt of chat knowledge to only refer to context Add the web static files and uncomment previous unused APIs Add back routers Enable KNOWLEDGE_CHAT_SHOW_RELATIONS Display relation based on CFG.KNOWLEDGE_CHAT_SHOW_RELATIONS Stop reference add to last_output if KNOWLEDGE_CHAT_SHOW_RELATIONS is false Fix always no reference Improve chinese prompts Update requirements.txt Improve prompt Improve prompt Fix prompt variable name Use openhermes-2.5-mistral-7b.Q4_K_M.gguf 1. Fix the delete issue of LlamaCppModel 2. Disable verbose log 3. Update diskcache 4. Remove conda-pack Update chinese prompt and process the model response Extract result from varying tags Add back missing content_matches and put tags regex into variable Update english prompt and decide CANNOT_ANSWER based on language configuration Add 3 new models entries and upgrade bitsandbytes Add few chat templates Update model conversation with fastchat code Revert "Update model conversation with fastchat code" This reverts commit a5dc4b5. Revert "Add few chat templates" This reverts commit e6b6c99. Add OpenHermes-2.5-Mistral-7B chat template Fix missing messages and offset in chat template Update fschat Remove model adapter debugging logs and added conversation template Update chinese chat knowledge prompt Avoid to save the long chat history messages Update chinese chat knowledge prompt Temporary workaround to make the GGUF file use different chat template Use ADD_COLON_SINGLE instead of FALCON_CHAT for separator style Allow no model_name in chat completion request Use starling-lm-7b-alpha.Q5_K_M.gguf Add empty string as system for openchat_3.5 chat template Undo response regex in generate_streaming refactor: Refactor storage and new serve template (eosphoros-ai#947) feat(core): Add API authentication for serve template (eosphoros-ai#950) ci: Add python unit test workflows (eosphoros-ai#954) feat(model): Support Mixtral-8x7B (eosphoros-ai#959) feat(core): Support multi round conversation operator (eosphoros-ai#986) chore(build): Fix typo and new pre-commit config (eosphoros-ai#987) feat(model): Support SOLAR-10.7B-Instruct-v1.0 (eosphoros-ai#1001) refactor: RAG Refactor (eosphoros-ai#985) Co-authored-by: Aralhi <[email protected]> Co-authored-by: csunny <[email protected]> Upgrade english prompt for chat knowledge
Hopshine
pushed a commit
to Hopshine/DB-GPT
that referenced
this pull request
Sep 10, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #956
Other