Skip to content

Commit

Permalink
docs: add rag-in-action arch
Browse files Browse the repository at this point in the history
  • Loading branch information
csunny committed Oct 23, 2023
1 parent 1e7e614 commit 00cd535
Show file tree
Hide file tree
Showing 5 changed files with 35 additions and 5 deletions.
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,17 +98,23 @@ Run on an RTX 4090 GPU.

Currently, we have released multiple key features, which are listed below to demonstrate our current capabilities:
- Private KBQA & data processing

The DB-GPT project offers a range of features to enhance knowledge base construction and enable efficient storage and retrieval of both structured and unstructured data. These include built-in support for uploading multiple file formats, the ability to integrate plug-ins for custom data extraction, and unified vector storage and retrieval capabilities for managing large volumes of information.

- Multiple data sources & visualization

The DB-GPT project enables seamless natural language interaction with various data sources, including Excel, databases, and data warehouses. It facilitates effortless querying and retrieval of information from these sources, allowing users to engage in intuitive conversations and obtain insights. Additionally, DB-GPT supports the generation of analysis reports, providing users with valuable summaries and interpretations of the data.

- Multi-Agents&Plugins

Supports custom plug-ins to perform tasks, natively supports the Auto-GPT plug-in model, and the Agents protocol adopts the Agent Protocol standard

- Fine-tuning text2SQL

An automated fine-tuning lightweight framework built around large language models, Text2SQL data sets, LoRA/QLoRA/Pturning and other fine-tuning methods, making TextSQL fine-tuning as convenient as an assembly line. [DB-GPT-Hub](https://github.com/eosphoros-ai/DB-GPT-Hub)

- Multi LLMs Support, Supports multiple large language models, currently supporting

Massive model support, including dozens of large language models such as open source and API agents. Such as LLaMA/LLaMA2, Baichuan, ChatGLM, Wenxin, Tongyi, Zhipu, etc.
- [Vicuna](https://huggingface.co/Tribbiani/vicuna-13b)
- [vicuna-13b-v1.5](https://huggingface.co/lmsys/vicuna-13b-v1.5)
Expand Down Expand Up @@ -144,6 +150,7 @@ Currently, we have released multiple key features, which are listed below to dem
- [x] [ChatGLM](http://open.bigmodel.cn/)

- Privacy and security

The privacy and security of data are ensured through various technologies such as privatized large models and proxy desensitization.

- Support Datasources
Expand Down Expand Up @@ -185,6 +192,11 @@ The core capabilities mainly consist of the following parts:
6. Privacy & Secure: You can be assured that there is no risk of data leakage, and your data is 100% private and secure.
7. Text2SQL: We enhance the Text-to-SQL performance by applying Supervised Fine-Tuning (SFT) on large language models

### RAG-IN-Action
<p align="center">
<img src="./assets/RAG-IN-ACTION.jpg" width="800px" />
</p>

### SubModule
- [DB-GPT-Hub](https://github.com/eosphoros-ai/DB-GPT-Hub) Text-to-SQL performance by applying Supervised Fine-Tuning (SFT) on large language models.
- [DB-GPT-Plugins](https://github.com/eosphoros-ai/DB-GPT-Plugins) DB-GPT Plugins, Can run autogpt plugin directly
Expand Down
28 changes: 23 additions & 5 deletions README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,15 +59,15 @@ DB-GPT 是一个开源的以数据库为基础的GPT实验项目,使用本地

##### Chat Excel
![excel](https://github.com/eosphoros-ai/DB-GPT/assets/13723926/0474d220-2a9f-449f-a940-92c8a25af390)
##### Chat Plugin
#### Chat Plugin
![auto_plugin_new](https://github.com/eosphoros-ai/DB-GPT/assets/13723926/7d95c347-f4b7-4fb6-8dd2-c1c02babaa56)
##### LLM Management
#### LLM Management
![llm_manage](https://github.com/eosphoros-ai/DB-GPT/assets/13723926/501d6b3f-c4ce-4197-9a6f-f016f8150a11)
##### FastChat && vLLM
#### FastChat && vLLM
![vllm](https://github.com/eosphoros-ai/DB-GPT/assets/13723926/0c9475d2-45ee-4573-aa5a-814f7fd40213)
##### Trace
#### Trace
![trace_new](https://github.com/eosphoros-ai/DB-GPT/assets/13723926/69bd14b8-14d0-4ca9-9cb7-6cef44a2bc93)
##### Chat Knowledge
#### Chat Knowledge
![kbqa_new](https://github.com/eosphoros-ai/DB-GPT/assets/13723926/72266a48-edef-4c6d-88c6-fbb1a24a6c3e)

#### 根据自然语言对话生成分析图表
Expand Down Expand Up @@ -115,14 +115,23 @@ DB-GPT 是一个开源的以数据库为基础的GPT实验项目,使用本地
目前我们已经发布了多种关键的特性,这里一一列举展示一下当前发布的能力。

- 私域问答&数据处理

支持内置、多文件格式上传、插件自抓取等方式自定义构建知识库,对海量结构化,非结构化数据做统一向量存储与检索

- 多数据源&可视化

支持自然语言与Excel、数据库、数仓等多种数据源交互,并支持分析报告。

- 自动化微调

围绕大语言模型、Text2SQL数据集、LoRA/QLoRA/Pturning等微调方法构建的自动化微调轻量框架, 让TextSQL微调像流水线一样方便。详见: [DB-GPT-Hub](https://github.com/eosphoros-ai/DB-GPT-Hub)

- Multi-Agents&Plugins

支持自定义插件执行任务,原生支持Auto-GPT插件模型,Agents协议采用Agent Protocol标准

- 多模型支持与管理

海量模型支持,包括开源、API代理等几十种大语言模型。如LLaMA/LLaMA2、Baichuan、ChatGLM、文心、通义、智谱等。
- 支持多种大语言模型, 当前已支持如下模型:
- [Vicuna](https://huggingface.co/Tribbiani/vicuna-13b)
Expand Down Expand Up @@ -155,6 +164,10 @@ DB-GPT 是一个开源的以数据库为基础的GPT实验项目,使用本地
- [x] [Wenxin](https://cloud.baidu.com/product/wenxinworkshop?track=dingbutonglan)
- [x] [ChatGLM](http://open.bigmodel.cn/)

- 隐私安全

通过私有化大模型、代理脱敏等多种技术保障数据的隐私安全。

- 支持数据源

| DataSource | support | Notes |
Expand Down Expand Up @@ -196,6 +209,11 @@ DB-GPT基于 [FastChat](https://github.com/lm-sys/FastChat) 构建大模型运
6. 隐私和安全: 您可以放心,没有数据泄露的风险,您的数据100%私密和安全。
7. Text2SQL: 我们通过在大型语言模型监督微调(SFT)来增强文本到SQL的性能

### RAG生产落地实践架构
<p align="center">
<img src="./assets/RAG-IN-ACTION.jpg" width="800px" />
</p>

### 子模块
- [DB-GPT-Hub](https://github.com/csunny/DB-GPT-Hub) 通过微调来持续提升Text2SQL效果
- [DB-GPT-Plugins](https://github.com/csunny/DB-GPT-Plugins) DB-GPT 插件仓库, 兼容Auto-GPT
Expand Down
Binary file modified assets/DB-GPT.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/DB-GPT_zh.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/RAG-IN-ACTION.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 00cd535

Please sign in to comment.