diff --git a/README.md b/README.md index 6d098a9..d5fd01f 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,267 @@ [**简体中文**](README.zh.md) | [**Discord**](https://discord.gg/7uQnPuveTY) | [**Wechat**](https://github.com/eosphoros-ai/DB-GPT/blob/main/README.zh.md#%E8%81%94%E7%B3%BB%E6%88%91%E4%BB%AC) | [**Huggingface**](https://huggingface.co/eosphoros) | [**Community**](https://github.com/eosphoros-ai/community) +## Baseline +- update time: 2023/12/08 +- metric: execution accuracy (ex) +- more details refer to [docs/eval-llm-result.md](https://github.com/eosphoros-ai/DB-GPT-Hub/blob/main/docs/eval_llm_result.md) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModelMethodEasyMediumHardExtraAll
base00000
Llama2-7B-Chatlora0.8870.6410.4890.3310.626
qlora0.8470.6230.4660.3610.608
base00000
Llama2-13B-Chatlora0.9070.7290.5520.3430.68
qlora0.9110.70.5520.3190.664
base0.2140.1770.0920.0360.149
CodeLlama-7B-Instructlora0.9230.7560.5860.3490.702
qlora0.9110.7510.5980.3310.696
base0.6980.6010.4080.2710.539
CodeLlama-13B-Instructlora0.940.7890.6840.4040.746
qlora0.940.7740.6260.3920.727
base0.5770.3520.2010.066335
Baichuan2-7B-Chatlora0.8710.630.4480.2950.603
qlora0.8910.6370.4890.3310.624
base0.5810.4130.2640.1870.392
Baichuan2-13B-Chatlora0.9030.7020.5690.3920.678
qlora0.8950.6750.580.3430.659
base0.3950.2560.1380.0420.235
Qwen-7B-Chatlora0.8550.6880.5750.3310.652
qlora0.9110.6750.5750.3430.662
base0.8710.6320.3680.1810.573
Qwen-14B-Chatlora0.8950.7020.5520.3310.663
qlora0.9190.7440.5980.3670.701
base00000
ChatGLM3-6blora0.8550.6050.4770.2710.59
qlora0.8430.6030.5060.2110.581
+ + ## Contents - [DB-GPT-Hub: Text-to-SQL parsing with LLMs](#db-gpt-hub-text-to-sql-parsing-with-llms) - [Contents](#contents) diff --git a/README.zh.md b/README.zh.md index 7dbe21d..4657a1c 100644 --- a/README.zh.md +++ b/README.zh.md @@ -26,6 +26,267 @@ [**英文**](README.md) | [**Discord**](https://discord.gg/7uQnPuveTY) | [**Wechat**](https://github.com/eosphoros-ai/DB-GPT/blob/main/README.zh.md#%E8%81%94%E7%B3%BB%E6%88%91%E4%BB%AC) | [**Huggingface**](https://huggingface.co/eosphoros) | [**Community**](https://github.com/eosphoros-ai/community) + +## Baseline +- 更新日期: 2023/12/08 +- 评价指标: execution accuracy (ex) +- 详情参考[docs/eval-llm-result.md](https://github.com/eosphoros-ai/DB-GPT-Hub/blob/main/docs/eval_llm_result.md) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModelMethodEasyMediumHardExtraAll
base00000
Llama2-7B-Chatlora0.8870.6410.4890.3310.626
qlora0.8470.6230.4660.3610.608
base00000
Llama2-13B-Chatlora0.9070.7290.5520.3430.68
qlora0.9110.70.5520.3190.664
base0.2140.1770.0920.0360.149
CodeLlama-7B-Instructlora0.9230.7560.5860.3490.702
qlora0.9110.7510.5980.3310.696
base0.6980.6010.4080.2710.539
CodeLlama-13B-Instructlora0.940.7890.6840.4040.746
qlora0.940.7740.6260.3920.727
base0.5770.3520.2010.066335
Baichuan2-7B-Chatlora0.8710.630.4480.2950.603
qlora0.8910.6370.4890.3310.624
base0.5810.4130.2640.1870.392
Baichuan2-13B-Chatlora0.9030.7020.5690.3920.678
qlora0.8950.6750.580.3430.659
base0.3950.2560.1380.0420.235
Qwen-7B-Chatlora0.8550.6880.5750.3310.652
qlora0.9110.6750.5750.3430.662
base0.8710.6320.3680.1810.573
Qwen-14B-Chatlora0.8950.7020.5520.3310.663
qlora0.9190.7440.5980.3670.701
base00000
ChatGLM3-6blora0.8550.6050.4770.2710.59
qlora0.8430.6030.5060.2110.581
+ ## Contents - [DB-GPT-Hub:利用LLMs实现Text-to-SQL](#db-gpt-hub利用llms实现text-to-sql) - [Contents](#contents)