Skip to content

Commit

Permalink
update merge weights , suit for DB-GPT
Browse files Browse the repository at this point in the history
  • Loading branch information
wangzaistone committed Nov 2, 2023
1 parent b8a2a98 commit def504d
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 8 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ The value of the parameter `--predicted_out_filename` is the file name of the m

### 3.5 Model Weights
You can find the second corresponding model weights from Huggingface [hg-eosphoros-ai
](https://huggingface.co/Wangzaistone123/CodeLlama-13b-sql-lora) ,we uploaded the LoRA weights in October,which execution accuracy on the Spider evaluation set reached 0.789.
](https://huggingface.co/Wangzaistone123/CodeLlama-13b-sql-lora) ,we uploaded the LoRA weights in October,which execution accuracy on the Spider evaluation set reached 0.789.

#### 3.5.1 Model and fine-tuned weight merging

Expand All @@ -225,7 +225,7 @@ sh ./dbgpt_hub/scripts/export_merge.sh
```

Be sure to replace the parameter path values in the script with the paths corresponding to your project.

### 3.6 Model Evaluation
To evaluate model performance on the dataset, default is spider dev dataset.
Run the following command:
Expand Down
2 changes: 1 addition & 1 deletion README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ sh ./dbgpt_hub/scripts/predict_sft.sh
```bash
sh ./dbgpt_hub/scripts/export_merge.sh
```
注意将脚本中的相关参数路径值替换为你项目所对应的路径。
注意将脚本中的相关参数路径值替换为你项目所对应的路径。


### 3.6、模型评估
Expand Down
21 changes: 16 additions & 5 deletions dbgpt_hub/scripts/export_merge.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,19 @@
# llama2 series
python dbgpt_hub/train/export_model.py \
--model_name_or_path Your_base_model_path_like_Baichuan2-13B-Chat \
--template Your_template_like_baichuan2_eval \
--model_name_or_path /home/LLM/CodeLlama-13b-Instruct-hf \
--template llama2 \
--finetuning_type lora \
--checkpoint_dir Your_ckpt_path_checkpoint-100 \
--output_dir Your_export_model_like_output_merge_model_baichuan2-13b-qlora_merge \
--checkpoint_dir dbgpt_hub/output/adapter/CodeLlama-13b-sql-lora \
--output_dir dbgpt_hub/output/codellama-13b-sql-sft \
--fp16
# --bf16


## Baichuan2
# python dbgpt_hub/train/export_model.py \
# --model_name_or_path Your_base_model_path_like_Baichuan2-13B-Chat \
# --template Your_template_like_baichuan2_eval \
# --finetuning_type lora \
# --checkpoint_dir Your_ckpt_path_checkpoint-100 \
# --output_dir Your_export_model_like_output_merge_model_baichuan2-13b-qlora_merge \
# --fp16
# # --bf16

0 comments on commit def504d

Please sign in to comment.