使用llama3_lora_pretrain.yaml微调模型后，模型生成能力严重丧失

[njzfw1024]

Reminder

• I have read the README and searched the existing issues.

System Info

llama-factory) root@c4ac27a83963:/workspace# llamafactory-cli env
[2024-11-01 02:02:11,224] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to cuda (auto detect)

• llamafactory version: 0.8.4.dev0
• Platform: Linux-5.15.0-116-generic-x86_64-with-glibc2.31
• Python version: 3.10.14
• PyTorch version: 2.3.1+cu121 (GPU)
• Transformers version: 4.42.4
• Datasets version: 2.20.0
• Accelerate version: 0.32.1
• PEFT version: 0.11.1
• TRL version: 0.9.6
• GPU type: NVIDIA A800 80GB PCIe
• DeepSpeed version: 0.14.0

Reproduction

训练数据格式：
[
{"text": "书中的文本3000字"}，
{"text": "书中的文本3000字"}，
。。。。
]
训练脚本为 llama3_lora_pretrain.yaml 训练轮次3

Expected behavior

期待不丧失原有的生成能力

Others

如需其他信息，后面可以补充，请大佬们指教