FinetuneGLMWithPeft

Strongly recommend:

https://github.com/mymusise/ChatGLM-Tuning

https://github.com/ssbuild/chatglm_finetuning

https://github.com/lich99/ChatGLM-finetune-LoRA

Simple implementation of using lora form the peft library to fine-tune the chatglm-6b

This project is just a demo for sharing ideas about using lora from peft to fine-tune chatglm-6b.

Any seggestions are welcomed！

Requirements

datasets

peft

transformers

tqdm

torch

TODO

• Hugging face Trainer

• Deepspeed stage3 config

• Accelerator

• validation