From 90441639b9b86424f66e60665238e0169be4802b Mon Sep 17 00:00:00 2001
From: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
Date: Sat, 11 Nov 2023 21:11:51 -0500
Subject: [PATCH] Feature/llama factory llama2 pipeline (#95)

* added llama-factory under llm_rl

* added sft training bash

* added datasets from llama-factory; will delete later

* finished llama-2-13b train and inference

* fixed minor errors

* changed config

* added deepspeed config

* added more training config to train bash

* adding fix for wandb tags and distributed ranks

* added fastchat data to replicate training for 2k

* tyring to replicate fastchat as close as possible

* before merging

* changed finetue scripts for better performance

* added new data

* example bash

* example bash for mistral