Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
soroush-abbasi authored Apr 30, 2024
1 parent 7eb38bc commit adffb34
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions llama/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@

# NOLA -- Large Language Model Finetuning (LLaMA)

Our code is based on [QLoRA](https://github.com/artidoro/qlora).

We modified [QLoRA](https://github.com/artidoro/qlora) code and added NOLA to LoRA model. Please install dependencies in a virtual environment:

```
cd llama
pip install -U -r requirements.txt
```

Next, please use the corresponding scripts to fine-tune LLaMA models with NOLA. For instance, to fine-tune LLaMA-2 70B, execute the following script.
Please download LLaMA models from Hugging Face. For example, you can access the LLaMA-2 13B model from [here](https://huggingface.co/meta-llama/Llama-2-13b-hf). Alternatively, you can also use LLaMA-3. Please note that we have not conducted experiments with LLaMA-3 in our paper.

Next, please use the corresponding scripts to fine-tune LLaMA models with NOLA. Please change the `--model_name_or_path` to the path for LLaMA model. For instance, to fine-tune LLaMA-2 13B on the Alpaca dataset, execute the following script.

```
bash scripts/finetune_llama2_guanaco_70b_nola.sh
bash scripts/finetune_llama2_guanaco_13b_nola.sh
```

0 comments on commit adffb34

Please sign in to comment.