Fine-Tuning LLMs with Hugging Face

This repository contains code for fine-tuning large language models (LLMs) using the Hugging Face Transformers library.

Features:

• Supervised Fine-Tuning of LLMs with peft for parameter efficiency.
• 4-bit quantization using nf4 data type for reduced model size and faster inference.
• Causal language modeling tasks.
• LORA regularization for overfitting prevention.
• Easy usage with pre-defined training arguments and data loading.

Requirements:

• The source dataset: https://huggingface.co/datasets/gamino/wiki_medical_terms
• The formatted dataset: https://huggingface.co/datasets/aboonaji/wiki_medical_terms_llam2_format
• The pre-trained Llama 2 model: https://huggingface.co/aboonaji/llama2finetune-v2
• The from_pretrained method from the AutoModelForCausalLM class: https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoModelForCausalLM.from_pretrained
• The BitsAndBytesConfig class from the transformers library: https://huggingface.co/docs/transformers/main_classes/quantization#transformers.BitsAndBytesConfig
• The from_pretrained method from the AutoTokenizer class: https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoTokenizer.from_pretrained
• The TrainingArguments class from the transformers library: https://huggingface.co/docs/transformers/main_classes/trainer#transformers.TrainingArguments
• The SFTTrainer class from the trl library: https://huggingface.co/docs/trl/sft_trainer#trl.SFTTrainer
• The LoraConfig class from the peft library: https://huggingface.co/docs/peft/v0.7.1/en/package_reference/lora#peft.LoraConfig
• The pipeline function from the transformers library: https://huggingface.co/docs/transformers/main/en/quicktour#pipeline

Outputs