In this repo you have the relevant code to train a CodeLLama model on instruction dataset gathered by the wandbot.
- It uses the Huggignface Integration with Sagemaker
- Formats the dataset accordingly
- Evaluates the model in freshly gathered data
More info on the HF tools used: transformers, datasets.
W&B workspace: https://wandb.ai/capecape/aws_llm_workshop
Before we can start make sure you have met the following requirements
- AWS Account with quota
- AWS CLI installed
- AWS IAM user configured in CLI with permission to create and manage ec2 instances