This is a repository for training a small GPT2-based model locally. It utilizes the distilgpt2 model from HuggingFace: https://huggingface.co/distilgpt2

The source text is Grimm's Fairy Tales, chosen due to its status as public domain and common familiarity with its stories: https://www.gutenberg.org/files/2591/2591-h/2591-h.htm

Run within the project folder to install dependencies in your Python environment:

pip install -r requirements.txt

Execute the code in each cell of main.ipynb to train a GPT model on your CPU or GPU (Nvidia), then load it and test it with your prompts. Training requires 12GB of RAM/VRAM.