The nanoGPT folder contains an implementation of GPT completly from scratch using just numpy & torch.
- The first use case explored here is language generation. Specifically, the tiny Shakespeare dataset. Ablation studies are carried out to determine the relative importance of different components of the Transformer architecture using the perfromance of the model.
- Text Generation using this model is compared against ChatGPT, GPT-2 & Falcon7B-Instruct. The GPT-2 model is fine tuned first on the Shakespeare dataset.