This project is a homework assignment on the implementation of a GPT model for autocompletion. It includes an additional component where we push students to make their own reflections on the societal impact of the model. The main part of the HW is based on Andrej Karpathy's smaller version of the GPT model, called minGPT. minGPT is originally a PyTorch re-implementation of GPT, a highly successful language modeling framework developed by OpenAI. It is a lightweight and efficient implementation of GPT, designed to be easy to use and highly customizable. With its modular architecture and flexible design, minGPT is a powerful tool for researchers and practitioners working in natural language processing and related fields. Students will train the minGPT model on a compilation of different works by Nietzsche - “a German philosopher, prose poet, cultural critic, philologist, and composer.” After implementing the model, students will be able to autocomplete text from sentences fed into the model.
The mingpt_pytorch directory only contains scripts that are direct copies from: https://github.com/karpathy/minGPT. None of the code in this directory is written by us. Furthermore, the code we have implemented is a "JAX/Flax" translation of this PyTorch implementation.