Project Title

A descriptive title for your project goes here.

Description

This project focuses on developing and implementing machine learning models using PyTorch. It includes various Jupyter Notebooks demonstrating different aspects of model training, tokenization, and inference. Key components include:

• train.ipynb: Training models using PyTorch.
• inference.ipynb: Running inference with trained models.
• mha_GPT.ipynb and my_GPT.ipynb: Implementations of custom GPT models.
• train_mha.ipynb, train_mha_avec_mp.ipynb, and train_orca.ipynb: Advanced training techniques with multi-head attention and other optimizations.
• dataset_bes.ipynb: Notebook for handling and preparing the dataset.
• tinystories_tokeniser.model and tinystories_tokeniser.vocab: Tokenizer files for processing text data.

Installation

To run the notebooks, ensure you have Jupyter and PyTorch installed. You can install these using pip:

pip install torch torchvision torchaudio
pip install jupyterlab

Clone the repository.

1. Open the desired .ipynb file in Jupyter Lab or Notebook.
2. Follow the instructions within each notebook.

Contributing

Contributions to this project are welcome. Please fork the repository and submit a pull request with your changes.

License

This project is licensed under the MIT License - see the LICENSE.md file for details.

Acknowledgments

Thanks to great Recurrent Rebels team