ml-refs

Useful open source references for studying ML. Quality over quantity.

Backprop Libraries

Ordered by ascending complexity.

• Andrej Karpathy: micrograd
• George Hotz: teenygrad
• George Hotz tinygrad

Deep Learning

• Complete course including The Little Book of Deep Learning by François Fleuret from University of Geneva: Deep Learning Course
• The complete CS231n: Deep Learning for Computer Vision lecture has a great introduction to Deep Learning basics, especially backpropagation. It is run by Fei-Fei Li but Andrej Karpathy worked a lot on the material, too.

Generative Models

• Complete lecture with videos by Volodymyr Kuleshov from Cornell University: CS 6785 Deep Probabilistic and Generative Models 2023

Hyperparameter Optimization

• Google's Deep Learning Tuning Playbook

Reinforcement Learning

Complete course with videos by Paderborn University: Reinforcement Learning

Transformers

• Lecture slides by John Hewitt from CS 234N (NLP with DL) at Stanford University in 2023: Lecture 8: Self-Attention and Transformers
• Video by Niels Rogge about how a Transformer works at inference vs. training time