NLP with Transformers - Follow Along

Motivation

• Consolidate learnings from reading the book Natural Language Processign with Transformers, by reimplementing its most interesting chapters.
• Apply some of the refactoring techniques and software design principles learned from #ArjanCodes to strive for maintainability, readability and scalability.

Method and Results

Approach:

• read a chapter
• reproduce the experiments manually (no copy pasting)
• refactor some of the code (mostly by breaking large code blcoks into smaller functions, bringing some OOP to encapsulate the more complex workflows, decoupling config from code, using protocols when relevant, etc...)

How to Setup Environment

For each chapter, you can setup a dedicated environment by executing the instructions presented in the chapter README.

More Resources

• L. Tunstall & al., "NLP with Transformers", O'Reilly, 2022
• book's official github repo
• ArjanCodes: great resource to learn software design principles applied to Python

Progress

• Chapter 0 - Datasets creation and processing
• Chapter 2 - Text Classification
• Chapter 4 - Multilingual NER
• Chapter 7 - Question Answering
• Chapter 8 - Model Compression
• Chapter 9 - Few to No Labels