Beyond the Hype: Building and Evaluating Robust RAG Systems

Building a Retrieval-Augmented Generation (RAG) system that actually works is harder than it looks. Great retrieval? Not enough. Fancy prompts? Still not enough. Without rigorous evaluation, you're just guessing.

This workshop is all about evaluation-driven development—a structured way to measure, diagnose, and improve your RAG system without the trial-and-error chaos. We’ll see a pragmatic, effective and relatively easy way of bootstrapping an evaluation system, show how to fix what’s broken, and give you a clear roadmap to iterate with confidence.

By the end, you’ll know how to go beyond the hype and build a RAG system that truly delivers. 🚀

name	open in
Embed Movies into Vectors
Simple RAG Pipeline
You Need a Domain Expert
LLM-As-a-Judge
HyDE: Hypothetical Document Embeddings
Leverage Metadata To Enanche Retrieval

🤗 How to contribute

1. Clone the repo.

Note

The project uses Python 3.12

2. Install uv, following the official docs

3. Run the following:

uv sync

4. It is highly recommended to use nbstripout to avoid pushing the output of Jupyter Notebooks. Install it with:

uv run pre-commit install --install-hooks