dbt Training with Jaffle Data

Welcome to the dbt training session! In this training, the Jaffle data will be used to explore and understand the fundamentals of dbt. By the end of this session, participants will have a strong grasp of how to utilize dbt to transform and manage data effectively.

Overview

This training session is designed to provide a comprehensive understanding of dbt, covering its core concepts and functionalities. Participants will learn how to use dbt to build and maintain data models, perform data transformations, and ensure data quality through rigorous testing and documentation.

Working with the project

2. Click 'Code', then 'Codespaces, then 'Create codespace on main'.

This will create a new codespace, a sandbox with everything you need for the training.

Once the codespace is finished setting up, you will need to:

1. Update the jaffle_shop_training/profiles.yml with your credentials:
   • Username & password: provided to you individually
   • Schema: dbt_firstname_lastname (eg. dbt_lucy_sheppard)
2. Change directory using the command cd jaffle_shop_training in the terminal
3. Run dbt debug to test your credentials and connection

Data Overview

The Jaffle data set will be used throughout this training. Jaffle data is a sample dataset that simulates a simple e-commerce store, including customers, orders, and payments. This dataset provides a practical context for learning and applying dbt concepts, making it easier to understand real-world applications of dbt.

Agenda

1. dbt Fundamentals

Introduction to the basics of dbt, including its purpose, key components, and how it integrates into the modern data stack. Concepts such as models, runs, builds and configurations will be explained.

2. Sources

Instructions on how to define and manage data sources in dbt. The session will cover the importance of sources, how to declare them in a dbt project, and best practices for managing source freshness.

3. Documentation

Exploration of how to document dbt models and projects effectively. The importance of maintaining documentation and using dbt's built-in documentation features to keep projects organized and understandable will be discussed.

4. Tests

Guidance on writing and running tests in dbt, including schema tests and data tests, to ensure transformations produce accurate results. Emphasis will be placed on the importance of testing for data integrity.

5. Macros and Jinja

Understanding the power of dbt macros and the Jinja templating language. The session will cover how to write reusable SQL snippets using macros and leverage Jinja to make dbt models more dynamic and flexible.

6. Materializations

Overview of different types of materializations in dbt, such as views, tables, and incremental models. Discussion on when and why to use each type, and how to configure them in a dbt project.

7. Incremental Models

Explanation of incremental models and their significance for efficient data processing. Instructions on how to set up and manage incremental models in dbt to process large datasets incrementally rather than reprocessing everything from scratch.