dbt blogs

The purpose of this repository is to handle the end to end pipeline of organising posts in r/dataengineering into pre defined categories via an ML model.

Table of Contents

• Quick Start
• Documentation

Quick Start

  Show all details

Create a .env from the example.env via cp .example.env .env and update the env variables to match your settings.

Build the docker images:

• make pg-build: build the postgres image
• make dbt-build: build the dbt image

Start the Postgres and dbt containers:

• make run-services-local

Serve the the dbt documentation at 127.0.0.1:8001

• make dbt-serve-docs

Run dbt commands:

• make dbt-exec-container: enter dbt container
• dbt run: run dbt transformations

Documentation

The full documentation can be found in the docs/ folder.