Skip to content

Commit

Permalink
fix heading
Browse files Browse the repository at this point in the history
  • Loading branch information
Adrian authored and Adrian committed Oct 18, 2023
1 parent 2663384 commit 1abbeac
Showing 1 changed file with 12 additions and 12 deletions.
24 changes: 12 additions & 12 deletions docs/website/blog/2023-10-19-dbt-runners.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ tags: [dbt runner, dbt cloud runner, dbt core runner]
>

## Outline:
# Outline:

1. **What is dbt, and what’s the use case for core and cloud?**
- **The Problem dbt Solves**
Expand All @@ -38,23 +38,23 @@ tags: [dbt runner, dbt cloud runner, dbt core runner]

---

# 1. **What is dbt, and what’s the use case for core and cloud?**
### 1. **What is dbt, and what’s the use case for core and cloud?**

**Dbt (data build tool)** is an open-source software that plays a crucial role in the data transformation process. It empowers data analysts and engineers to create, manage, and document data transformation workflows using SQL (Structured Query Language). Dbt primarily focuses on solving the transformation aspect in ELT (Extract, Load, Transform) data processing.

## **The Problem dbt Solves**
### **The Problem dbt Solves**

Dbt addresses the challenge of efficient data transformation, streamlining the 'Transform' stage in ELT workflows. Traditionally, transforming raw data into a structured, analyzable format has been complex and laborious. Dbt simplifies and automates this process, allowing users to define data transformations through SQL queries.

## **What is dbt Core?**
### **What is dbt Core?**

Dbt Core is the fundamental open-source version of dbt. It provides the essential features and functionalities for developing and running data transformation workflows using SQL scripts. Dbt Core offers local execution capabilities, making it suitable for small to medium-scale projects run within a user's environment.

## **What is dbt Cloud?**
### **What is dbt Cloud?**

Dbt Cloud is a cloud-based platform provided by Fishtown Analytics, the company behind dbt. Dbt Cloud offers a managed environment for running dbt, providing additional features and capabilities beyond what dbt Core offers. It is hosted on the cloud, providing a centralized, collaborative, and scalable solution for data transformation needs.

## **When to Use One or the Other?**
### **When to Use One or the Other?**

The choice between dbt Core and dbt Cloud depends on various factors, including the scale of your data transformation needs, collaboration requirements, and resource constraints.

Expand All @@ -69,7 +69,7 @@ The choice between dbt Core and dbt Cloud depends on various factors, including

But, dbt core is free and open source, where dbt cloud is paid. So let’s look into why we would use the paid service:

## **Use Cases of dbt Cloud Over Core**
### **Use Cases of dbt Cloud Over Core**

We could summarize this as: Cloud is the best solution if your Analytics engineer team wants analytics engineer specific tooling and does not want to concern itself with data-engineer specific tooling.

Expand All @@ -80,7 +80,7 @@ We could summarize this as: Cloud is the best solution if your Analytics enginee

So dbt cloud is kind of like a standalone orchestrator, IDE and more.

# 2. What are the use cases for running dbt core or cloud from python?
## 2. What are the use cases for running dbt core or cloud from python?

### Case 1: You have an Analytics engineering team and a data engineering team that work with different tools.

Expand Down Expand Up @@ -114,7 +114,7 @@ In such cases, you would want to create a venv or run the job off the orchestrat

Such, both the **cloud** runner, and the **core** runner with virtual env would fit well here.

# 3. Introducing the dbt runners we have created in open source
## 3. Introducing the dbt runners we have created in open source

Here at dlt we solve the EL in the ELT - so naturally we want to kick off dbt to solve the T.

Expand Down Expand Up @@ -201,9 +201,9 @@ for m in models:
f"and message {m.message}"
```

# 4. A short demo on how to do that with dlt’s dbt runner.
## 4. A short demo on how to do that with dlt’s dbt runner.

## Dbt cloud runner
### Dbt cloud runner

In this example we start from the Pokemon API, load some data with dlt, and then kick off the dbt run in our dbt cloud account.

Expand All @@ -218,7 +218,7 @@ Article: [BQ-dlt-dbt_core-motherduck](https://dlthub.com/docs/blog/dlt-motherduc
Accompanying Github repo: [dbt core runner example](https://github.com/dlt-hub/bigquery-motherduck)


# In conclusion
## In conclusion

Running dbt from python is an obvious need to a data team that also uses python for ingestion, orchestration, or analysis. Having the 2 options, to run Cloud or Core versions of dbt enables better integration between the Transform component and the rest of the data stack.

Expand Down

0 comments on commit 1abbeac

Please sign in to comment.