Skip to content

Commit

Permalink
Add staging CD jobs (#23)
Browse files Browse the repository at this point in the history
Adds GHAs to call the Staging Deploy jobs for BQ and SF on merge to
`staging`.

This supports the change to make Staging the default branch in Jaffle
Shop (already done), and completes the process of getting us to the
following state:
- 🌉 **Feature branches branch off Staging**
- 🌅 **Feature branches are PR'd against Staging** and run CI
jobs via API call across all active adapters (currently SF and BQ) in
the Staging Environment
- 🆕 **PRs are merged into Staging**, kicking off a CD build job in
Staging (our default build command is `dbt build --exclude
"resource_type:seed" because seeds are depended on as a source, and
running a regular build will create a race condition where some models
beat their source seeds into the warehouse and cause failing tests)
- 🚀 **When we're ready for a Jaffle Shop Release we merge Staging
into Main** kicking off CD jobs via API call in the Prod Environment
across all active adapters
- 🤓 **Dev Environments in dbt Cloud auto defer to Staging**,
as we've set that up as a Staging Environment in Cloud

---------

Co-authored-by: Winnie Winship <[email protected]>
Co-authored-by: dave-connors-3 <[email protected]>
  • Loading branch information
3 people authored Apr 8, 2024
1 parent 2044876 commit 9e622f7
Show file tree
Hide file tree
Showing 3 changed files with 112 additions and 1 deletion.
3 changes: 2 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
name: Run dbt Cloud CI job
name: dbt Cloud CI

on:
pull_request:
branches:
- main
- staging

jobs:
run_snowflake:
Expand Down
55 changes: 55 additions & 0 deletions .github/workflows/prod_cd.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: dbt Cloud Deploy

on:
push:
branches:
- main

jobs:
run_snowflake:
name: dbt Cloud Deploy Prod Snowflake
runs-on: macos-latest

env:
DBT_ACCOUNT_ID: 188483
DBT_PROJECT_ID: 283328
DBT_PR_JOB_ID: 409009
DBT_API_KEY: ${{ secrets.DBT_CLOUD_API_KEY }}
DBT_JOB_CAUSE: "GitHub Actions Request"
DBT_JOB_BRANCH: main

steps:
- uses: "actions/checkout@v4"
- uses: "actions/setup-python@v5"
with:
python-version: "3.12"
- name: Install uv
run: python3 -m pip install uv
- name: Install deps
run: uv pip install -r requirements.txt --system
- name: Run dbt Cloud job
run: python3 .github/workflows/scripts/dbt_cloud_run_job.py

run_bigquery:
name: dbt Cloud Deploy Prod BigQuery
runs-on: macos-latest

env:
DBT_ACCOUNT_ID: 188483
DBT_PROJECT_ID: 275557
DBT_PR_JOB_ID: 553247
DBT_API_KEY: ${{ secrets.DBT_CLOUD_API_KEY }}
DBT_JOB_CAUSE: "GitHub Actions Request"
DBT_JOB_BRANCH: main

steps:
- uses: "actions/checkout@v4"
- uses: "actions/setup-python@v5"
with:
python-version: "3.12"
- name: Install uv
run: python3 -m pip install uv
- name: Install deps
run: uv pip install -r requirements.txt --system
- name: Run dbt Cloud job
run: python3 .github/workflows/scripts/dbt_cloud_run_job.py
55 changes: 55 additions & 0 deletions .github/workflows/staging_cd.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: dbt Cloud Deploy Staging

on:
push:
branches:
- staging

jobs:
run_snowflake:
name: dbt Cloud Deploy Staging Snowflake
runs-on: macos-latest

env:
DBT_ACCOUNT_ID: 188483
DBT_PROJECT_ID: 283328
DBT_PR_JOB_ID: 565266
DBT_API_KEY: ${{ secrets.DBT_CLOUD_API_KEY }}
DBT_JOB_CAUSE: "GitHub Actions Request"
DBT_JOB_BRANCH: main

steps:
- uses: "actions/checkout@v4"
- uses: "actions/setup-python@v5"
with:
python-version: "3.12"
- name: Install uv
run: python3 -m pip install uv
- name: Install deps
run: uv pip install -r requirements.txt --system
- name: Run dbt Cloud job
run: python3 .github/workflows/scripts/dbt_cloud_run_job.py

run_bigquery:
name: dbt Cloud Deploy Staging BigQuery
runs-on: macos-latest

env:
DBT_ACCOUNT_ID: 188483
DBT_PROJECT_ID: 275557
DBT_PR_JOB_ID: 560539
DBT_API_KEY: ${{ secrets.DBT_CLOUD_API_KEY }}
DBT_JOB_CAUSE: "GitHub Actions Request"
DBT_JOB_BRANCH: main

steps:
- uses: "actions/checkout@v4"
- uses: "actions/setup-python@v5"
with:
python-version: "3.12"
- name: Install uv
run: python3 -m pip install uv
- name: Install deps
run: uv pip install -r requirements.txt --system
- name: Run dbt Cloud job
run: python3 .github/workflows/scripts/dbt_cloud_run_job.py

0 comments on commit 9e622f7

Please sign in to comment.