Skip to content

Commit

Permalink
adds duckdb test workflow
Browse files Browse the repository at this point in the history
  • Loading branch information
rudolfix committed Feb 7, 2023
1 parent 7d21ab3 commit 0e3b187
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 14 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/test_dbt_runner.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ on:

env:
# all credentials must be present to be passed to dbt runner
DESTINATION__POSTGRES__CREDENTIALS: postgres://[email protected]:5432/dlt_data
DESTINATION__REDSHIFT__CREDENTIALS: postgres://[email protected]:5439/chat_analytics_rasa_ci
DESTINATION__POSTGRES__CREDENTIALS: postgresql://[email protected]:5432/dlt_data
DESTINATION__REDSHIFT__CREDENTIALS: postgresql://[email protected]:5439/chat_analytics_rasa_ci
DESTINATION__CREDENTIALS__PASSWORD: ${{ secrets.PG_PASSWORD }}

DESTINATION__CREDENTIALS__PROJECT_ID: chat-analytics-rasa-ci
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

name: test loader bigquery
name: test bigquery

on:
pull_request:
Expand All @@ -13,7 +13,6 @@ env:
CREDENTIALS__CLIENT_EMAIL: chat-analytics-loader@chat-analytics-rasa-ci.iam.gserviceaccount.com
CREDENTIALS__PRIVATE_KEY: ${{ secrets.BQ_CRED_PRIVATE_KEY }}
CREDENTIALS__TOKEN_URI: https://oauth2.googleapis.com/token
DATASET_NAME: workflowtest

RUNTIME__LOG_LEVEL: ERROR

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

name: test loader redshift and postgres
name: test redshift, postgres and duckdb

on:
pull_request:
Expand All @@ -9,18 +9,18 @@ on:
workflow_dispatch:

env:
DESTINATION__POSTGRES__CREDENTIALS: postgres://[email protected]:5432/dlt_data
DESTINATION__REDSHIFT__CREDENTIALS: postgres://[email protected]:5439/chat_analytics_rasa_ci
DESTINATION__POSTGRES__CREDENTIALS: postgresql://[email protected]:5432/dlt_data
DESTINATION__REDSHIFT__CREDENTIALS: postgresql://[email protected]:5439/chat_analytics_rasa_ci
DESTINATION__DUCKDB__CREDENTIALS: duckdb:///_storage/test_quack.duckdb
# password is the same so it will be shared
CREDENTIALS__PASSWORD: ${{ secrets.PG_PASSWORD }}
DATASET_NAME: workflowtest

RUNTIME__LOG_LEVEL: ERROR

jobs:

run_loader:
name: Tests redshift loader
name: test destinations redshift, postgres and duckdb
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -63,7 +63,7 @@ jobs:

- name: Install dependencies
# if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction -E redshift
run: poetry install --no-interaction -E redshift -E duckdb

# - name: Install self
# run: poetry install --no-interaction
Expand Down
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@
<p align="center">

[![PyPI version](https://badge.fury.io/py/python-dlt.svg)](https://pypi.org/project/python-dlt/)
[![LINT Badge](https://github.com/scale-vector/dlt/actions/workflows/lint.yml/badge.svg)](https://github.com/scale-vector/dlt/actions/workflows/lint.yml)
[![TEST COMMON Badge](https://github.com/scale-vector/dlt/actions/workflows/test_common.yml/badge.svg)](https://github.com/scale-vector/dlt/actions/workflows/test_common.yml)
[![TEST REDSHIFT Badge](https://github.com/scale-vector/dlt/actions/workflows/test_loader_redshift.yml/badge.svg)](https://github.com/scale-vector/dlt/actions/workflows/test_loader_redshift.yml)
[![TEST BIGQUERY Badge](https://github.com/scale-vector/dlt/actions/workflows/test_loader_bigquery.yml/badge.svg)](https://github.com/scale-vector/dlt/actions/workflows/test_loader_bigquery.yml)
[![LINT Badge](https://github.com/dlt-hub/dlt/actions/workflows/lint.yml/badge.svg)](https://github.com/dlt-hub/dlt/actions/workflows/lint.yml)
[![TEST COMMON Badge](https://github.com/dlt-hub/dlt/actions/workflows/test_common.yml/badge.svg)](https://github.com/dlt-hub/dlt/actions/workflows/test_common.yml)
[![TEST DESTINATIONS Badge](https://github.com/dlt-hub/dlt/actions/workflows/test_destinations.yml/badge.svg)](https://github.com/dlt-hub/dlt/actions/workflows/test_destinations.yml)
[![TEST BIGQUERY Badge](https://github.com/dlt-hub/dlt/actions/workflows/test_destination_bigquery.yml/badge.svg)](https://github.com/dlt-hub/dlt/actions/workflows/test_destination_bigquery.yml)
[![TEST DBT Badge](https://github.com/dlt-hub/dlt/actions/workflows/test_dbt_runner.yml/badge.svg)](https://github.com/dlt-hub/dlt/actions/workflows/test_dbt_runner.yml)


</p>

Expand Down

0 comments on commit 0e3b187

Please sign in to comment.