dbt-redshift lineage metadata for external databases like awsdata catalog #618
Labels
pkg:dbt-redshift
Issue affects dbt-redshift
triage:product
In Product's queue
type:bug
Something isn't working as documented
Is this a new bug in dbt-redshift?
Current Behavior
Hi Team
I am working on a use case where i have to create dbt models on redshift.
On top of these models we have to build data lineage graph on marquez.
My dbt model has 2 tables (one staging table and other one fact table)
Staging table-it queries data from aws data catalog like this
`
{{ config(materialized='view',bind=False) }}
with source_data as (
)
select *
from source_data
fact table model looks like this
{{ config(materialized='table') }}
SELECT book,hfm_code
FROM
{{ ref('staging_fa_capital_local_af') }}
where entity = '5001'
`
my expectation is that i should see 3 nodes in the lineage graph
1st node: representing source as glue table
2nd node:representing the staging table
3rd node:representing the fact table
However lineage medata data does not provide any information about source coming from external database like awsdata catalog
Lineage event generated is:
Expected Behavior
my expectation is that i should see 3 nodes in the lineage graph
1st node: representing source as glue table
2nd node:representing the staging table
3rd node:representing the fact table
Steps To Reproduce
my expectation is that i should see 3 nodes in the lineage graph
1st node: representing source as glue table
2nd node:representing the staging table
3rd node:representing the fact table
Relevant log output
No response
Environment
Additional Context
No response
The text was updated successfully, but these errors were encountered: