You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
I'm trying to implement a custom incremental strategy based on this documentation.
But there's a specific macro on dbt-bigquery that checks for the name of the incremental strategy and raises an error in case it's none of the predefined ones:
{% macro dbt_bigquery_validate_get_incremental_strategy(config) %}
{#-- Find and validate the incremental strategy #}
{%- set strategy = config.get("incremental_strategy") or 'merge' -%}
{% set invalid_strategy_msg -%}
Invalid incremental strategy provided: {{ strategy }}
Expected one of: 'merge', 'insert_overwrite', 'microbatch'
{%- endset %}
{% if strategy not in ['merge', 'insert_overwrite', 'microbatch'] %}
{% do exceptions.raise_compiler_error(invalid_strategy_msg) %}
{% endif %}
{% if strategy == 'microbatch' %}
{% do bq_validate_microbatch_config(config) %}
{% endif %}
{% do return(strategy) %}
{% endmacro %}
Expected Behavior
The expected behavior is that it checks for custom incremental strategies as in the doc I shared and set it as valid in case it exists.
Steps To Reproduce
dbt cloud (versionless env)
Use an environment connected to a BigQuery project
Create a custom incremental strategy macro (mine was named get_incremental_merge_upsert_sql)
Create an incremental model based on that, as an example:
Invalid incremental strategy provided: merge_upsert
Expected one of: 'merge', 'insert_overwrite', 'microbatch'>in macro dbt_bigquery_validate_get_incremental_strategy (macros/materializations/incremental.sql)
> called by macro materialization_incremental_bigquery (macros/materializations/incremental.sql)
> called by model stg_mymodel (models/stg/stg_mymodel.sql)
amychen1776
changed the title
[Bug] Custom incremental strategies do not work with BigQuery
[Feature] Custom incremental strategies supported on BigQuery
Nov 18, 2024
Hello @jackodes! Unfortunately we actually don't support Custom Incremental Stratergies on BQ at this time. Please see this issue that covers the remaining work dbt-labs/dbt-core#9290
What I'm going to do is actually close this issue for the time (since we are tracking on that dbt-core issue and this is a duplicate) and get our docs updated accordingly to call out this.
Is this a new bug in dbt-bigquery?
Current Behavior
I'm trying to implement a custom incremental strategy based on this documentation.
But there's a specific macro on dbt-bigquery that checks for the name of the incremental strategy and raises an error in case it's none of the predefined ones:
Expected Behavior
The expected behavior is that it checks for custom incremental strategies as in the doc I shared and set it as valid in case it exists.
Steps To Reproduce
get_incremental_merge_upsert_sql
)Relevant log output
Additional Context
No response
The text was updated successfully, but these errors were encountered: