Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support dbt clone #1268

Open
1 task
tatiana opened this issue Oct 21, 2024 · 0 comments · May be fixed by #1326
Open
1 task

Support dbt clone #1268

tatiana opened this issue Oct 21, 2024 · 0 comments · May be fixed by #1326
Assignees
Labels
profile:bigquery Related to BigQuery ProfileConfig profile:databricks Related to Databricks ProfileConfig profile:snowflake Related to Snowflake ProfileConfig
Milestone

Comments

@tatiana
Copy link
Collaborator

tatiana commented Oct 21, 2024

Context

Cosmos 1.7 does not support dbt clone.

The dbt clone command clones selected nodes from the specified state to the target schema(s). This command makes use of the clone materialization:

  • If your data platform supports zero-copy cloning of tables (Snowflake, Databricks, or BigQuery), and this model exists as a table in the source environment, dbt will create it in your target environment as a clone.
  • Otherwise, dbt will create a simple pointer view (select * from the source object)
  • By default, dbt clone will not recreate pre-existing relations in the current target. To override this, use the --full-refresh flag.
  • You may want to specify a higher number of threads to decrease execution time since individual clone statements are independent of one another.

The clone command is useful for:

  • blue/green continuous deployment (on data warehouses that support zero-copy cloning tables)
  • cloning current production state into development schema(s)
  • handling incremental models in dbt Cloud CI jobs (on data warehouses that support zero-copy cloning tables)
  • testing code changes on downstream dependencies in your BI tool

https://docs.getdbt.com/reference/commands/clone

Acceptance criteria

  • Implement a Cosmos operator that supports dbt clone for either Snowflake, Databricks or BQ
@dosubot dosubot bot added profile:bigquery Related to BigQuery ProfileConfig profile:databricks Related to Databricks ProfileConfig profile:snowflake Related to Snowflake ProfileConfig labels Oct 21, 2024
@tatiana tatiana added this to the Cosmos 1.8.0 milestone Oct 30, 2024
@phanikumv phanikumv assigned pankajkoti and pankajastro and unassigned pankajkoti Nov 4, 2024
@pankajastro pankajastro linked a pull request Nov 15, 2024 that will close this issue
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
profile:bigquery Related to BigQuery ProfileConfig profile:databricks Related to Databricks ProfileConfig profile:snowflake Related to Snowflake ProfileConfig
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants