-
Notifications
You must be signed in to change notification settings - Fork 126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove unused rows_affected
column from test_executions
#195
Labels
bug
Something isn't working
Comments
glsdown
pushed a commit
that referenced
this issue
Sep 13, 2023
13 tasks
13 tasks
This should be possible once #381 has been merged, however, I initially included it in there, and found that DBX is having issues when a field gets removed as it gives an error that it's unable to find the field. Runtime Error in model test_executions (models/sources/test_executions.sql)
Cannot resolve rows_affected in UPDATE clause given columns DBT_INTERNAL_SOURCE.command_invocation_id, DBT_INTERNAL_SOURCE.node_id, DBT_INTERNAL_SOURCE.run_started_at, DBT_INTERNAL_SOURCE.was_full_refresh, DBT_INTERNAL_SOURCE.thread_id, DBT_INTERNAL_SOURCE.status, DBT_INTERNAL_SOURCE.compile_started_at, DBT_INTERNAL_SOURCE.query_completed_at, DBT_INTERNAL_SOURCE.total_node_runtime, DBT_INTERNAL_SOURCE.failures, DBT_INTERNAL_SOURCE.message, DBT_INTERNAL_SOURCE.adapter_response.; line 11 pos 2 When I have some time I will look into why removing the field causes this error (Snowflake and BQ are fine), but recording it here for future. |
glsdown
added a commit
that referenced
this issue
Sep 18, 2023
* Add named columns * Remove unused `rows_affected` column in `test_executions` #195 * Reorganise macro folder * Add documentation for macros * Consolidate logic in upload results * Simplify how nodes are extracted * Generalise identifying objects to load further * Have all results included in the loop * Split out get_dataset_content * Move the get table content into its own macro * Bug fixing * Move relation definition to insert macro to avoid quoting issues * Revert "Remove unused `rows_affected` column in `test_executions` #195" This reverts commit 9182fdc. * Remove ---debug left in databricks tox * Apply suggestions from code review Co-authored-by: Jared Rimmer <[email protected]> --------- Co-authored-by: Jared Rimmer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This looks like it may have been copied over from
model_executions
in error.Removing it is a non-trivial task because of the migration path — our insert statements don't enumerate the column names, so just removing all mentions of the column will cause inserts for existing projects to fail. We'll need to find a way to get only the column names that exist in the current incremental source model and use those.
This will be generically useful any time we need to drop a column, which we're likely to need to do again.
The text was updated successfully, but these errors were encountered: