Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix unraised DbtDatabaseError #434

Merged
merged 2 commits into from
Sep 6, 2024
Merged

Fix unraised DbtDatabaseError #434

merged 2 commits into from
Sep 6, 2024
+5 −3

Conversation

Jeremynadal33
Copy link
Contributor

resolves #433

Description

Modified the DbtDatabaseError from dbt to dbt_common

Checklist

  • I have signed the CLA
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt-glue next" section.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

FYI : I am not sure wether this change needs testing, if it does, I would be interested in understanding how to!

Tests that were run locally

Same as in the #433 issue but then got the following output :

09:19:04  Running with dbt=1.8.4
09:19:05  Registered adapter: glue=1.8.1
09:19:05  Found 41 models, 2 snapshots, 1 seed, 2 operations, 1 test, 1 source, 1367 macros
09:19:05  
09:19:07  
09:19:07  
09:19:07  1 of 1 START sql incremental model xxx.test_database_error ............ [RUN]
09:19:17  Glue adapter: Glue returned `error` for statement None for code 

spark.sql("""


select "a" as id
""").write.format("delta").mode("overwrite").save("s3://xxx/xxx/test_database_error")
SqlWrapper2.execute("""select 1""")
, AnalysisException: Failed to merge fields 'id' and 'id'. Failed to merge incompatible data types IntegerType and StringType
09:19:17  1 of 1 ERROR creating sql incremental model xxx.test_database_error ... [ERROR in 10.82s]
09:19:35  
09:19:35  
09:19:35  Finished running 1 incremental model, 2 project hooks in 0 hours 0 minutes and 29.55 seconds (29.55s).
09:19:35  
09:19:35  Completed with 1 error and 0 warnings:
09:19:35  
09:19:35    Database Error in model test_database_error (src/models/test_database_error.sql)
  GlueDeltaWriteTableFailed
09:19:35  
09:19:35  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1

The error is well raised

@Jeremynadal33 Jeremynadal33 changed the title fix dbtDatabaseError Fix unraised DbtDatabaseError Aug 30, 2024
@Jeremynadal33 Jeremynadal33 force-pushed the chore/fix-dbtDatabaseError branch from 77623b6 to af592ec Compare August 30, 2024 10:11
@moomindani moomindani added the enable-functional-tests This label enable functional tests label Sep 6, 2024
@moomindani moomindani merged commit 525b8a9 into aws-samples:main Sep 6, 2024
1 check passed
@moomindani
Copy link
Collaborator

Thanks for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@moomindani moomindani moomindani approved these changes

Assignees
No one assigned
Labels
enable-functional-tests This label enable functional tests repeat-contributor
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

DbtDatabaseError not found resulting in wrongly Succeded model runs
2 participants
@Jeremynadal33 @moomindani