Skip to content

Commit

Permalink
[destination-motherduck] append batch tmp table instead replace durin…
Browse files Browse the repository at this point in the history
…g override (#50425)

Co-authored-by: Guen Prawiroatmodjo <[email protected]>
Co-authored-by: Octavia Squidington III <[email protected]>
  • Loading branch information
3 people authored Dec 26, 2024
1 parent 589705c commit c519289
Show file tree
Hide file tree
Showing 5 changed files with 698 additions and 666 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -261,17 +261,10 @@ def _write_temp_table_to_target_table(
sync_mode: DestinationSyncMode,
) -> None:
"""Write the temp table into the final table using the provided write strategy."""
if sync_mode == DestinationSyncMode.overwrite:
# Note: No need to check for schema compatibility
# here, because we are fully replacing the table.
self._swap_temp_table_with_final_table(
stream_name=stream_name,
temp_table_name=temp_table_name,
final_table_name=final_table_name,
)
return

if sync_mode == DestinationSyncMode.append:
if sync_mode == DestinationSyncMode.append or sync_mode == DestinationSyncMode.overwrite:
# Because overwrite drops the table and reinsert all the data
# we can use the same logic as append.
# The table is dropped during (_ensure_table_exists)
self._ensure_compatible_table_schema(
stream_name=stream_name,
table_name=final_table_name,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ data:
connectorSubtype: database
connectorType: destination
definitionId: 042ee9b5-eb98-4e99-a4e5-3f0d573bee66
dockerImageTag: 0.1.16
dockerImageTag: 0.1.17
dockerRepository: airbyte/destination-motherduck
githubIssueLabel: destination-motherduck
icon: duckdb.svg
Expand Down
Loading

0 comments on commit c519289

Please sign in to comment.