Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persist deleted entities who still have _report records to end models #26

Merged
merged 2 commits into from
Nov 19, 2024

Conversation

fivetran-jamie
Copy link
Contributor

@fivetran-jamie fivetran-jamie commented Nov 15, 2024

PR Overview

This PR will address the following Issue/Feature:
No github issue, but will include Ads and other entities that can be hard-deleted from the source _history tables. They still have records in the _report tables, though their IDs get updated to -1 .

This PR will result in the following new package version:

v0.4.1

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Bug Fixes

  • Adjusts inner joins in each end model to be left joins.
    • When ads and other entities are deleted from Apple Search Ads, their records are hard-deleted from the <entity>_history source tables, though any associated <entity>_report records persist with new <entity>_id values of -1. This update ensures that the report records persist into the package's end models in these cases.

Note: This will likely increase the row count of your data models. To remove these newly included records, filter out rows where the <entity>_id = -1.

  • We have accordingly made sure to select fields, specifically entity IDs, from the left side of these joins (the report tables).

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt run –full-refresh && dbt test
  • dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

  • The appropriate issue has been linked, tagged, and properly assigned
  • All necessary documentation and version upgrades have been applied
  • docs were regenerated (unless this PR does not include any code or yml updates)
  • BuildKite integration tests are passing
  • Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

Validation tests updated and passing

image

If you had to summarize this PR in an emoji, which would it be?

🍏

Copy link
Collaborator

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@fivetran-catfritz fivetran-catfritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@fivetran-jamie fivetran-jamie merged commit 566a3fe into main Nov 19, 2024
8 checks passed
@fivetran-jamie fivetran-jamie deleted the bugfix/left-join-reports branch November 19, 2024 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants