[Feature] Brand and organization variable configs #181

fivetran-avinash · 2024-12-26T13:31:35Z

PR Overview

This PR will address the following Issue/Feature: [#176]

This PR will result in the following new package version: 0.19.2

This adds variable configs that are set to true by default so it doesn't impact the nature of the downstream models unless explicitly set.

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

New Features

Introduced new config variables using_brands and using_organizations to allow the customers to enable and disable brand and organization source data.
End models that could be impacted by these variables include:
- zendesk__ticket_backlog: using_brands and/or using_organizations can be set to false to disable brand and/or organization fields.
- zendesk__ticket_enriched: using_organizations can be set to false to remove organization fields from the final model.
Intermediate models that could be impacted by these variables include:
- int_zendesk__organization_aggregates: using_organizations can be utilized to completely disable the model.
- int_zendesk__ticket_aggregates: using_brands can be set to false to remove brand fields from the final model.
- int_zendesk__updater_information: using_organizations can be set to false to remove organization fields from the final model.

Under the Hood

Updated table_variables in the quickstart.yml with the new brand and organization tables.
Updated our Buildkite model run script to ensure we test for when using_brands and using_organizations is set to either true or false.

Documentation Updates

Updated README with instructions on how to disable brand and organization sources.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

dbt run –full-refresh && dbt test
dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

The appropriate issue has been linked, tagged, and properly assigned
All necessary documentation and version upgrades have been applied
docs were regenerated (unless this PR does not include any code or yml updates)
BuildKite integration tests are passing
Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

Various versions of variable configs with the new fields were introduced, run and tested to ensure the variable configs worked for various permutations of organization/brand enablement or disablement. Validation testing on our Zendesk environment also successfully passed (with an additional filter provided for a new ticket that did not pass but was not introduced for the purposes of this issue)

If you had to summarize this PR in an emoji, which would it be?

🦷

fivetran-avinash

reminder to commit before merge.

fivetran-avinash · 2024-12-26T13:33:11Z

packages.yml

+  - git: https://github.com/fivetran/dbt_zendesk_source.git
+    revision: feature/organization-brand-enable-disable
+    warn-unpinned: false


Suggested change

- git: https://github.com/fivetran/dbt_zendesk_source.git

revision: feature/organization-brand-enable-disable

warn-unpinned: false

- package: fivetran/zendesk_source

version: [">=0.14.0", "<0.15.0"]

fivetran-reneeli · 2025-01-03T19:22:53Z

models/intermediate/int_zendesk__organization_aggregates.sql

Since we have the using_organizations config at the top, I don't think it would be needed in the rest of the model since in order for the model to run, it must be true.

fivetran-reneeli · 2025-01-03T19:40:57Z

models/zendesk__ticket_enriched.sql

        requester_org.organization_tags as requester_organization_tags,
        {% endif %}
+        --If you use organizations this will be included, if not it will be ignored.
+        {% if var('using_organizations', True) %}


Perhaps we can combine with line 89 into just 1 config?

requester_org.name as requester_organization_name,

fivetran-reneeli · 2025-01-03T19:44:01Z

README.md

@@ -147,6 +147,8 @@ vars:
    using_domain_names:         False         #Disable if you are not using domain names
    using_user_tags:            False         #Disable if you are not using user tags
    using_ticket_form_history:  False         #Disable if you are not using ticket form history
+    using_brands:               False         #Disable if you are not using brands
+    using_organizations:        False         #Disable if you are not using organizations


Same callout as in source about also adding a note for organization_tag here

fivetran-reneeli · 2025-01-03T19:44:50Z

.quickstart/quickstart.yml

+  using_brands:
+    - brand
+  using_organizations:
+    - organization


Believe we also need to add organization_tag here

fivetran-reneeli · 2025-01-03T19:46:42Z

integration_tests/dbt_project.yml

Reminder to up the version

fivetran-reneeli · 2025-01-03T19:48:23Z

integration_tests/tests/integrity/sla_count_match.sql

Wondering what this change was for and if we need to document this in the changelog?

fivetran-reneeli · 2025-01-03T19:50:01Z

CHANGELOG.md

+[PR #181](https://github.com/fivetran/dbt_zendesk/pull/181) includes the following updates:
+
+## New Features
+- Introduced new config variables `using_brands` and `using_organizations` to allow the customers to enable and disable `brand` and `organization` source data. 


Suggested change

- Introduced new config variables `using_brands` and `using_organizations` to allow the customers to enable and disable `brand` and `organization` source data.

- Introduced new config variables `using_brands` and `using_organizations` to allow customers to enable and disable `brand` and `organization` source data, resulting in potential downstream impacts:

fivetran-reneeli

Thanks @fivetran-avinash ! Have some suggestions in addition to a reminder to update the run script to test for using_brands and using_organizations being set to false.

Brand and organization variable configs

f4ddd16

fivetran-avinash self-assigned this Dec 26, 2024

fivetran-avinash commented Dec 26, 2024

View reviewed changes

fivetran-reneeli reviewed Jan 3, 2025

View reviewed changes

integration_tests/dbt_project.yml

Copy link

Contributor

fivetran-reneeli Jan 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder to up the version

fivetran-reneeli reviewed Jan 3, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Brand and organization variable configs #181

[Feature] Brand and organization variable configs #181

fivetran-avinash commented Dec 26, 2024 •

edited

Loading

fivetran-avinash left a comment

fivetran-avinash Dec 26, 2024

fivetran-reneeli Jan 3, 2025

fivetran-reneeli Jan 3, 2025

fivetran-reneeli Jan 3, 2025

fivetran-reneeli Jan 3, 2025

fivetran-reneeli Jan 3, 2025

fivetran-reneeli Jan 3, 2025

fivetran-reneeli Jan 3, 2025

fivetran-reneeli left a comment

	- Introduced new config variables `using_brands` and `using_organizations` to allow the customers to enable and disable `brand` and `organization` source data.
	- Introduced new config variables `using_brands` and `using_organizations` to allow customers to enable and disable `brand` and `organization` source data, resulting in potential downstream impacts:

[Feature] Brand and organization variable configs #181

Are you sure you want to change the base?

[Feature] Brand and organization variable configs #181

Conversation

fivetran-avinash commented Dec 26, 2024 • edited Loading

PR Overview

New Features

Under the Hood

Documentation Updates

PR Checklist

Basic Validation

Detailed Validation

If you had to summarize this PR in an emoji, which would it be?

fivetran-avinash left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fivetran-reneeli left a comment

Choose a reason for hiding this comment

fivetran-avinash commented Dec 26, 2024 •

edited

Loading