Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Enable User models by default #17

Merged
merged 3 commits into from
Jan 7, 2025

Conversation

fivetran-avinash
Copy link
Contributor

@fivetran-avinash fivetran-avinash commented Dec 27, 2024

PR Overview

This PR will address the following Issue/Feature: [#14] and [#16]

This PR will result in the following new package version: 0.4.0

New models are now enabled by default so this will create new end models for customers, particularly in Quickstart.

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Breaking Changes

  • Enabled servicenow__user_aggregated and servicenow__user_enhanced by default by changing the default servicenow__using_roles value to true.
    • By setting the servicenow__using_roles variable to true, we now also enable the upstream stg_servicenow_* models that flow into these user tables, which derive from the sys_user_grmember, sys_user_has_role, and sys_user_role source tables.
  • Because this change will introduce new end model tables to users because they were initially disabled by default, we've classified this as a breaking change.

Documentation Update

  • Updated the variable configuration section of the README since servicenow__using_roles is now set to true by default.
  • Moved badges at top of the README below the H1 header to be consistent with popular README formats.

Under the Hood

  • Changed Buildkite scripts to run models when servicenow__using_roles is set to False (since it's now True by default on all models).

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt run –full-refresh && dbt test
  • [NA] dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

  • The appropriate issue has been linked, tagged, and properly assigned
  • All necessary documentation and version upgrades have been applied
  • docs were regenerated (unless this PR does not include any code or yml updates)
  • BuildKite integration tests are passing
  • Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

Ran and validated that the tables compiled when servicenow__using_roles was True or False, then ensured validation tests passed on our development data.

Screenshot 2024-12-27 at 3 01 36 AM

If you had to summarize this PR in an emoji, which would it be?

🍾

@fivetran-avinash fivetran-avinash self-assigned this Dec 27, 2024
Copy link
Collaborator

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-avinash thanks for this update! A few change requests before this will be ready for approval.

CHANGELOG.md Outdated

## Breaking Changes
- Enabled `servicenow__user_aggregated` and `servicenow__user_enhanced` by default by changing the default `servicenow__using_roles` value to true.
- By setting the `servicenow__using_roles` variable to true, we now also enable the upstream `stg_servicenow_*` models that flow into these user tables, which derive from the `sys_user_grmember`, `sys_user_has_role`, and `sys_user_role` source tables.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to clearly callout what new models will be materialized in the warehouse by default. Can you add onto this entry sub-bullets which list out all the new models which will be enabled by default.

Additionally, we should mention that Quickstart will dynamically handle the enable/disabling based on the existence of the source tables or via the Quickstart data model selection tab. However, non Quickstart users will be required to define the variable themselves in their dbt_project.yml if they do not have the required source tables.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated!

CHANGELOG.md Outdated
## Breaking Changes
- Enabled `servicenow__user_aggregated` and `servicenow__user_enhanced` by default by changing the default `servicenow__using_roles` value to true.
- By setting the `servicenow__using_roles` variable to true, we now also enable the upstream `stg_servicenow_*` models that flow into these user tables, which derive from the `sys_user_grmember`, `sys_user_has_role`, and `sys_user_role` source tables.
- Because this change will introduce new end model tables to users because they were initially disabled by default, we've classified this as a breaking change.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also a breaking change because we are changing the default behavior of a variable. Please also include here that users who do not have these source tables will need to define the variable in their dbt_project.yml

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated!

models/servicenow__user_aggregated.sql Show resolved Hide resolved
Copy link
Contributor Author

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-joemarkiewicz Ready for re-review.

Copy link
Collaborator

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-avinash approved with just a few final requests before release review. Thanks!

CHANGELOG.md Outdated
Comment on lines 6 to 9
- Staging models now also enabled by default include:
- `stg_servicenow__sys_user_grmember`
- `stg_servicenow__sys_user_has_role`
- `stg_servicenow__sys_user_role`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also include the base tables.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

CHANGELOG.md Outdated
- `stg_servicenow__sys_user_has_role`
- `stg_servicenow__sys_user_role`
- Because this change will introduce new end model tables to users because they were initially disabled by default and changes the default behavior of a variable, we've classified this as a breaking change.
- Quickstart users will be able to handle disabling of these tables by [utilizing the Quickstart data model tab](https://fivetran.com/docs/using-fivetran/fivetran-dashboard/transformations#datamodels) within your ServiceNow connection, and choose to un-sync the tables required for these models.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Quickstart users will be able to handle disabling of these tables by [utilizing the Quickstart data model tab](https://fivetran.com/docs/using-fivetran/fivetran-dashboard/transformations#datamodels) within your ServiceNow connection, and choose to un-sync the tables required for these models.
- Quickstart users will be able to handle disabling of these tables by [utilizing the Quickstart data model tab](https://fivetran.com/docs/transformations/quickstart) within your ServiceNow connection, and choose to un-sync the tables required for these models.

Let's use this link instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

CHANGELOG.md Outdated
- `stg_servicenow__sys_user_role`
- Because this change will introduce new end model tables to users because they were initially disabled by default and changes the default behavior of a variable, we've classified this as a breaking change.
- Quickstart users will be able to handle disabling of these tables by [utilizing the Quickstart data model tab](https://fivetran.com/docs/using-fivetran/fivetran-dashboard/transformations#datamodels) within your ServiceNow connection, and choose to un-sync the tables required for these models.
- Non-Quickstart users who do not have these source tables will need to re-define the variable in their `dbt_project.yml`. [See the README](https://github.com/fivetran/dbt_servicenow/blob/main/README.md#optional-step-4-additional-configurations) for more instructions.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Non-Quickstart users who do not have these source tables will need to re-define the variable in their `dbt_project.yml`. [See the README](https://github.com/fivetran/dbt_servicenow/blob/main/README.md#optional-step-4-additional-configurations) for more instructions.
- Non-Quickstart users who do not have these source tables will need to define the variable in their `dbt_project.yml`. [See the README](https://github.com/fivetran/dbt_servicenow/blob/main/README.md#optional-step-4-additional-configurations) for more instructions.

It's more likely this variable wasn't defined at all in the past. So we should clarify that this needs to be defined as opposed to re-define.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Copy link
Contributor

@fivetran-reneeli fivetran-reneeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@fivetran-avinash fivetran-avinash merged commit 4dde125 into main Jan 7, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Update default enabled/disabled config [Documentation] Move badges below H1 heading
3 participants