Merge pull request #48 from fivetran/bug/use-lead-for-spine-end-date

Use lead for end-date in spine + document empty opportunity table solution

.github/workflows/auto-release.yml

@@ -0,0 +1,13 @@
+name: 'auto release'
+on:
+  pull_request:
+    types:
+      - closed
+    branches:
+      - main
+
+jobs:
+  call-workflow-passing-data:
+    if: github.event.pull_request.merged
+    uses: fivetran/dbt_package_automations/.github/workflows/auto-release.yml@main
+    secrets: inherit

.quickstart/quickstart.yml

@@ -0,0 +1,26 @@
+database_key: salesforce_database
+schema_key: salesforce_schema
+
+dbt_versions: ">=1.3.0 <2.0.0"
+
+table_variables:
+  salesforce__user_role_enabled:
+    - user_role
+  salesforce__lead_enabled:
+    - lead
+  salesforce__event_enabled:
+    - event
+  salesforce__task_enabled:
+    - task
+  salesforce__opportunity_line_item_enabled:
+    - opportunity_line_item
+  salesforce__order_enabled:
+    - order
+  salesforce__product_2_enabled:
+    - product_2
+
+destination_configurations:
+  databricks:
+    dispatch:
+      - macro_namespace: dbt_utils
+        search_order: [ 'spark_utils', 'dbt_utils' ]

CHANGELOG.md

@@ -1,3 +1,16 @@
+# dbt_salesforce v1.0.1
+[PR #48](https://github.com/fivetran/dbt_salesforce/pull/48) includes the following updates:
+
+## Bug Fix
+- Aligns the `last_date_query` logic in the `int_salesforce__date_spine` model with the `first_date_query` logic. This ensures that users with empty `opportunity` tables will have a valid end-date (based on `lead` instead of `opportunity`) for the `salesforce__daily_activity` end model.
+  - Also adds coalesce-logic to `int_salesforce__date_spine` to ensure a succesful run without `lead` data.
+
+## Documentation
+- Documents how users without an `opportunity` table can still have the package run successfully for them. See [README](https://github.com/fivetran/dbt_salesforce?tab=readme-ov-file#working-without-an-opportunity-table) for details.
+
+## Under the Hood
+- Included auto-releaser GitHub Actions workflow to automate future releases.
+
 # 🚨 Notice for Quickstart Data Model Users Only 🚨
 Please note that this data model will now create a new transformation for **all** your Salesforce objects (tables) to replicate and include the relevant Salesforce formula fields. With the addition of formula fields, your transformation schema will change to <connector_schema> + `_quickstart`, rather than inheriting the schema from your connector. Please make sure you adjust downstream queries accordingly. If you wish to disable any of these new transformations you may remove them within the UI.
 

README.md

@@ -107,6 +107,20 @@ vars:
 ```
 The corresponding metrics from the disabled tables will not populate in the downstream models.
 
+### Working without an `OPPORTUNITY` Table
+If you do not have the `OPPORTUNITY` table, there is no variable to turn off opportunity-related transformations, as this table is largely the backbone of the Salesforce package. 
+
+However, you may still find value in this package without opportunity data, specifically in the `salesforce__contact_enhanced`, `salesforce__daily_activity`, `salesforce__account_daily_history` and `salesforce__contact_daily_history` (if using History Mode) end models.
+
+For this use case, to ensure the package runs successfully, we recommend leveraging this [Fivetran Feature](https://fivetran.com/docs/using-fivetran/features#syncingemptytablesandcolumns) to create an empty `opportunity` table. To do so, follow these steps:
+1. Navigate to your Salesforce connector in the "Connectors" tab within the Fivetran UI.
+2. Click on the "Schema" tab. 
+3. Scroll down to `Opportunity` and click on its checkbox to add it into your schema.
+4. Click "Save Changes" in the upper righthand corner of the screen.
+5. Either click "Resync" for the `Opportunity` table specifically or wait for your next connector-level sync.
+
+> Note that all other end models (`salesforce__opportunity_enhanced`, `salesforce__opportunity_line_item_enhanced`, `salesforce__manager_performance`, `salesforce__owner_performance`, `salesforce__sales_snapshot`, and `salesforce__opportunity_daily_history`) will still materialize after a blanket `dbt run` but will be largely empty/null.
+
 ## (Optional) Step 4: Utilizing Salesforce History Mode records
 If you have Salesforce [History Mode](https://fivetran.com/docs/getting-started/feature/history-mode) enabled for your connector, we now include support for the `account`, `contact`, and `opportunity` tables directly. These staging models from our `dbt_salesforce_source` package flow into our daily history models. This will allow you access to your historical data for these tables while taking advantage of incremental loads to help with compute.
 

dbt_project.yml

@@ -1,6 +1,6 @@
 config-version: 2
 name: 'salesforce'
-version: '1.0.0'
+version: '1.0.1'
 require-dbt-version: [">=1.3.0", "<2.0.0"]
 models:
   salesforce:

integration_tests/dbt_project.yml

@@ -1,5 +1,5 @@
 name: 'salesforce_integration_tests'
-version: '1.0.0'
+version: '1.0.1'
 config-version: 2
 
 profile: 'integration_tests'

models/salesforce/intermediate/int_salesforce__date_spine.sql

@@ -6,7 +6,7 @@ with spine as (
         {% if var('salesforce__lead_enabled', True) %}
             select  min( created_date ) as min_date from {{ source('salesforce', 'lead') }}
         {% else %}
-            select  min( created_date ) as min_date from {{ source('salesforce', 'opportunity') }}
+            select  coalesce(min( created_date ), '2015-01-01') as min_date from {{ source('salesforce', 'opportunity') }}
         {% endif %}   
 
     {% endset %}
@@ -25,7 +25,12 @@ with spine as (
 
     {% if execute %}
     {% set last_date_query %}
-        select  max( created_date ) as max_date from {{ source('salesforce', 'opportunity') }}
+        {% if var('salesforce__lead_enabled', True) %}
+            select  max( created_date ) as max_date from {{ source('salesforce', 'lead') }}
+        {% else %}
+        select  coalesce(max( created_date ), '2024-01-01') as max_date from {{ source('salesforce', 'opportunity') }}
+        {% endif %}
+
     {% endset %}
 
     {% set current_date_query %}