convert cent based fields to major currency unit based

.buildkite/scripts/run_models.sh

@@ -19,7 +19,7 @@ dbt deps
 dbt seed --target "$db" --full-refresh
 dbt run --target "$db" --full-refresh
 dbt test --target "$db"
-dbt run --vars '{stripe__using_invoice_line_sub_filter: false, stripe__using_credit_notes: true, stripe__using_price: false, stripe__using_subscription_history: false}' --full-refresh --target "$db"
+dbt run --vars '{stripe__using_invoice_line_sub_filter: false, stripe__using_credit_notes: true, stripe__using_price: false, stripe__using_subscription_history: false, stripe__convert_values: true}' --full-refresh --target "$db"
 dbt test --target "$db"
 
 dbt run-operation fivetran_utils.drop_schemas_automation --target "$db"

CHANGELOG.md

@@ -1,6 +1,24 @@
-# dbt_stripe_source version.version
+# dbt_stripe_source v0.12.2
+[PR [#87](https://github.com/fivetran/dbt_stripe_source/pull/87)] includes the following updates:
+
+## Feature Updates: Optionally Convert Amounts to Major Units
+- Stripe passes amount-based fields, such as `amount`, `net`, and `fee`, in the smallest denomination as raw form. This means, if your currency has minor and major units such as USD, 100 represents 100 cents, the minor unit, or 1 USD, the major unit. Alternatively, if your currency doesn't use minor units such as JPY, 100 represents 100 JPY. 
+
+- This PR introduces a variable `stripe__convert_values` (`false` by default) which allows users the option to divide all amount-based fields by 100.
+  - For information on how to enable the division, refer to the [README]((https://github.com/fivetran/dbt_stripe_source?tab=readme-ov-file#enabling-cent-to-dollar-conversion)) on configuring the `stripe__convert_values` variable.
+  - Otherwise, amount-based fields will be brought through in their raw form.
+  - Examples of currencies using minor units (in which enabling `stripe__convert_values` is relevant) include USD, Euro, and CAD.
+  - Examples of currencies NOT using minor units (in which it makes more sense to keep the amount-based fields in raw form) include Japanese Yen (JPY), Indonesian Rupiah (IDR), and Korean Won (KRW).
+
+## Under the Hood
+- Introduces the `convert_values` macro that holds the logic for enabling the division by 100, referenced by all amount-based fields.
+- Updates the `run_models.sh` script to test for when `stripe__convert_values` is set to True.
+
+## Notes
+- Currently this package does not support multiple currencies, but we have created a [feature request to support multiple currencies](https://github.com/fivetran/dbt_stripe/issues/102) where you are welcome to provide feedback or contribute to the discussion.
 
 ## Documentation
+- Updates the descriptions for all amount-based fields to specify the grain of the values and add information about the `stripe__convert_values` variable.
 - Corrected references to connectors and connections in the README. ([#88](https://github.com/fivetran/dbt_stripe_source/pull/88))
 
 # dbt_stripe_source v0.12.1

README.md

@@ -1,4 +1,6 @@
-<p align="center">
+# Stripe Source dbt Package ([Docs](https://fivetran.github.io/dbt_stripe_source/))
+
+<p align="left">
     <a alt="License"
         href="https://github.com/fivetran/dbt_stripe_source/blob/main/LICENSE">
         <img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" /></a>
@@ -13,7 +15,6 @@
         <img src="https://img.shields.io/badge/Fivetran_Quickstart_Compatible%3F-yes-green.svg" /></a>
 </p>
 
-# Stripe Source dbt Package ([Docs](https://fivetran.github.io/dbt_stripe_source/))
 ## What does this dbt package do?
 <!--section="stripe_source_model"-->
 - Materializes [Stripe staging tables](https://fivetran.github.io/dbt_stripe_source/#!/overview/stripe_source/models/?g_v=1&g_e=seeds) which leverage data in the format described by [this ERD](https://fivetran.com/docs/applications/stripe/#schemainformation). These staging tables clean, test, and prepare your Stripe data from [Fivetran's connector](https://fivetran.com/docs/applications/stripe) for analysis by doing the following:
@@ -178,6 +179,17 @@ vars:
     stripe__subscription_metadata: ['the', 'list', 'of', 'property', 'fields'] # Note: this is case-SENSITIVE and must match the casing of the property as it appears in the JSON
 ```
 
+#### Enabling Cent to Dollar Conversion
+
+Amount-based fields, such as `amount` and `net`, are typically displayed in the smallest denomination (e.g., cents for USD). By default, amount-based fields will be in this raw form. However, some currencies use major and minor units (for example, cents and dollars when using USD). In these cases, it may be useful to divide the amounts by 100, converting amounts to major units (dollars for USD). To enable the division, configure the `stripe__convert_values` to `true` in your project.yml: 
+
+```yml
+vars:
+    stripe__convert_values: true  # default is false 
+```
+
+If you are working in a currency that does not differentiate between minor and major units, such as JPY or KRW, it may make more sense to keep the amount-based fields in raw form and therefore the package can be ran without configuration. As `stripe__convert_values` is disabled by default, these fields will not be impacted.
+
 #### Passing Through Additional Fields
 This package includes all source columns defined in the macros folder. You can add more columns using our pass-through column variables. These variables allow for the pass-through fields to be aliased (`alias`) and casted (`transform_sql`) if desired, but not required. Datatype casting is configured via a sql snippet within the `transform_sql` key. You may add the desired sql while omitting the `as field_name` at the end and your custom pass-though fields will be casted accordingly. Use the below format for declaring the respective pass-through variables:
 

dbt_project.yml

@@ -1,6 +1,6 @@
 config-version: 2
 name: 'stripe_source'
-version: '0.12.1'
+version: '0.12.2'
 require-dbt-version: [">=1.3.0", "<2.0.0"]
 
 models:

integration_tests/ci/sample.profiles.yml

@@ -16,13 +16,13 @@ integration_tests:
       pass: "{{ env_var('CI_REDSHIFT_DBT_PASS') }}"
       dbname: "{{ env_var('CI_REDSHIFT_DBT_DBNAME') }}"
       port: 5439
-      schema: stripe_source_integrations_tests_6
+      schema: stripe_source_integrations_tests_8
       threads: 8
     bigquery:
       type: bigquery
       method: service-account-json
       project: 'dbt-package-testing'
-      schema: stripe_source_integrations_tests_6
+      schema: stripe_source_integrations_tests_8
       threads: 8
       keyfile_json: "{{ env_var('GCLOUD_SERVICE_KEY') | as_native }}"
     snowflake:
@@ -33,7 +33,7 @@ integration_tests:
       role: "{{ env_var('CI_SNOWFLAKE_DBT_ROLE') }}"
       database: "{{ env_var('CI_SNOWFLAKE_DBT_DATABASE') }}"
       warehouse: "{{ env_var('CI_SNOWFLAKE_DBT_WAREHOUSE') }}"
-      schema: stripe_source_integrations_tests_6
+      schema: stripe_source_integrations_tests_8
       threads: 8
     postgres:
       type: postgres
@@ -42,13 +42,13 @@ integration_tests:
       pass: "{{ env_var('CI_POSTGRES_DBT_PASS') }}"
       dbname: "{{ env_var('CI_POSTGRES_DBT_DBNAME') }}"
       port: 5432
-      schema: stripe_source_integrations_tests_6
+      schema: stripe_source_integrations_tests_8
       threads: 8
     databricks:
       catalog: "{{ env_var('CI_DATABRICKS_DBT_CATALOG') }}"
       host: "{{ env_var('CI_DATABRICKS_DBT_HOST') }}"
       http_path: "{{ env_var('CI_DATABRICKS_DBT_HTTP_PATH') }}"
-      schema: stripe_source_integrations_tests_6
+      schema: stripe_source_integrations_tests_8
       threads: 2
       token: "{{ env_var('CI_DATABRICKS_DBT_TOKEN') }}"
       type: databricks

integration_tests/dbt_project.yml

@@ -1,14 +1,14 @@
 config-version: 2
 
 name: 'stripe_source_integration_tests'
-version: '0.12.1'
+version: '0.12.2'
 
 
 profile: 'integration_tests'
 
 vars:
-  stripe_schema: stripe_source_integrations_tests_6
-  
+  stripe_schema: stripe_source_integrations_tests_8
+
   stripe_source:
     card_pass_through_columns:
       - name: "description"

macros/convert_values.sql

@@ -0,0 +1,7 @@
+{% macro convert_values(field_name, divide_by=100.0, divide_var=var('stripe__convert_values',false), alias=None) %}
+    {% if divide_var == true %}
+        {{ field_name }} / {{ divide_by }} as {{ alias if alias else field_name }}
+    {% else %}
+        {{ field_name }} as {{ alias if alias else field_name }}
+    {% endif %}
+{% endmacro %}

models/docs.md

@@ -37,3 +37,7 @@ ZIP or postal code.
 {% docs state -%}
 State, county, province, or region.
 {%- enddocs %}
+
+{% docs convert_values -%}
+Values can be expressed either in the smallest currency unit (default) or in the major currency unit (divided by 100), as determined by the `stripe__convert_values` variable. Refer to the [README]((https://github.com/fivetran/dbt_stripe_source?tab=readme-ov-file#enabling-cent-to-dollar-conversion)) for more information.
+{%- enddocs %}