convert cent based fields to major currency unit based

.buildkite/scripts/run_models.sh

@@ -19,7 +19,7 @@ dbt deps
 dbt seed --target "$db" --full-refresh
 dbt run --target "$db" --full-refresh
 dbt test --target "$db"
-dbt run --vars '{stripe__using_invoice_line_sub_filter: false, stripe__using_credit_notes: true, stripe__using_price: false, stripe__using_subscription_history: false}' --full-refresh --target "$db"
+dbt run --vars '{stripe__using_invoice_line_sub_filter: false, stripe__using_credit_notes: true, stripe__using_price: false, stripe__using_subscription_history: false, stripe__convert_values: false}' --full-refresh --target "$db"
 dbt test --target "$db"
 
 dbt run-operation fivetran_utils.drop_schemas_automation --target "$db"

CHANGELOG.md

@@ -1,3 +1,17 @@
+# dbt_stripe_source v0.13.0
+[PR [#87](https://github.com/fivetran/dbt_stripe_source/pull/87)] includes the following updates:
+
+## Breaking Change
+- This package assumes that amount-based fields, such as `amount`, `net`, and `fee`, which in raw form are represented in the smallest denomination in Stripe, are cent-based (minor units). Therefore, for example, an amount of 100 cents would be equal to 1 dollar (major unit).
+
+- This PR shifts the existing conversion from cents to dollars to further upstream. Previously, amount-based fields were automatically converted in downstream models, but they are now converted directly in the staging models, with an option to disable this behavior. As the values of amount-based fields have changed, this introduces a breaking change.
+
+- If your Stripe data is *not* using a currency involving minor units, you may want to disable this default conversion in order to retain the amount-based fields as raw form. For information on how to do so, refer to the [README]((https://github.com/fivetran/dbt_stripe_source?tab=readme-ov-file#disabling-cent-to-dollar-conversion)) on disabling the `stripe__convert_values` variable.
+  - Examples of currencies using minor units include USD, Euro, and CAD.
+  - Examples of currencies NOT using minor units include Japanese Yen (JPY), Indonesian Rupiah (IDR), and Korean Won (KRW).
+
+- Currently this package does not support multiple currencies, but we have created a [feature flag to support multiple currencies](https://github.com/fivetran/dbt_stripe/issues/102) where you are welcome to provide feedback or contribute to the discussion.
+
 # dbt_stripe_source v0.12.1
 
 ## Feature Updates

README.md

@@ -1,4 +1,6 @@
-<p align="center">
+# Stripe Source dbt Package ([Docs](https://fivetran.github.io/dbt_stripe_source/))
+
+<p align="left">
     <a alt="License"
         href="https://github.com/fivetran/dbt_stripe_source/blob/main/LICENSE">
         <img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" /></a>
@@ -13,7 +15,6 @@
         <img src="https://img.shields.io/badge/Fivetran_Quickstart_Compatible%3F-yes-green.svg" /></a>
 </p>
 
-# Stripe Source dbt Package ([Docs](https://fivetran.github.io/dbt_stripe_source/))
 ## What does this dbt package do?
 <!--section="stripe_source_model"-->
 - Materializes [Stripe staging tables](https://fivetran.github.io/dbt_stripe_source/#!/overview/stripe_source/models/?g_v=1&g_e=seeds) which leverage data in the format described by [this ERD](https://fivetran.com/docs/applications/stripe/#schemainformation). These staging tables clean, test, and prepare your Stripe data from [Fivetran's connector](https://fivetran.com/docs/applications/stripe) for analysis by doing the following:
@@ -45,7 +46,7 @@ If you  are **not** using the [Stripe transformation package](https://github.com
 ```yaml
 packages:
   - package: fivetran/stripe_source
-    version: [">=0.12.0", "<0.13.0"]
+    version: [">=0.13.0", "<0.14.0"]
 ```
 ### Step 3: Define database and schema variables
 By default, this package runs using your destination and the `stripe` schema. If this is not where your stripe data is (for example, if your stripe schema is named `stripe_fivetran`), add the following configuration to your root `dbt_project.yml` file:
@@ -178,6 +179,15 @@ vars:
     stripe__subscription_metadata: ['the', 'list', 'of', 'property', 'fields'] # Note: this is case-SENSITIVE and must match the casing of the property as it appears in the JSON
 ```
 
+#### Disabling Cent to Dollar Conversion
+
+Amount-based fields, such as `amount` and `net`, are typically displayed in the smallest denomination (e.g., cents for USD). By default, these values are automatically converted to major units (dollars for USD) by dividing by `100.0`. However, you may want to disable this default conversion if you are working in a currency that does not have minor units, such as JPY or KRW. To disable this conversion and retain the values in their smallest denomination, set the `stripe__convert_values` variable to `False` as shown below.
+
+```yml
+vars:
+    stripe__convert_values: False
+```
+
 #### Passing Through Additional Fields
 This package includes all source columns defined in the macros folder. You can add more columns using our pass-through column variables. These variables allow for the pass-through fields to be aliased (`alias`) and casted (`transform_sql`) if desired, but not required. Datatype casting is configured via a sql snippet within the `transform_sql` key. You may add the desired sql while omitting the `as field_name` at the end and your custom pass-though fields will be casted accordingly. Use the below format for declaring the respective pass-through variables:
 

dbt_project.yml

@@ -1,6 +1,6 @@
 config-version: 2
 name: 'stripe_source'
-version: '0.12.1'
+version: '0.13.0'
 require-dbt-version: [">=1.3.0", "<2.0.0"]
 
 models:

integration_tests/dbt_project.yml

@@ -1,7 +1,7 @@
 config-version: 2
 
 name: 'stripe_source_integration_tests'
-version: '0.12.1'
+version: '0.13.0'
 
 
 profile: 'integration_tests'

macros/convert_values.sql

@@ -0,0 +1,7 @@
+{% macro convert_values(field_name, divide_by=100.0, divide_var='stripe__convert_values', alias=None) %}
+    {% if var(divide_var, True) %}
+        {{ field_name }} / {{ divide_by }} as {{ alias if alias else field_name }}
+    {% else %}
+        {{ field_name }} as {{ alias if alias else field_name }}
+    {% endif %}
+{% endmacro %}

models/docs.md

@@ -37,3 +37,7 @@ ZIP or postal code.
 {% docs state -%}
 State, county, province, or region.
 {%- enddocs %}
+
+{% docs convert_values -%}
+By default, this value is divided by 100 to get the major currency unit. This works for scenarios in which the currency has a minor unit, which is displayed in the raw data, and major unit. For example, if in USD, if the raw value is 100, this represents 100 cents which then gets converted to $1 dollar, the major currency unit. If your currency does not contain minor units (for example the Japanese Yen ¥ which does not use decimals), then you should disable the variable `stripe__convert_values` to turn off the division by 100 (so that the raw value of 100 remains ¥100).
+{%- enddocs %}