Skip to content

Commit

Permalink
Updated Shopify docs for Shopify partner API
Browse files Browse the repository at this point in the history
  • Loading branch information
dat-a-man committed Jan 30, 2024
1 parent 8321b4e commit ef3b19e
Showing 1 changed file with 98 additions and 14 deletions.
112 changes: 98 additions & 14 deletions docs/website/docs/dlt-ecosystem/verified-sources/shopify.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,25 +7,30 @@ or [book a call](https://calendar.app.google/kiLhuMsWKpZUpfho6) with our support
:::

[Shopify](https://www.shopify.com/) is a user-friendly e-commerce solution that enables anyone to easily create and manage their
own online store.
own online store. Whereas a [Shopify partner](https://partners.shopify.com/) is an individual or company that
develops e-commerce stores, applications, or themes for Shopify merchants, often earning revenue through services, app sales, or
referrals.

This Shopify `dlt` verified source and
[pipeline example](https://github.com/dlt-hub/verified-sources/blob/master/sources/shopify_dlt_pipeline.py)
loads data using Shopify API to the destination of your choice.
loads data using 'Shopify API' or 'Shopify Partner API' to the destination of your choice.

The resources that this verified source supports are:

| Name | Description |
|-----------|-------------------------------------------------------------------------------------|
| customers | Individuals or entities who have created accounts on a Shopify-powered online store |
| orders | Transactions made by customers on an online store |
| products | The individual items or goods that are available for sale |
| Name | Description |
|-----------------------|-------------------------------------------------------------------------------------|
| customers | Individuals or entities who have created accounts on a Shopify-powered online store |
| orders | Transactions made by customers on an online store |
| products | The individual items or goods that are available for sale |
| shopify_partner_query | To query data using GraphQL queries from Shopify partner API |

## Setup Guide

### Grab credentials

#### Grab API token
#### Grab Admin API access token
To load data using Shopify API, you need an Admin API access token. This token can be obtained by following
these steps:

1. Log in to Shopify.
1. Click the settings icon⚙️ at the bottom left.
Expand All @@ -36,9 +41,18 @@ The resources that this verified source supports are:
1. Grant read access in “Admin API access scopes.”
1. Save the configuration.
1. Hit “Install app” and confirm.
1. Reveal and copy the Admin API token from “secrets.toml”. Store safely; it's shown only once.
1. Reveal and copy the Admin API token. Store safely; it's shown only once.

> Note: The Shopify UI, which is described here, might change.
#### Grab Partner API access token
To load data using Shopify Partner API, you need an Partner API access token. This token can be obtained by following
these steps:

1. Log in to Shopify Partners and click the settings icon⚙️ at the bottom left.
1. Scroll to 'Partner API client' in partner account settings and select 'Manage Partner API clients'.
1. Create an API client with a suitable name and assign necessary permissions.
1. Save and create the API client, then click to show and copy the access token securely.

> Note: The Shopify and Shopify Partner UI, described here might change.
The full guide is available at [this link.](https://www.shopify.com/partners/blog/17056443-how-to-generate-a-shopify-api-token)

### Initialize the verified source
Expand Down Expand Up @@ -76,24 +90,32 @@ For more information, read the
```toml
#shopify
[sources.shopify_dlt]
private_app_password=" Please set me up !" #Admin API access token copied above
private_app_password="Please set me up!" #Admin API access token
access_token=" Please set me up!" #Partner API acess token
```

1. Update `private_app_password` with the `API access token` that
[you copied above](shopify.md#grab-api-token).
1. Update `private_app_password` with the `Admin API access token`.
1. Similarly, update the `access_token` with the `Partner API access token`.

>To load data using Shopify API, update the `private_app_password`.
>To load data using Shopify partner API, update the `access_token`.

1. Next, store your pipeline configuration details in the `.dlt/config.toml`.

Here's what the `config.toml` looks like:

```toml
[sources.shopify_dlt]
shop_url = "Please set me up !" # please set me up!
shop_url = "Please set me up !"
organization_id = "Please set me up!"
```

1. Update `shop_url` with the URL of your Shopify store. For example,
"https://shop-123.myshopify.com/%E2%80%9D".

1. Update `organization_id` with a code from your Shopify partner URL. For example in
"https://partners.shopify.com/1234567", the code '1234567' is the organization ID.

1. Next, follow the [destination documentation](../../dlt-ecosystem/destinations) instructions to
add credentials for your chosen destination, ensuring proper routing of your data to the final
destination.
Expand Down Expand Up @@ -188,6 +210,39 @@ def products(
Similar to the mentioned resource, there are two more resources "orders" and "customers", both
support incremental loading and pagination.

### Resource `shopify_partner_query`:
This resource can be used to run custom GraphQL queries to load paginated data.

```python
@dlt.resource
def shopify_partner_query(
query: str,
data_items_path: jp.TJsonPath,
pagination_cursor_path: jp.TJsonPath,
pagination_variable_name: str = "after",
variables: Optional[Dict[str, Any]] = None,
access_token: str = dlt.secrets.value,
organization_id: str = dlt.config.value,
api_version: str = DEFAULT_PARTNER_API_VERSION,
) -> Iterable[TDataItem]:
```

`query`: The GraphQL query for execution.

`data_items_path`: JSONPath to array items in query results.

`pagination_cursor_path`: The JSONPath to the pagination cursor in the query result, will be piped to the next query via variables.

`pagination_variable_name`: The name of the variable to pass the pagination cursor to.

`variables`: Mapping of additional variables used in the query.

`access_token`: The Partner API Client access token, created in the Partner Dashboard.

`organization_id`: Your Organization ID, found in the Partner Dashboard.

`api_version`: The API version to use (e.g. 2024-01). Use `unstable` for the latest version.

## Customization

### Create your own pipeline
Expand Down Expand Up @@ -255,3 +310,32 @@ verified source.
)
print(load_info)
```
1. To load the first 10 transactions via GraphQL query from the Shopify Partner API.
```python
# Construct query to load transactions 100 per page, the `$after` variable is used to paginate
query = """query Transactions($after: String) {
transactions(after: $after, first: 10) {
edges {
cursor
node {
id
}
}
}
}
"""

# Configure the resource with the query and json paths to extract the data and pagination cursor
resource = shopify_partner_query(
query,
# JSON path pointing to the data item in the results
data_items_path="data.transactions.edges[*].node",
# JSON path pointing to the highest page cursor in the results
pagination_cursor_path="data.transactions.edges[-1].cursor",
# The variable name used for pagination
pagination_variable_name="after",
)

load_info = pipeline.run(resource)
print(load_info)
```

0 comments on commit ef3b19e

Please sign in to comment.