Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[source-stripe] Make subscription_items & invoice_line_items able to use Incremental sync mode #48426

Closed
1 task done
tomsseisums opened this issue Nov 8, 2024 · 1 comment

Comments

@tomsseisums
Copy link

Connector Name

source-stripe

Connector Version

5.6.2

What step the error happened?

During the sync

Relevant information

We have subscriptions for our app, but even if we only get a couple of new subscriptions every day, whenever Airbyte runs, it has to load every single Invoice Line Item and Subscription Item, making the sync rather ineffective, especially when using Airbyte Cloud.

For instance, we have a total 24k Invoice Line Items in DB, but even if we get 13 new invoices in our hourly AirByte sync, every hour all of them are being refreshed.
for 13 invoices, 24k invoice lines are loaded

Same for Subscription Items, having loaded just 5 new subscriptions, we still had to refresh all 7.5k items.
for 5 subscriptions, 7.5k subscription items are loaded


As can be seen, there is not even an option to use Incremental modes here.
Image
Image


I'd be open to implement the change myself, but I'd need some pointers.

Relevant log output

No response

Contribute

  • Yes, I want to contribute
@marcosmarxm
Copy link
Member

Duplicate of #30073
@tomsseisums this is a WIP #45860 I'm going try to speed up the review process asap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants