Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Recurly: Remove definition redundancy, schema redundancy, update to latest CDK and make compatibility with builder #48474

Merged
merged 9 commits into from
Nov 19, 2024

Conversation

btkcodedev
Copy link
Collaborator

@btkcodedev btkcodedev commented Nov 13, 2024

What

Closes #48479
Fix for Stream Unique_coupons: SubstreamPartitionRouter's parent stream doesn't match any other stream

Cause:
Record filter:

record_filter:
condition: "{{ record['coupon_type'] == 'bulk' }}"

Other problems:

  1. The manifest had unwanted repetitions totalling 25000+ lines.
  2. Two definitions were present for all streams
  3. Dual Schema definitions
  4. Non builder compatible

How

  1. Use refs for partition router and append the record_filter
  2. Removed redundancy manually
  3. Added another parent stream with record_filter for unique_coupons stream

@btkcodedev btkcodedev self-assigned this Nov 13, 2024
Copy link

vercel bot commented Nov 13, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Nov 19, 2024 1:20am

@octavia-squidington-iii octavia-squidington-iii added the area/connectors Connector related issues label Nov 13, 2024
@btkcodedev
Copy link
Collaborator Author

btkcodedev commented Nov 13, 2024

/format-fix

Format-fix job started... Check job output.

🟦 Job completed successfully (no changes).

@octavia-squidington-iv octavia-squidington-iv requested a review from a team November 13, 2024 05:43
@btkcodedev btkcodedev changed the title Source Recurly: Remove definition, schema redundancy, Update to latest CDK and make compatibility with builder Source Recurly: Remove definition redundancy, schema redundancy, update to latest CDK and make compatibility with builder Nov 13, 2024
@btkcodedev
Copy link
Collaborator Author

btkcodedev commented Nov 13, 2024

/bump-version type="minor" changelog="Remove definition and schema redundancy, update to latest CDK and make compatibility with builder"

Bump Version job started... Check job output.

✅ Changes applied successfully. (b4ca209)

@btkcodedev btkcodedev changed the title Source Recurly: Remove definition redundancy, schema redundancy, update to latest CDK and make compatibility with builder Source Recurly: Remove definition redundancy, schema redundancy, update to latest CDK Nov 13, 2024
@btkcodedev btkcodedev changed the title Source Recurly: Remove definition redundancy, schema redundancy, update to latest CDK Source Recurly: Remove definition redundancy, schema redundancy, update to latest CDK and make compatibility with builder Nov 14, 2024
Copy link
Contributor

@erohmensing erohmensing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes make sense to me, thanks for the clever fix with the separate parent stream.

Does the unique_coupon_codes stream resolve as its own stream? If so, can you test it in builder and commit back the stream hash?

@btkcodedev
Copy link
Collaborator Author

Verified the stream via builder and updated hash ✔️

@btkcodedev btkcodedev enabled auto-merge (squash) November 19, 2024 01:22
@btkcodedev btkcodedev merged commit f63f4e1 into master Nov 19, 2024
39 checks passed
@btkcodedev btkcodedev deleted the btkcodedev/fixRecurly branch November 19, 2024 01:30
tryangul pushed a commit that referenced this pull request Nov 21, 2024
…te to latest CDK and make compatibility with builder (#48474)

Co-authored-by: Octavia Squidington III <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/recurly
Projects
None yet
3 participants