Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Source Recharge: migrate to Low-Code #35450

Merged
merged 64 commits into from
Mar 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
e114402
added source.py
bazarnov Feb 14, 2024
dc9b4ee
added Products stream
bazarnov Feb 15, 2024
0b8030c
updated draft
bazarnov Feb 20, 2024
88798d6
updated draft
bazarnov Feb 20, 2024
05216bb
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 21, 2024
ba03fc2
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 22, 2024
6cfb2e8
added metafields and other streams, added RETRY for 200 and broken re…
bazarnov Feb 23, 2024
fca134d
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 23, 2024
95bf6a0
updated poetry lock, formated
bazarnov Feb 23, 2024
92966aa
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 26, 2024
4dc9d3e
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 27, 2024
4e892a2
extended schemas with missing properties
bazarnov Feb 27, 2024
fd32488
updated expected records, after schema changes
bazarnov Feb 27, 2024
6d6b48c
refactored old unit_tests up to the changes
bazarnov Feb 27, 2024
055f94b
added freezegun to dependencies
bazarnov Feb 28, 2024
0498bbb
added integration tests for CAT missing streams
bazarnov Feb 28, 2024
e4ae69b
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 28, 2024
e2f0fef
updated version
bazarnov Feb 28, 2024
f13df0c
renamed the test folder
bazarnov Feb 28, 2024
e1327cf
formatted
bazarnov Feb 28, 2024
3672be3
updated tests
bazarnov Feb 28, 2024
88fce0b
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 28, 2024
d815a5c
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 28, 2024
19dd6c7
fixed abnormal state
bazarnov Feb 28, 2024
b3ae16c
formatted
bazarnov Feb 29, 2024
e21affc
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 29, 2024
a6d7c10
fixed qa checks and unit test issue
bazarnov Feb 29, 2024
c3f492f
added empty migration guide
bazarnov Feb 29, 2024
344f2ae
added dummy run test
bazarnov Feb 29, 2024
d5b6a9e
removed old python base
bazarnov Feb 29, 2024
57010bd
added header to the migration guide
bazarnov Feb 29, 2024
ac4da3d
removed junk
bazarnov Feb 29, 2024
a8d775f
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Feb 29, 2024
5790743
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 1, 2024
df0ecd4
Merge branch 'master' into baz/source/recharge/migrate-to-low-code
bazarnov Mar 4, 2024
cadd610
Merge branch 'master' into baz/source/recharge/migrate-to-low-code
bazarnov Mar 5, 2024
601e6c0
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 6, 2024
917ef2b
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 6, 2024
fc48386
Merge branch 'baz/source/recharge/migrate-to-low-code' of https://git…
bazarnov Mar 6, 2024
9a1f2bd
Merge branch 'master' into baz/source/recharge/migrate-to-low-code
bazarnov Mar 6, 2024
d4207ee
Merge branch 'master' into baz/source/recharge/migrate-to-low-code
bazarnov Mar 7, 2024
945b828
Merge branch 'master' into baz/source/recharge/migrate-to-low-code
bazarnov Mar 7, 2024
d24416e
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 7, 2024
d48f9bb
Merge branch 'master' into baz/source/recharge/migrate-to-low-code
bazarnov Mar 11, 2024
a9a0086
Merge branch 'baz/source/recharge/migrate-to-low-code' of https://git…
bazarnov Mar 11, 2024
fe3979b
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 11, 2024
eb731ea
changed the cdk:python > cdk:low-code tag in metadata.yaml
bazarnov Mar 11, 2024
972ea68
added sort_by and adjusted tests accordingly
bazarnov Mar 12, 2024
3e8fa0a
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 12, 2024
eae18a8
resolved conflicts
bazarnov Mar 12, 2024
c36257f
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 13, 2024
a5c1143
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 14, 2024
0af9d22
updated after review
bazarnov Mar 14, 2024
082019c
formatted
bazarnov Mar 14, 2024
316c936
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 18, 2024
d34893b
updated migration guides
bazarnov Mar 18, 2024
f1ee0d0
updated migration guides including OSS instructions
bazarnov Mar 18, 2024
9c59002
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 20, 2024
dfe3eb2
removed CustomComponents, moved to the latest CDK, removed breaking c…
bazarnov Mar 20, 2024
7b42e9d
corrected version bumps
bazarnov Mar 20, 2024
8bfb6c9
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 20, 2024
a1d74e0
fixed unit_tests
bazarnov Mar 20, 2024
04fc570
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 20, 2024
7aaf834
Merge remote-tracking branch 'origin/master' into baz/source/recharge…
bazarnov Mar 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,23 @@ acceptance_tests:
- config_path: secrets/config.json
empty_streams:
- name: collections
bypass_reason: "volatile data"
bypass_reason: "The stream is tested with `Integration Tests`, since no data is available"
- name: discounts
bypass_reason: "volatile data"
bypass_reason: "The stream is tested with `Integration Tests`, since no data is available"
- name: onetimes
bypass_reason: "no data from stream"
timeout_seconds: 7200
bypass_reason: "The stream is tested with `Integration Tests`, since no data is available"
expect_records:
path: "integration_tests/expected_records.jsonl"
exact_order: no
fail_on_extra_columns: false
- config_path: secrets/config_order_modern_api.json
empty_streams:
- name: collections
bypass_reason: "volatile data"
bypass_reason: "The stream is tested with `Integration Tests`, since no data is available"
- name: discounts
bypass_reason: "volatile data"
bypass_reason: "The stream is tested with `Integration Tests`, since no data is available"
- name: onetimes
bypass_reason: "no data from stream"
timeout_seconds: 7200
bypass_reason: "The stream is tested with `Integration Tests`, since no data is available"
expect_records:
path: "integration_tests/expected_records_orders_modern_api.jsonl"
exact_order: no
fail_on_extra_columns: false
connection:
tests:
- config_path: secrets/config.json
Expand All @@ -36,7 +30,7 @@ acceptance_tests:
discovery:
tests:
- backward_compatibility_tests_config:
disable_for_version: 1.1.2
disable_for_version: 1.1.5
config_path: secrets/config.json
full_refresh:
tests:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,49 +2,49 @@
{
"type": "STREAM",
"stream": {
"stream_state": { "updated_at": "2050-05-18T00:00:00" },
"stream_state": { "updated_at": "2050-05-18T00:00:00Z" },
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is look like breaking change as you change state format

"stream_descriptor": { "name": "addresses" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "updated_at": "2050-05-18T00:00:00" },
"stream_state": { "updated_at": "2050-05-18T00:00:00Z" },
"stream_descriptor": { "name": "charges" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "updated_at": "2050-05-18T00:00:00" },
"stream_state": { "updated_at": "2050-05-18T00:00:00Z" },
"stream_descriptor": { "name": "customers" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "updated_at": "2050-05-18T00:00:00" },
"stream_state": { "updated_at": "2050-05-18T00:00:00Z" },
"stream_descriptor": { "name": "discounts" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "updated_at": "2050-05-18T00:00:00" },
"stream_state": { "updated_at": "2050-05-18T00:00:00Z" },
"stream_descriptor": { "name": "onetimes" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "updated_at": "2050-05-18T00:00:00" },
"stream_state": { "updated_at": "2050-05-18T00:00:00Z" },
"stream_descriptor": { "name": "orders" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "updated_at": "2050-05-18T00:00:00" },
"stream_state": { "updated_at": "2050-05-18T00:00:00Z" },
bazarnov marked this conversation as resolved.
Show resolved Hide resolved
"stream_descriptor": { "name": "subscriptions" }
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,19 @@
"destination_sync_mode": "append",
"cursor_field": ["updated_at"]
},
{
"stream": {
"name": "onetimes",
"json_schema": {},
"supported_sync_modes": ["full_refresh", "incremental"],
"source_defined_cursor": true,
"default_cursor_field": ["updated_at"],
"source_defined_primary_key": [["id"]]
},
"sync_mode": "incremental",
"destination_sync_mode": "append",
"cursor_field": ["updated_at"]
},
{
"stream": {
"name": "discounts",
Expand Down Expand Up @@ -108,19 +121,6 @@
"sync_mode": "full_refresh",
"destination_sync_mode": "overwrite"
},
{
"stream": {
"name": "subscriptions",
"json_schema": {},
"supported_sync_modes": ["full_refresh", "incremental"],
"source_defined_cursor": true,
"default_cursor_field": ["updated_at"],
"source_defined_primary_key": [["id"]]
},
"sync_mode": "incremental",
"destination_sync_mode": "append",
"cursor_field": ["updated_at"]
},
{
"stream": {
"name": "metafields",
Expand Down

Large diffs are not rendered by default.

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions airbyte-integrations/connectors/source-recharge/metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ data:
connectorSubtype: api
connectorType: source
connectorBuildOptions:
baseImage: docker.io/airbyte/python-connector-base:1.1.0@sha256:bd98f6505c6764b1b5f99d3aedc23dfc9e9af631a62533f60eb32b1d3dbab20c
baseImage: docker.io/airbyte/python-connector-base:1.2.0@sha256:c22a9d97464b69d6ef01898edf3f8612dc11614f05a84984451dde195f337db9
definitionId: 45d2e135-2ede-49e1-939f-3e3ec357a65e
dockerImageTag: 1.1.6
dockerImageTag: 1.2.0
dockerRepository: airbyte/source-recharge
githubIssueLabel: source-recharge
icon: recharge.svg
Expand All @@ -26,7 +26,7 @@ data:
documentationUrl: https://docs.airbyte.com/integrations/sources/recharge
tags:
- language:python
- cdk:python
- cdk:low-code
ab_internal:
sl: 200
ql: 400
Expand Down
34 changes: 24 additions & 10 deletions airbyte-integrations/connectors/source-recharge/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = [ "poetry-core>=1.0.0",]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
version = "1.1.6"
version = "1.2.0"
name = "source-recharge"
description = "Source implementation for Recharge."
authors = [ "Airbyte <[email protected]>",]
Expand All @@ -17,7 +17,8 @@ include = "source_recharge"

[tool.poetry.dependencies]
python = "^3.9,<3.12"
airbyte-cdk = "==0.60.1"
airbyte-cdk = "^0"
freezegun = "^1.4.0"

[tool.poetry.scripts]
source-recharge = "source_recharge.run:run"
Expand Down
Loading
Loading