Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Destinations: Refreshes: CDK updates #38067

Merged
merged 1 commit into from
Jun 17, 2024
Merged

Conversation

edgao
Copy link
Contributor

@edgao edgao commented May 8, 2024

closes https://github.com/airbytehq/airbyte-internal-issues/issues/7606

These are the changes that break platform compatibility. We shouldn't release this until platform is ready for it, and we can turn on the supportsRefreshes metadata in destinations.

Copy link

vercel bot commented May 8, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Jun 17, 2024 3:32pm

@octavia-squidington-iii octavia-squidington-iii added the CDK Connector Development Kit label May 8, 2024
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch 3 times, most recently from 9f8ae69 to 50c1e4b Compare May 8, 2024 18:40
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from 50c1e4b to e857ce6 Compare May 8, 2024 21:13
Copy link
Contributor Author

edgao commented May 8, 2024

DestinationSyncMode.APPEND_DEDUP,
primaryKey,
Optional.of(cursor),
COLUMNS
COLUMNS,
0,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should use nonzero generation id, so that tests are slightly more stringent?

DestinationSyncMode.OVERWRITE,
mock(),
mock(),
mock()
mock(),
0,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for a later stacked PR: I think we need to add generation-related logic into DefaultTyperDeduper? so we'll want more test cases in this class...

@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from e857ce6 to 982948e Compare May 8, 2024 21:21
import java.util.*
import kotlin.collections.LinkedHashMap

data class StreamConfig(
val id: StreamId,
val syncMode: SyncMode,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the source sync mode, which we should never look at. Delete it.

@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch 6 times, most recently from 6839b3f to d361d45 Compare May 8, 2024 22:00
@edgao edgao requested a review from gisripa May 8, 2024 23:00
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch 3 times, most recently from a3bb96c to eca8010 Compare May 10, 2024 17:10
@edgao edgao changed the base branch from master to edgao/catalog_parser_improvements May 10, 2024 17:10
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from 7aff2a5 to 2047aec Compare May 10, 2024 17:18
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from eca8010 to 7058132 Compare May 10, 2024 17:18
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from 2047aec to 2f4b6ac Compare May 10, 2024 17:33
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from 7058132 to 1306644 Compare May 10, 2024 17:33
@gisripa gisripa mentioned this pull request May 10, 2024
2 tasks
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from 9dd4a9d to ea03c9b Compare June 7, 2024 23:13
@edgao edgao force-pushed the edgao/bigquery_more_cdk_bump branch from eed66f8 to da3e829 Compare June 10, 2024 16:13
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from ea03c9b to f00bab9 Compare June 10, 2024 16:13
@edgao edgao force-pushed the edgao/bigquery_more_cdk_bump branch from da3e829 to 39971f7 Compare June 10, 2024 16:41
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from f00bab9 to 0850f44 Compare June 10, 2024 16:41
@edgao edgao force-pushed the edgao/bigquery_more_cdk_bump branch from 39971f7 to 292aa89 Compare June 10, 2024 17:02
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from 0850f44 to e7f351c Compare June 10, 2024 17:02
@edgao edgao force-pushed the edgao/bigquery_more_cdk_bump branch from 292aa89 to bc71af7 Compare June 10, 2024 17:03
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from e7f351c to 5f84378 Compare June 10, 2024 17:04
@octavia-squidington-iii octavia-squidington-iii removed the area/connectors Connector related issues label Jun 10, 2024
Base automatically changed from edgao/bigquery_more_cdk_bump to master June 10, 2024 18:35
@gisripa gisripa force-pushed the destinations_refreshes_cdk_changes branch from 5f84378 to 8234563 Compare June 11, 2024 00:03
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from 8234563 to 21ed84f Compare June 12, 2024 17:40
@gisripa gisripa force-pushed the destinations_refreshes_cdk_changes branch from 21ed84f to 12be697 Compare June 12, 2024 21:44
@gisripa gisripa force-pushed the destinations_refreshes_cdk_changes branch from 12be697 to 47f9fff Compare June 13, 2024 19:34
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from 47f9fff to b3e1c66 Compare June 13, 2024 21:41
Copy link
Contributor

@gisripa gisripa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting the actual platform version in the error message seems to be the only pending thing.

@edgao
Copy link
Contributor Author

edgao commented Jun 13, 2024

yep, I actually pushed that just now (platform released 0.63.0 with full refreshes support)

@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from b3e1c66 to 04d1404 Compare June 13, 2024 22:31
@gisripa gisripa force-pushed the destinations_refreshes_cdk_changes branch from 04d1404 to c43df8c Compare June 14, 2024 00:26
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from c43df8c to 04d1404 Compare June 14, 2024 15:52
@stephane-airbyte stephane-airbyte force-pushed the destinations_refreshes_cdk_changes branch from 04d1404 to b116855 Compare June 14, 2024 17:52
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from b116855 to b3d7092 Compare June 17, 2024 14:37
@edgao edgao force-pushed the destinations_refreshes_cdk_changes branch from b3d7092 to 10a1cbb Compare June 17, 2024 15:31
@edgao
Copy link
Contributor Author

edgao commented Jun 17, 2024

/publish-java-cdk

🕑 https://github.com/airbytehq/airbyte/actions/runs/9550522415
✅ Successfully published Java CDK version=0.39.0!

@edgao edgao merged commit 2432cc8 into master Jun 17, 2024
25 checks passed
@edgao edgao deleted the destinations_refreshes_cdk_changes branch June 17, 2024 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants