-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[destination-snowflake] java.lang.RuntimeException: Unable to deserialize PartialAirbyteMessage. #37621
Comments
Is this a new sync or it was workign before? |
When we tested the sync in the past with the config 👇, it worked.
Then we upgraded to:
Could something change on the connector? Or is it related to the message size? |
hello, I encountered the same error while syncing mysql to snowflake, the error message is Is there any workaround for this? can we get older version of the connector to try? Thanks, |
We're encountering the same issue with the following versions:
|
I raised this to the destination team. Hope to get an updated about what is causing this errors. |
We haven't tried because we migrated everything to the last version on the Platform, but we tried to downgrade the destination and are still having the same issue. |
Hello everyone! 👋 Please upgrade to the latest version of Destination Snowflake 3.8.0. This version uses the most recent Airbyte Destination CDK, which should resolve the issue you're currently facing. Let me know if the problem is solved after the upgrade. Thanks! |
Hey @marcosmarxm, We upgraded to:
The problem is still there: |
Can you also confirm that you are using the latest version of the Airbyte platform? we changed how we handle large records recently |
Hey, @evantahler, I can confirm that we upgraded to the last platform version, 0.59.1. I can provide more context and narrow down the issue. After reading your message, I found the column that was causing the issue (due to the large record). I don't know if it helps, but I can tell you that the conflicted column is a JSON type on Postgres.
|
https://github.com/airbytehq/airbyte-internal-issues/issues/7113 will make the error clearer |
Hey @evantahler, I think the link points to a private repository. |
The next version of Snowflake will solve this issue (as well as implement refreshes) - stay tuned. |
Hey @evantahler , I have been following this thread. Is it possible that the new release will also fix the issues as described in #38641 since it seems like both the issues might be related to inability to deserialise some message? |
Got this error when syncing postgres (3.3.27) -> BigQuery (2.4.12) on Airbyte 0.60.0.
|
@killthekitten this issue is for snowflake. The latest version of the bigquery connector (v2.8.0) has the fix for that connector. |
Closed by #39473 - v3.11.0 |
I am getting same error on some connectors, following are the version I am using, if someone could point me in the right direction to solve this will be appreciated. Airbyte version:
|
Connector Name
destination-snowflake
Connector Version
3.6.6
What step the error happened?
During the sync
Relevant information
Environment:
Source:
airbyte/source-postgres:3.3.27
Destination:
airbyte/destination-snowflake:3.6.6
(upgraded from 3.4.10)Platform:
0.56.0
(upgraded from 0.50.33)Issue:
When attempting to ingest all fields, the process fails with the following error:
Additional message
Observation:
The process succeeds when one of the large-sized fields is discarded. So, we assume the issue is with the column length, but it is an assumption. Has something on the snowflake connector been changed?
Request:
I'm looking for help obtaining more detailed logs for debugging purposes. To facilitate debugging, it would be helpful to identify the affected rows or similar information. The lack of detailed error messages makes it challenging to pinpoint the exact cause of failure.
Log information
attempt_1.txt
attempt_2.txt
attempt_3.txt
attempt_4.txt
attempt_5.txt
Relevant log output
No response
Contribute
The text was updated successfully, but these errors were encountered: