[destination-postgres] java.lang.RuntimeException: Unable to deserialize PartialAirbyteMessage. #41991
Comments
misza80
added
area/connectors
|
@misza80, is this a new sync, or did you upgrade the platform and connectors? |
marcosmarxm
added
connectors/destination/postgres
team/destinations
this is new sync. only started with Airbyte few weeks back. Unless the connectors got upgraded at some point, not sure on this, I could have hit upgrade all on the connectors page, cant say if the MSSQL or Postgres were upgraded at that time. |
|
@misza80 is there a successful sync previously? Can you share the logs for that? Also update the complete log file. |
|
@marcosmarxm I was not able to perform a successful sync. please see attached. Thank you for looking into this. airbyteconnector.mp4 |
|
@airbytehq/destinations can someone take a look into this issue? Thanks! |
|
What we believe is happening here is that your source is sending a record that is either huge (>20mb) or has some unexpected break characters (e.g. lots of If you are running airbyte via docker-compose, can you please add the following environment variable |
|
reproduced using |
|
@ogirardot if you could share the content (e.g. a sqldump) of the data that produced this error in your source, that would be very helpful |
|
I've tried to narrow the possibilities (columns and lines) |
|
considering the code for serde PartialAirbyteMessage has changed quite a bit in the last "hours/week" Would it be possible to release a new version of the destination/postgres @evantahler to get a more recent stacktrace (or check if it wouldn't be fixed) ? |
|
Diving into existing issues it seems to be a common problem related to the future support_refreshes feature c.f. #39473 , #38641 and #37621 I tried to build a custom destination-postgres and deploy, bumping the platform version to 0.63.8 (helm chart 0.293.4) and bumping the cdk version for destination-postgres : but as the connector does not support refreshes the deployment fails with a issue like this one #40606 and the fix to set support_refreshes to true makes the connector fail with a missing _airbyte_generation_id field... So I guess the retro-compatibility got broken at some point developing the support for refreshes ? If I take into account OP's original issue and my own the specs are :
do we need to align those ? |
|
The cdk versions for the convectors shouldn’t have any impact on
compatibility. The only compatibility error we’re aware off WRT refreshes
feature is between the platform and the destination (you need to upgrade
the platform to 0.63 before you upgrade your destination to a version that
supports refreshes.
There’s a new destination-postgres coming very soon that introduces support
for refreshes
…---
Stephane Geneix
Staff Engineer
GitHub <https://github.com/airbytehq/airbyte> | Twitter
<https://twitter.com/AirbyteHQ> | LinkedIn
<https://www.linkedin.com/company/airbytehq/>
We're hiring, come work with me! <https://airbyte.io/careers> [image: 🚀]
On Sun, Jul 21, 2024 at 11:59 PM Olivier Girardot ***@***.***> wrote:
Diving into existing issues it seems to be a common problem related to the
future support_refreshes feature c.f. #39473
<#39473> , #38641
<#38641> and #37621
<#37621>
I tried to build a custom destination-postgres and deploy, bumping the
platform version to 0.63.8 (helm chart 0.293.4) and bumping the cdk version
for destination-postgres :
- cdkVersionRequired = '0.35.15'
+ cdkVersionRequired = '0.41.2'
but as the connector does not support refreshes the deployment fails with
a issue like this one #40606
<#40606> and the fix to
set support_refreshes to true makes the connector fail with a missing
_airbyte_generation_id field...
So I guess the retro-compatibility got broken at some point developing the
support for refreshes ?
Do we need to have sources and destination using the same
cdkVersionRequired ?
If I take into account OP's original issue and my own the specs are :
- destination/postgres 2.0.15 : cdkVersionRequired = '0.35.15'
- source/mssql 4.0.30 : cdkVersionRequired = '0.40.1'
- source/postgres 3.6.2 : cdkVersionRequired = '0.42.2'
do we need to align those ?
—
Reply to this email directly, view it on GitHub
<#41991 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/BDDFPOD2HGYJXTEXNP77IKLZNSUVRAVCNFSM6AAAAABK6ZYA22VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBSGIZDQMBSHE>
.
You are receiving this because you are on a team that was mentioned.Message
ID: ***@***.***>
|
|
Hello @stephane-airbyte I've seen the support for refreshes come and go, sorry to ask so bluntly, but should we wait ? is it still under development ? |
|
@evantahler Apologies for the late reply, I was away on an annual leave. I have isolated the stream that was causing the error and set the reporting as per your request. Unfortunately each time I want to retrieve the logs I receive and error. |
|
@misza80 are you able to see the logs from your airbyte server when that error occurs and get a stack trace? |
|
@ogirardot no need to apologize, I understand the support for refreshes in postgres was pulled without warning. We found critical issues that warranted a rollback, and we're still very much working on releasing the feature for good |
|
With #42540, destination-postgres version 2.3.0 is out, which should fix this bug, and re-enable support for refreshes. |
|
after updating connectors to the latest version getting the following error.
|
|
@misza80 it would be immensely helpful if you could provide an example record that causes this crash. Can you enable your streams 1-by-1 to find the offending content? Can you run your airbyte platform with the environment variable |
|
@evantahler Is there any other way I can export logs for the given job, beside the GUI? |
|
It depends on how you are running airbyte. If you are using docker compose, the job logs will be the summation of the stdout of the source, destination, and orchestrator containers during the sync - you could grab them from docker, or in the airbyte peristent volume. If you run syncs with abctl, there should be a log volume... and if you deploy with K8s, you should have an S3/GCS bucket which holds your sync logs. A reminder - we care less about your sync logs, and more about the actual record content in the source that is causing the crash. e.g. can you PGDUMP the table (or part of it) that is causing the problem? |
|
@evantahler Please find attached two entries from the DB I'm trying to sync.File:export.csv |
|
@misza80 it looks like you've got some rather large records, but not that large (e.g. under the 50mb parsing limit we have set). I'm curious if there are some line breaks or other "secret" break characters in that content that aren't serializing properly. |
Connector Name
destination-postgres
Connector Version
v2.0.12
What step the error happened?
During the sync
Relevant information
Source: airbyte/source-mssql:v4.0.30
Destination: airbyte/destination-postgres:2.0.12
Platform: 0.63.1
Issue:
When attempting to ingest all fields, the process fails with the following error:
Relevant log output
Contribute
The text was updated successfully, but these errors were encountered: