Blob data corrupted after migration

[akinsgre]

I'm moving Blob data from MySQL 5.6.10 to Postgresql 9.3.1.0 using Python 2.7.5

The MySQL data contains PDF files as Blobs. After the migration, the files are corrupted.

I haven't started doing any debugging of the py-mysql2pgsql scripts, other than just checking that the data is good in MySQL and bad in Postgres immediately after doing the conversion.

But I was hoping this was something other people might have run into.

[akinsgre]

Checking the md5 values are different in mysql and postgres after the migration

[kworr]

I see this, raw data in mysql database is converted to hexadecimal representation in postrgres like it would be strings. The data in postgres can be recovered to raw with convert_from(, 'LATIN1').

Correct me if I'm wrong but it's impossible to work with raw bytea without escaping data in postgres?

[oyvindberg]

This is probably what I'm seeing as well, and might be bad if it applies to everybodys binary data. I have just started to debug it, but what I've found so far is that there seems to be done some erroneous escaping of binary data.

Given a value x'27' in a blob column in mysql, after import to postgresql the value will be x'2727'. 0x27 is ascii ', so I'm quite sure there must be some escaping going on. The rest of the data seems... fine.

[oyvindberg]

i patched it and fixed my immediate problem, but I have a hard time seeing the possible consequences (postgres_writer.py):
elif isinstance(row[index], (str, unicode, basestring)):
if column_type == 'bytea':
quoted = Binary(row[index]).getquoted()[1:-8].replace("''", "'")
row[index] = quoted if row[index] else row[index]