You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I had an error when trying to open a parquet file:
Traceback (most recent call last):
File "/local/workplace/lib/python3.6/site-packages/lambda_handlers/parquet_test.py", line 57, in lambda_handler
for row in parquet.reader(fin):
File "/local/workplace/lib/python3.6/site-packages/parquet/init.py", line 470, in reader
dict_items)
File "/local/workplace/lib/python3.6/site-packages/parquet/init.py", line 340, in read_data_page
if schema_element.converted_type is not None else read_values
File "/local/workplace/lib/python3.6/site-packages/parquet/converted_types.py", line 68, in convert_column
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "/local/workplace/lib/python3.6/site-packages/parquet/converted_types.py", line 68, in
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "/local/workplace/lib/python3.6/site-packages/parquet/converted_types.py", line 42, in intbig
return int.from_bytes(data, 'big', signed=True)
TypeError: cannot convert 'int' object to bytes
The text was updated successfully, but these errors were encountered:
I got a similar error and determined it was choking on a column of type FIXED_LEN_BYTE_ARRAY. I was able to work around this by excluding the column.
Traceback (most recent call last):
File ".\parquetexport3.py", line 102, in
for row in parquet.DictReader(fileIn, columns=targetColumns):
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet_init_.py", line 415, in DictReader
for row in reader(file_obj, columns):
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet_init_.py", line 464, in reader
values = read_data_page(file_obj, schema_helper, page_header, cmd,
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet_init_.py", line 334, in read_data_page
read_values = convert_column(read_values, schema_element)
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet\converted_types.py", line 66, in convert_column
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet\converted_types.py", line 66, in
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet\converted_types.py", line 40, in intbig
return int.from_bytes(data, 'big', signed=True)
TypeError: cannot convert 'int' object to bytes
I had an error when trying to open a parquet file:
Traceback (most recent call last):
File "/local/workplace/lib/python3.6/site-packages/lambda_handlers/parquet_test.py", line 57, in lambda_handler
for row in parquet.reader(fin):
File "/local/workplace/lib/python3.6/site-packages/parquet/init.py", line 470, in reader
dict_items)
File "/local/workplace/lib/python3.6/site-packages/parquet/init.py", line 340, in read_data_page
if schema_element.converted_type is not None else read_values
File "/local/workplace/lib/python3.6/site-packages/parquet/converted_types.py", line 68, in convert_column
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "/local/workplace/lib/python3.6/site-packages/parquet/converted_types.py", line 68, in
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "/local/workplace/lib/python3.6/site-packages/parquet/converted_types.py", line 42, in intbig
return int.from_bytes(data, 'big', signed=True)
TypeError: cannot convert 'int' object to bytes
The text was updated successfully, but these errors were encountered: