multiple partitions for hive not delimited correctly #234
Labels
bug
Something isn't working
hive
Improvements to or issues with Hive functionality
needs triage
Awaiting triage by a dask-sql maintainer
To get a list of hive partitions the command
SHOW PARTITIONS
is used:dask-sql/dask_sql/input_utils/hive.py
Lines 265 to 277 in ece7ec7
For tables with multiple partition keys, it will return a list that is
/
delimited. For example, if the table has two partition keys,date
andregion
, one of the entries returned might be like this:date=20210101/region=south
This string is used without modifications to get additional information about each partition using
DESCRIBE FORMATTED {table_name} PARTITION ({partition})
. If multiple partition keys exist when using this command, the list should be,
separated, not/
separated.dask-sql/dask_sql/input_utils/hive.py
Lines 200 to 203 in ece7ec7
Because of this, when I try to read in a table with multiple hive partition keys dask-sql throws an error.
The issue #179 might also be somewhat related to the example above.
The text was updated successfully, but these errors were encountered: