You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When the s3 partition is blunt, athena's query take long time.
In my environment, the s3 sink connector encode partition by FieldPartitioner.
As a result, one of s3 partition has so many objects.
and it cause long time query when using athena.
if s3 sink connector can configure more than one partitioner(like FieldPartitioner + DailyPartitioner), the s3 objects are encoded in detail.
So more than one partitioning is needed. plz support this 🙏
The text was updated successfully, but these errors were encountered:
Hi All :)
TL;DR
When the s3 partition is blunt, athena's query take long time.
In my environment, the s3 sink connector encode partition by FieldPartitioner.
As a result, one of s3 partition has so many objects.
and it cause long time query when using athena.
if s3 sink connector can configure more than one partitioner(like FieldPartitioner + DailyPartitioner), the s3 objects are encoded in detail.
So more than one partitioning is needed. plz support this 🙏
The text was updated successfully, but these errors were encountered: