You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
currently, blaze hangs on TPC-DS q40 case (1TB) because of a severely skewed join (1,463,474,467 join 5 records). spark supports an option skew=true in SMJ which blaze does not.
without this feature, we should not convert skewed SMJ to native. otherwise the joining is very slow or even hangs.
Describe the solution you'd like
understand how spark performs skew join optimization. implement the same logic in blaze.
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
currently, blaze hangs on TPC-DS q40 case (1TB) because of a severely skewed join (1,463,474,467 join 5 records). spark supports an option
skew=true
in SMJ which blaze does not.without this feature, we should not convert skewed SMJ to native. otherwise the joining is very slow or even hangs.
Describe the solution you'd like
understand how spark performs skew join optimization. implement the same logic in blaze.
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered: