Skip to content

Commit

Permalink
[fix](inverted index) Fix errors caused by enable_need_read_data_opt (a…
Browse files Browse the repository at this point in the history
…pache#42064)

## Proposed changes

In the current backend implementation, it is not correctly handle counts
when `is null` predicate exists, so we forbid the storage layer's count
currently.

for example:
```
select count(b) from test where b is null 
```

When apply the is null filter, the result array will fill the default
value of the column type and the count operator can not detect whether
the result is null, so the count operator compute the wrong result.
  • Loading branch information
csun5285 authored Oct 22, 2024
1 parent 50bc168 commit a17f3e2
Show file tree
Hide file tree
Showing 3 changed files with 173 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,8 @@ public List<Rule> buildRules() {
Set<Slot> aggSlots = funcs.stream()
.flatMap(f -> f.getInputSlots().stream())
.collect(Collectors.toSet());
return conjuncts.stream().allMatch(expr -> checkSlotInOrExpression(expr, aggSlots));
return conjuncts.stream().allMatch(expr -> checkSlotInOrExpression(expr, aggSlots)
&& checkIsNullExpr(expr, aggSlots));
})
.thenApply(ctx -> {
LogicalAggregate<LogicalFilter<LogicalOlapScan>> agg = ctx.root;
Expand Down Expand Up @@ -163,7 +164,8 @@ public List<Rule> buildRules() {
Set<Slot> aggSlots = funcs.stream()
.flatMap(f -> f.getInputSlots().stream())
.collect(Collectors.toSet());
return conjuncts.stream().allMatch(expr -> checkSlotInOrExpression(expr, aggSlots));
return conjuncts.stream().allMatch(expr -> checkSlotInOrExpression(expr, aggSlots)
&& checkIsNullExpr(expr, aggSlots));
})
.thenApply(ctx -> {
LogicalAggregate<LogicalProject<LogicalFilter<LogicalOlapScan>>> agg = ctx.root;
Expand Down Expand Up @@ -492,6 +494,22 @@ private boolean checkSlotInOrExpression(Expression expr, Set<Slot> aggSlots) {
return true;
}

private boolean checkIsNullExpr(Expression expr, Set<Slot> aggSlots) {
if (expr instanceof IsNull) {
Set<Slot> slots = expr.getInputSlots();
if (slots.stream().anyMatch(aggSlots::contains)) {
return false;
}
} else {
for (Expression child : expr.children()) {
if (!checkIsNullExpr(child, aggSlots)) {
return false;
}
}
}
return true;
}

private boolean isDupOrMowKeyTable(LogicalOlapScan logicalScan) {
if (logicalScan != null) {
KeysType keysType = logicalScan.getTable().getKeysType();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
-- This file is automatically generated. You should know what you did if you want to edit this
-- !sql --
0

149 changes: 149 additions & 0 deletions regression-test/suites/inverted_index_p0/test_index_rqg_bug8.groovy

Large diffs are not rendered by default.

0 comments on commit a17f3e2

Please sign in to comment.