ConnectionMode.MEMORY_STRICTLY Should Use StreamQueryResult If Use Page Query

[xuejmnet]

ConnectionMode.MEMORY_STRICTLY == connectionMode ? new JDBCStreamQueryResult(resultSet) : new JDBCMemoryQueryResult(resultSet)

is one node sql query result,but if this group has more than one query parallel，sub node parallel should use stream query result,if use memory query result in page query, limit 10,10 will rewrite limit 0,20 if parallel query result use memory result not stream query result,memory will has more than scan rows.for example: sql.skip(10).take(10) in one data base 2 table will route, if ConnectionMode is CONNECTION_STRICTLY sharding sphere will use MemoryQueryResult, memory has 40 total rows,

public abstract class JDBCExecutorCallback<T> implements ExecutorCallback<JDBCExecutionUnit, T> {
    
    
    @Override
    public final Collection<T> execute(final Collection<JDBCExecutionUnit> executionUnits, final boolean isTrunkThread, final Map<String, Object> dataMap) throws SQLException {
        // TODO It is better to judge whether need sane result before execute, can avoid exception thrown
        Collection<T> result = new LinkedList<>();
        for (JDBCExecutionUnit each : executionUnits) {
           //once query memory add 20 rows
           //once query memory add 20 rows
           //once query memory add 20 rows
            T executeResult = execute(each, isTrunkThread, dataMap);
            if (null != executeResult) {
                result.add(executeResult);
            }
        }
//once query should return stream ,memory merge in foreach end
//and should input last memory result
// first only stream merge array do merge
//second and then should last memory result + current stream merge array do merge
        if(connectionMode==ConnectionMode.CONNECTION_STRICTLY ){
            (lastMemoryResult+result).merge()
        }
        return result;
    }
    }

so ConnectionMode is CONNECTION_STRICTLY still use StreamResult,but with other connection parallel result should combine

[xuejmnet]

now

optimize

[xuejmnet]

select * from order limit 10,10 ,now has 20rows for every node,memory is large

[strongduanmu]

Hi @xuejmnet I like this optimization, it seems to be effective in reducing memory usage.

But this optimization may require a lot of code adjustments, because at present, the results of all nodes are queried first, and then merged. To achieve this optimization, execution and merging need to be processed together.

[xuejmnet]

I'm looking forward to ShardingSphere better :)

[strongduanmu]

I'm looking forward to ShardingSphere better :)

@xuejmnet Thank you. We will try our best to optimize performance, and you are welcome to participate in the community.

[github-actions[bot]]

Hello , this issue has not received a reply for several days.
This issue is supposed to be closed.

[github-actions[bot]]

There hasn't been any activity on this issue recently, and in order to prioritize active issues, it will be marked as stale.