There is a pending PR possibly related to this topic #7215

@zml1206 Would we still be able to optimize against such issues after that PR gets landed?

Should be. Let me do a quick test once @zml1206 resolve conflicts

Conflicts resolved. But I think it can not be resoved by #7215, it need rewrite datafilters of FileSourceScanExec. After this pr, I can add a Rewrite rule to rewrite datafilters to resolve it. @FelixYBW @zhztheplayer

Can we determine whether expression supports offload? @zhztheplayer There are three layers of filters here. The first layer is pushedDownFilters of FileSourceScanExec, which is also used by vanilla spark scan. The second layer is the dataFilters of FileSourceScanExec, which is used to generate the pushedDownFilters of FileSourceScanExec, as well as the filters used after the current offload. The third layer is the condition of the filter. In theory, the filter condition contains dataFilters, and dataFilters contains pushedDownFilters.
The current implementation will push down dataFilters or condition filters. Simply rewrite dataFilters to pushedDownFilters may cause only pushedDownFilters can be pushed down, but dataFilters to be pushed down before.

@zml1206 Thanks for explanation.

Can we determine whether expression supports offload?

Let's see if #6754 on which @rui-mo is working could finally help here.