-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for executing Substrait plans #10
Comments
I plan to take a look at apache/datafusion#9299 upstream to get some more familiarity with how Datafusion serializes plans to substrait and what limitations it may have currently. |
In the current approach we're federating the logical plan. So, this would likely be done earlier then the error mentioned in that issue. It should be fairly simple to add Substrait-based federation to the FlightSQL example. There is fairly fleshed out import and export code for logical plans. My idea was to, in the short term, just have a boolean to determine if SQL or Substrait should be sent. The alternative/eventual goal could be to build out the Flight SQL introspection commands to automate that further. Adding support for federating (part of) physical plans is another avenue that could be explored in the future. |
…uery when executing (datafusion-contrib#10) * keep original qualified_field when wrap_projections * only mutate the table reference when sending plan to execute * formatting * rename method
The existing FlightSQLExecutor executes a sql string. We should add support for executing a serialized Substrait plan. In the longer term , it should be easier to support more advanced pushdown for engines which support consuming Substrait.
The text was updated successfully, but these errors were encountered: