Python dataframe interface with Ibis

[lostmygithubaccount]

disclaimer: I work on Ibis

Ibis is a portable Python dataframe library that decouples the API from the execution engine. it already supports DataFusion, Apache Flink, and RisingWave (the latter two being the first dedicated streaming backends). it'd be cool to have Arroyo as well and would provide it with a well-supported Pythonic dataframe API

I couldn't quite tell but it looks like Arroyo probably uses the DataFusion SQL dialect? if so, a lot of the DataFusion decompiler could be reused. if anyone's interested in contributing this to Ibis we'd be happy to collaborate on it

[jacksonrnewhouse]

Yeah, loved the talk at Data Council and I'm planning on doing this. Pretty close to DataFusion, but we have a couple of language modifications.

[deepyaman]

@jacksonrnewhouse Glad to hear it! Yeah, I learned from @mwylde the other day that the dialect isn't exactly compatible with any existing one, but that's not a problem at all (or, more like, one of the main problems Ibis helps resolve :)).