Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(pyspark): verify the existing backend-specific tests work in streaming mode #8888

Open
1 task done
mfatihaktas opened this issue Apr 4, 2024 · 0 comments
Open
1 task done
Assignees
Labels
pyspark The Apache PySpark backend tests Issues or PRs related to tests

Comments

@mfatihaktas
Copy link
Contributor

mfatihaktas commented Apr 4, 2024

Is your feature request related to a problem?

As described in the meta-issue #8868, our initial understanding is that Spark SQL has the same syntax for both batch and streaming. This is why, anything at the intersection of these two should just work.

Describe the solution you'd like

We need to verify our understanding (stated above) by testing the existing pyspark backend functionality in streaming mode. The first part of this is to verify the pyspark-specific tests under ibis/backends/pyspark/tests/ in streaming mode.

What version of ibis are you running?

8.0.0

What backend(s) are you using, if any?

pyspark

Code of Conduct

  • I agree to follow this project's Code of Conduct
@mfatihaktas mfatihaktas added tests Issues or PRs related to tests pyspark The Apache PySpark backend labels Apr 4, 2024
@mfatihaktas mfatihaktas self-assigned this Apr 4, 2024
@mfatihaktas mfatihaktas moved this from backlog to todo in Ibis planning and roadmap Apr 4, 2024
@mfatihaktas mfatihaktas modified the milestone: Q2 2024 Apr 4, 2024
@mfatihaktas mfatihaktas moved this from todo to cooking in Ibis planning and roadmap Apr 9, 2024
@mfatihaktas mfatihaktas changed the title test: verify that existing pyspark backend functionality works in streaming mode test(pyspark): verify that existing backend functionality works in streaming mode Apr 11, 2024
@mfatihaktas mfatihaktas changed the title test(pyspark): verify that existing backend functionality works in streaming mode test(pyspark): verify the existing backend-specific tests work in streaming mode Apr 11, 2024
@mfatihaktas mfatihaktas moved this from cooking to review in Ibis planning and roadmap Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pyspark The Apache PySpark backend tests Issues or PRs related to tests
Projects
Status: review
Development

No branches or pull requests

1 participant