Prep for 0.1.0rc2

[robtandy]

This PR is long but it does not affect the core functionality of DataFusion for Ray, and does not differ from 0.1.0rc1 which has been extensively used by me in benchmarking from test.pypi.

This PR, when merged should be tagged 0.1.0rc2 and I can push the artifacts created by CI to test.pypi. Upon successful tests, these same artifacts can be used for the actual 0.1.0 release.

Also, fixes #85

This PR squashed merged the work that represented building out a repeatable k8s benchmarking set up. It is far from perfect but it has enabled significant benchmarking that has resulted in intuition around execution, latency, and price trade offs that I think will help us communicate about the project. The scripts and instructions are written to start from a fresh ubuntu 24.04 ami on amazon, and will work on arm or x86_64 architectures.

Summary of changes

• k8s benchmarking scripts and instructions - the bulk of the PR
• format python code
• remove the last mentions of worker and replace with processor where appropriate to indicate DFRayProcessors
• adding benchmark results in doc/benchmarks

[robtandy]

@andygrove Here is the PR I mentioned to you that I would submit with benchmarking code and results.

I have some good graphs of the results, but i'll submit them in a subsequent PR post the 0.1.0rc2 tagging (that I request here)

[andygrove]

This PR, when merged should be tagged 0.1.0rc2 and I can push the artifacts created by CI to test.pypi. Upon successful tests, these same artifacts can be used for the actual 0.1.0 release.

We'll also need the formal vote before making an official release.

[andygrove]

Thanks @robtandy!