-
Notifications
You must be signed in to change notification settings - Fork 300
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Replace Python gRPC with Rust #2328
[WIP] Replace Python gRPC with Rust #2328
Conversation
Signed-off-by: Austin Liu <[email protected]> [wip]refactor: add rust grpc client Signed-off-by: Austin Liu <[email protected]> [wip]refactor: add rust grpc client Signed-off-by: Austin Liu <[email protected]> [wip]refactor: add rust grpc client Signed-off-by: Austin Liu <[email protected]> [wip]refactor: add rust grpc client Signed-off-by: Austin Liu <[email protected]> [wip]refactor: add rust grpc client Signed-off-by: Austin Liu <[email protected]> [wip]refactor: add rust grpc client Signed-off-by: Austin Liu <[email protected]> [wip]refactor: add rust grpc client Signed-off-by: Austin Liu <[email protected]>
I'm a rookie both in Rust and gRPC, so please do not hesitate to give me any kind of advice. |
Signed-off-by: Austin Liu <[email protected]>
Signed-off-by: Austin Liu <[email protected]>
Thank you for looking into this! Here are my high level thoughts for implementing a
|
Signed-off-by: Austin Liu <[email protected]>
Signed-off-by: Austin Liu <[email protected]>
Thank you so much, Thomas @thomasjpfan . I will discuss with Kevin @pingsutw in two hours.
That makes sense to me. Sounds like my original approach. Back then, I didn't know that deprecating the current
For sure, this is the tricky one. I will definitely look into a customized macro approach. Perhaps It can overcome the Rust coherence rule, or it can't.
So, I should modify this buf/generate stuff to better generate Rust service code for future usage. The current version has no client code like AdminServiceClient. It's a more unified way to manage proto files in one single place. |
@@ -0,0 +1,15 @@ | |||
[package] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use buf to generate the flyteidl code? Something like flyteorg/flyte#5187
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @eapolinario , I think this can help! I've already tried but I'm unable to generate Rust code correctly on my local machine.
Since you can compile, there shouldn't be many problems in CI.
this is awesome but @vlad-ivanov-name seems to have a good grasp. Can you please help review and help us drive a best choices in designing. @austin362667 also I have a PR #2307 |
If there is a way to get buf to only generate service code, then I like that solution. Otherwise, I am okay with the current solution of using https://github.com/tokio-rs/prost.
To answer this question, I would use this example protobuf and try different techniques for exposing it into Python. I'll start with making sure the Person works by itself and then work in the other objects:
I think this Rust<->Python data exchange is the unknown technical part of this work. I know removing our dependency on the Python |
I think figuring out the Rust<->Python data exchange and writing a Rust's
I feel like 2. is the unknown part right now, so I am pushing to figuring that out first. |
I think something's up with github as it first duplicated my review, and after I hid one duplicate the second disappeared 🙈 sorry about that, you can expand the comment above |
I think @eapolinario resolved this issue with PR flyteorg/flyte#5187.
I totally agree with this objective.
Kafka PMC @chia7712 is interested in the purpose of these refactorings. In his experience, projects with multiple languages often struggle, making consistency maintenance difficult. For example, Integrating Java with C/C++ can cause memory leaks, and Java with Scala may lead to binary compatibility issues. This complexity extends to build logic, requiring various tools and additional manpower for maintenance.
From what I know, this With respect to the Rust coherence rule, be careful when using external crates like For my understanding, we now have two very different solutions:
In the earlier sync with Kevin @pingsutw, he raised a concern: Can we really get rid of all IDL objects in Python? If we remove the dependency on |
Thank you! @vlad-ivanov-name , these suggestions are super helpful. We also need some advice to help us make the best choices in designing that @thomasjpfan and I just discussed. |
Thank you for the info! That is good to know.
Yup, I agree that are the two paths.
The overall size is a minor benefit. For me, getting rid of the
On a technical level going through Bytes and using the Python IDL + protobuf works. Without removing the Pros
Cons
I can get behind a Rust FlyteRemote if we can point to user facing issues around using Python's |
Thanks for @austin362667 to share the issue with me (and ping me here) I do enjoy learning the tech design so I talked with Austin for the issue. However, my comments may be produced without enough context and details. And so please feel free to correct my concerns as that is good for my education :) |
Sorry for the confusion, my fault for misleading you. By 'size of impact', I mean how much will be affect by removing Python IDL dependencies. But yeah, I get your points. Thanks for the thoughtful elaborations.
Mind letting me add the above analysis to the design doc? I've invited you as an editor, too. |
@chia7712 It's really nice of you sharing your adept insights with us here. |
Yup, you can add anything of my comments into your design doc. |
@thomasjpfan / @austin362667 / @eapolinario should we just make this a common FlyteRustClient? like in flyteidl and then create the binding layer only in flytekit? This way we will be pure rust in flyteidl that can be imported as a crate? I agree with @thomasjpfan I would love the entire grpc, protobuf dependencies to be rust only. and then we figure out how to make it work with python separately. This way the rust core client can be used in multiple places. |
@austin362667 I can't say I'm very familiar with flyte internals - only with some basics - but if you'd like to discuss any of more general rust-related technical questions I would be open to have a call |
I agree with having a rust crate with just a
XREF: Having a "rust core" for a Python library is a fairly common pattern: https://github.com/pydantic/pydantic-core |
Signed-off-by: Austin Liu <[email protected]>
Signed-off-by: Austin Liu <[email protected]>
Signed-off-by: Austin Liu <[email protected]> cleanup Signed-off-by: Austin Liu <[email protected]> cleanup Signed-off-by: Austin Liu <[email protected]> cleanup Signed-off-by: Austin Liu <[email protected]>
63cfba0
to
b132c89
Compare
Signed-off-by: Austin Liu <[email protected]>
Should we add redundant files from Most of them require The only way to check that if we get rid of Or somehow can we achieve this via lazy installation in Python? |
Signed-off-by: Austin Liu <[email protected]>
23d2c71
to
d704fcb
Compare
Signed-off-by: Austin Liu <[email protected]>
Signed-off-by: Austin Liu <[email protected]>
Signed-off-by: Austin Liu <[email protected]>
Signed-off-by: Austin Liu <[email protected]> refine testing Signed-off-by: Austin Liu <[email protected]>
a2a0cbd
to
9f20d3f
Compare
Signed-off-by: Austin Liu <[email protected]>
Signed-off-by: Austin Liu <[email protected]>
Tracking issue
flyteorg/flyte#5344
Users continuously face weird grpcio bugs, such as version errors, or even being unable to install grpcio.
Why are the changes needed?
To remove the dependency on gRPC in Python and solely replace it with Rust.
What changes were proposed in this pull request?
remote.py
.abi3-py38
Architecture
How was this patch tested?
Right now, only
get_task()
has been tested for the POC (proof of concept) purpose.There is still a lot of work to be done afterwards.
Setup process
Screenshots
The performance of FlyteKit slightly benefits from Rust, which remains super amazing.
Check all the applicable boxes
Related PRs
#2307
Docs link