Streaming support. #146
Merged
Streaming support. #146
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wllenyj
force-pushed
the
stream-1
branch
4 times, most recently
from
a728401 to
bda9829
Compare
|
@wllenyj Awesome! Thanks a ton for introducing this so important feature. @jodh-intel @mxpv @lifupan @liubin Please take a look at this. |
|
@dmcgowan Any time to help with this? Thanks a lot. |
|
@wllenyj thanks! I'll take a look |
Tim-Zhang
reviewed
Jun 9, 2022
Tim-Zhang
reviewed
Jun 9, 2022
Tim-Zhang
reviewed
Jun 9, 2022
wllenyj
force-pushed
the
stream-1
branch
2 times, most recently
from
84081f0 to
d216133
Compare
quanweiZhou
reviewed
Jun 12, 2022
| &self, | ||
| req: Request, | ||
| streaming_client: bool, | ||
| streaming_server: bool, |
There was a problem hiding this comment.
I think streaming_client: bool and streaming_server: bool are a bit confusing, it's more clear to define them as sendable: bool and receivable: bool, while in this case only 3 conditions sender stream, receiver stream, and duplex stream, maybe an enum is better.
There was a problem hiding this comment.
Try to keep this commit consistent with the golang version. Optimization will be done later.
wllenyj
commented
Jun 13, 2022
Codecov Report
@@ Coverage Diff @@
## master #146 +/- ##
===========================================
+ Coverage 11.68% 26.02% +14.34%
===========================================
Files 13 16 +3
Lines 1549 2413 +864
===========================================
+ Hits 181 628 +447
- Misses 1368 1785 +417
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Tim-Zhang
reviewed
Jul 12, 2022
Tim-Zhang
reviewed
Aug 8, 2022
Tim-Zhang
reviewed
Aug 9, 2022
Move the MessageHeader from `common.rs` to the `proto.rs` file. Move the `compiled::ttrpc` from `common.rs` to `proto.rs` file and re-export. Signed-off-by: wllenyj <[email protected]>
Introduce Codec trait for protobuf message encoding/decoding. Signed-off-by: wllenyj <[email protected]>
Added more unit test for proto.rs. Signed-off-by: wllenyj <[email protected]>
It is used for server-side graceful shutdown. Signed-off-by: wanglei01 <[email protected]>
The first block uses the standard library. The second block uses the third party crates, and the third block uses this crate. Signed-off-by: wllenyj <[email protected]>
Using shutdown can be used instead of channel composition. Signed-off-by: wllenyj <[email protected]>
Make message encoding/decoding uniform. Signed-off-by: wllenyj <[email protected]>
For abstract connections, our connections are handled as one sending task and one receiving task. We can use the same logic to handle it. Signed-off-by: wllenyj <[email protected]>
The client and server handle connections almost identically. Both use a sender side and a receiver side to handle the connection. Their respective differences are implemented using the delegate. Signed-off-by: wllenyj <[email protected]>
Added protocol support for streaming. Added `StreamInner` struct for streaming operations and Streaming related errors. Signed-off-by: wllenyj <[email protected]>
Added streaming support for client-side. Signed-off-by: wanglei01 <[email protected]>
Added streaming support for server-side. Signed-off-by: wanglei01 <[email protected]>
Added streaming support for generator. Signed-off-by: wanglei01 <[email protected]>
This example is the same as the golang version. See `https://github.com/containerd/ttrpc/tree/main/integration/streaming` for details. Signed-off-by: wllenyj <[email protected]>
This is required by unicode-ident crate. For details: ``` error[L001]: failed to satisfy license requirements ┌─ unicode-ident 1.0.3 (registry+https://github.com/rust-lang/crates.io-index):4:13 │ 4 │ license = "(MIT OR Apache-2.0) AND Unicode-DFS-2016" │ -^^^----^^^^^^^^^^------^^^^^^^^^^^^^^^^ │ ││ │ │ │ ││ │ rejected: not explicitly allowed │ ││ accepted: license is explicitly allowed │ │accepted: license is explicitly allowed │ license expression retrieved via Cargo.toml `license` │ = unicode-ident v1.0.3 ├── proc-macro2 v1.0.43 │ ├── async-trait v0.1.57 │ │ └── ttrpc v0.6.1 │ ├── futures-macro v0.3.21 │ │ └── futures-util v0.3.21 │ │ ├── futures v0.3.21 │ │ │ ├── tokio-vsock v0.3.2 │ │ │ │ └── ttrpc v0.6.1 (*) │ │ │ └── ttrpc v0.6.1 (*) │ │ └── futures-executor v0.3.21 │ │ └── futures v0.3.21 (*) │ ├── quote v1.0.21 │ │ ├── async-trait v0.1.57 (*) │ │ ├── futures-macro v0.3.21 (*) │ │ ├── syn v1.0.99 │ │ │ ├── async-trait v0.1.57 (*) │ │ │ ├── futures-macro v0.3.21 (*) │ │ │ ├── thiserror-impl v1.0.32 │ │ │ │ └── thiserror v1.0.32 │ │ │ │ └── ttrpc v0.6.1 (*) │ │ │ └── tokio-macros v1.8.0 │ │ │ └── tokio v1.20.1 │ │ │ ├── tokio-vsock v0.3.2 (*) │ │ │ └── ttrpc v0.6.1 (*) │ │ ├── thiserror-impl v1.0.32 (*) │ │ └── tokio-macros v1.8.0 (*) │ ├── syn v1.0.99 (*) │ ├── thiserror-impl v1.0.32 (*) │ └── tokio-macros v1.8.0 (*) └── syn v1.0.99 (*) ``` Signed-off-by: wanglei01 <[email protected]>
quanweiZhou
approved these changes
Aug 10, 2022
Tim-Zhang
approved these changes
Aug 10, 2022
Tim-Zhang
added a commit
to Tim-Zhang/ttrpc-rust
that referenced
this pull request
Aug 10, 2022
Bump the major version due to pr containerd#146. Signed-off-by: Tim Zhang <[email protected]>
Merged
Tim-Zhang
added a commit
to Tim-Zhang/ttrpc-rust
that referenced
this pull request
Aug 12, 2022
Bump the major version of compiler to 0.6.0 due to containerd#146. Signed-off-by: Tim Zhang <[email protected]>
Tim-Zhang
added a commit
to Tim-Zhang/ttrpc-rust
that referenced
this pull request
Aug 12, 2022
Bump the major version of ttrpc-compiler to 0.6.0 due to containerd#146. Signed-off-by: Tim Zhang <[email protected]>
Tim-Zhang
added a commit
to Tim-Zhang/ttrpc-rust
that referenced
this pull request
Aug 12, 2022
Bump the major version of ttrpc-compiler to 0.6.0 due to containerd#146. Signed-off-by: Tim Zhang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduce support for streaming(https://github.com/containerd/ttrpc/blob/main/PROTOCOL.md#streaming). This is already supported by Golang's TTRPC in containerd/ttrpc#107. But currently only async is implemented. Related issues #144.
This pull request changes include:
shutdownmodule to handle graceful shutdown.Usually, streaming supports 3 types:
These 3 types are the same as GRPC, see https://grpc.io/docs/what-is-grpc/core-concepts/#server-streaming-rpc for details.
A general
Bidirectional streamingprocess is as follows:The
Connectionis an abstraction of a connection. One connection is handled as oneWriteasync-task and oneReadasync-task.Writetask receives the message from the channel and writes it to the connection.Readtask receives the message from the connection and writes it to the channel.