Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gRPC xDS client stops working after a period of inactivity #11

Open
rdesgroppes opened this issue May 15, 2024 · 4 comments
Open

gRPC xDS client stops working after a period of inactivity #11

rdesgroppes opened this issue May 15, 2024 · 4 comments

Comments

@rdesgroppes
Copy link

rdesgroppes commented May 15, 2024

We've been using your xDS Server for gRPC on Kubernetes, which suits our needs and is easy to use.
However, we are experiencing problems when the channels switch to idle mode. Disabling idleness helps, but we'd now like to restore the default behavior.

I'm far from being an expert on the matter, that's why I'm kindly requesting your point of view on the issue that was originally filed against the gRPC-Java client: grpc/grpc-java#9632.

According to grpc/grpc-java#9632 (comment):

[...] This seems like a go-control-plane bug to me. Once grpc-java sends a request with the removed resources, go-control-plane needs to honor that; it should not need any subsequent ACK/NACK. [...]

Is the xDS Server for gRPC on Kubernetes behaving like the aforementioned go-control-plane?

@whs
Copy link
Member

whs commented May 16, 2024

Internally this software use go control plane, so any bugs by them will have to be fixed upstream.

Not sure if related but we do find idle bugs in grpc go, and currently our workaround is to pin grpc at a specific version that do not have revamped idle behavior

@rdesgroppes
Copy link
Author

rdesgroppes commented May 16, 2024

Not sure if related but we do find idle bugs in grpc go, and currently our workaround is to pin grpc at a specific version that do not have revamped idle behavior

Right, idleness is enabled by default since gRPC-Go 1.59.0. We had to create channels with an additional option (grpc.WithIdleTimeout(0)) to get around it.

@hanikesn
Copy link

hanikesn commented Aug 7, 2024

We're facing a very similar issue using grpc v1.65.0, where channels who resume from idle mode won't resolve endpoints via xDS correctly.

Here's the error so this issue is easier to find:

rpc error: code = Unavailable desc = name resolver error: produced zero addresses

I'm wondering whether that's something worth reporting upstream.

@hanikesn
Copy link

hanikesn commented Aug 7, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants