VirtualServer configuration of client_body_timeout

[tstraley]

Currently, there is no "native" way in NIC to configure the client_body_timeout directive.

When using NGINX for a gRPC request stream, or bi-directional (request and response stream), it is extremely common to need to set the grpc_send_timeout, grpc_read_timeout, and client_body_timeout directives higher than the default 60s in order to ensure the infrequent grpc keepalives and occasional traffic flow do not cause NGINX to close the connection with a 408 (Request Time-out) prematurely.

This is something that has been brought up and discussed in a variety of forums, including gitlab issues, stackoverflow, and even the community nginx-ingress controller documents these expectations: https://kubernetes.github.io/ingress-nginx/examples/grpc/#notes-on-using-responserequest-streams

Note that some discussions also brought up client_header_timeout and grpc_socket_keepalive directives as well, and while they may be related and may be required for some of these gRPC streaming situations, my particular issue only requires setting a higher client_body_timeout

NIC supports defining the grpc_send_timeout and grpc_read_timeout today, as both a global default configuration, as well as per upstream. Unfortunately, client_body_timeout can only be set using one of the snippets solutions. Would be great to have that configuration also available in the Upstream definition, and ideally also a global default in the global configmap configuration.

Thanks

[brianehlert]

I have been doing a bit of digging into this and I realize you are setting this value today. But I am wanting to double check that I am clear.

client_body_timeout appears to be specific to the http_core_module and happens on the client side of the request workflow.
https://nginx.org/en/docs/http/ngx_http_core_module.html
I don't see an equivalent for the http_grpc_module is that correct to your understanding?

Now we get into the question of where.
Is this fine at the global level in the configMap like the other settings you mention?
https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/configmap-resource/#general-customization

[brianehlert]

Really need more information to be able to give this any type of attention.
Can you please share your workaround? Especially if it meets your requirements.

[tstraley]

client_body_timeout appears to be specific to the http_core_module and happens on the client side of the request workflow
I don't see an equivalent for the http_grpc_module is that correct to your understanding?

Based on what I've found, this directive impacts HTTP and HTTP2/gRPC. Yes, it is part of client-side request phase.

Is this fine at the global level in the configMap like the other settings you mention?

That would be good, but as suggested, it would also be important to configure this per Upstream (just like those other timeout directives) because a user will very likely will want different configuration for a gRPC upstream vs an HTTP upstream.

Can you please share your workaround?

As stated, I use snippets. Specifically, for this case, I add a location-snippets: "client_body_timeout 10m;" in the specific Route resources that pass gRPC traffic to my gRPC upstream.

[brianehlert]

Okay.
So, from an nginx.conf perspective, this is location scoped.
Got it.
That should support separation concerns adequately and follows NGINX convention.