Dual TCP/UDP NLB shared across multiple services

[hongkongkiwi]

Hi there,

I would like to share a single NLB across two services with both UDP and TCP ports open.

For example:
serviceA - Port 550 UDP
serivceB - Port 8899 TCP

I couldn't seem to find a way to do this unless using an application load balancer and ingress routes.

Is there a way to do this in the v2.0.0 release?

The major blocker was that the targetgroup annotation was only supported at an ingress level (not a service level) so there just seems no way to share a LB.

[kishorj]

The service resource models closely to the AWS NLB resource, so we don't intend to support single LB across multiple service resources. See if specifying multiple service ports helps your use case.

If you are not able to specify both TCP and UDP ports on a service resource of type LoadBalancer, you can try using service of type NodePort. The only limitation currently is that the TCP and UDP ports cannot be the same value.

[hongkongkiwi]

Thanks @kishorj, that's a pretty clear explanation. I will try the NodePort approach.

I was wondering why I can't specify the targetgroup with NLB like I can with the ALB? That would also be a way to resolve this, just specify the same targetgroup for multiple services (such as I can do with ingress).

[philomory]

I also would love to see something akin to alb.ingress.kubernetes.io/group.name on NLBs (presumably as something akin to service.beta.kubernetes.io/aws-load-balancer-group-name).

NodePort seems like a nonstarter with an autoscaling cluster - you have to go manually create additional target groups and pin your autoscaling group to each of them every time you add a new service (though honestly this is more of a failing of the way NLB target groups work - ideally you should only need one target group, not one target group per port all with identical instance lists)

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[philomory]

As far as I'm aware this shouldn't be stale. I'd still love to see this, at least.

[M00nF1sh]

@philomory
This controller currently supports NodePort service as well. If you use the NLB-IP mode annotation service.beta.kubernetes.io/aws-load-balancer-type: "nlb-ip" on services, even it's a nodePort service, the controller will manage the loadbalancer and targetGroups for you automatically.

Also, there have been proposal in upstream to allow dual TCP/UDP.

[M00nF1sh]

/kind feature

[TBBle]

Mixed-protocol (TCP/UDP) Service is alpha in k8s 1.20, is this the feature ticket to track for support for the MixedProtocolLBService feature gate?

Or is this for multiple Services contributing to a single NLB, similar to #1707 and #1545?

[kishorj]

@TBBle, this is the correct issue for mixed protocol support.

Once the MixedProtocolLBService feature gate is enabled, service of type LoadBalancer with mixed protocols should work fine without further changes with the following limitations -

• cannot have same port for both UDP and TCP, for example TCP port 53 and UDP port 53. This is a limitation on the AWS side.
• NLB dual-stack currently doesn't support UDP

[philomory]

@TBBle What, exactly, is the limitation on the AWS side that causes this? Doesn't the NLB listener protocol TCP_UDP cover the case of the same port over both TCP and UDP?

[TBBle]

I assume you meant @kishorj with that question.

Per the docs, TCP_UDP is explicitly for the "same port in TCP and UDP" case.

To support both TCP and UDP on the same port, create a TCP_UDP listener. The target groups for a TCP_UDP listener must use the TCP_UDP protocol.