[Merged by Bors] - Initial listener-operator implementation

[nightkr]

This implements the basic concept, but is sitll missing a fair bit of documentation. The CRDs should also be moved into operator-rs since some operators will depend on them.

Essentially, this provides two new CRDs and a CSI driver:

---
apiVersion: lb.stackable.tech/v1alpha1
kind: LoadBalancer
metadata:
  name: example-public-lb
  namespace: default
spec:
  className: public
  podSelector:
    statefulset.kubernetes.io/pod-name: example-public-pod
  ports:
  - name: http
    port: 80
    protocol: TCP
status:
  ingressAddresses:
  - address: 172.18.0.3
    ports:
      http: 80
---
apiVersion: lb.stackable.tech/v1alpha1
kind: LoadBalancerClass
metadata:
  name: public
spec:
  serviceType: LoadBalancer

LoadBalancerClass defines the policy for how a specific "kind of service" (for example: public with dynamic discovery, vpc-internal with static discovery) should be deployed on a given network. LoadBalancer then links deploys a Service (or potentially something else in the future, where relevant) according to these rules, and gives back (via the status) a list of addresses that can be used to access the service, including whatever port remappings may be required.

The CSI driver can also be used to project this address information into the pod itself:

---
apiVersion: v1
kind: Pod
metadata:
  name: example-public-pod
spec:
  volumes:
    - name: lb
      ephemeral:
        volumeClaimTemplate:
          metadata:
            annotations:
              lb.stackable.tech/lb-name: example-public-lb
          spec:
            storageClassName: lb.stackable.tech
            accessModes:
              - ReadWriteMany
            resources:
              requests:
                storage: "1"
  containers:
    - name: nginx
      image: nginx
      volumeMounts:
        - name: lb
          mountPath: /lb

The CSI driver can also be configured to automatically create a LB based on the Pod's settings:

---
apiVersion: v1
kind: Pod
metadata:
  name: example-public-pod
spec:
  volumes:
    - name: lb
      ephemeral:
        volumeClaimTemplate:
          metadata:
            annotations:
              lb.stackable.tech/lb-class: public
          spec:
            storageClassName: lb.stackable.tech
            accessModes:
              - ReadWriteMany
            resources:
              requests:
                storage: "1"
  containers:
    - name: nginx
      image: nginx
      volumeMounts:
        - name: lb
          mountPath: /lb
      ports:
        - name: http
          containerPort: 80

Finally, the CSI driver also configures stickiness when used with persistent PVCs (such as manually provisioned ones, or ones created via StatefulSet.spec.volumeClaimTemplates), if it would be useful for the given LoadBalancerClass.

[nightkr]

I've tried to document basic operation now. Would appreciate if someone could check that it more or less makes sense...

Not at all sold on the CRD naming, since we end up with LoadBalancer uses LoadBalancerClass with type: LoadBalancer, creates a Service with type: LoadBalancer. I talked to @fhennig about renaming it to L4Ingress/Layer4Ingress, but that also feels pretty awkward.

A few other considered but discarded options....

• Service - conflicts with K8s-native Service
• Ingress - conflicts with K8s-native HTTP-level Ingress
• Gateway - conflicts with both Istio's Gateway (basically an Ingress controller) and K8s' new Gateway API
• Router implies that we're configuring a layer 3 router
• Route/IngressRoute - Sounds more like a single-destination rule (which is a use-case that we support, but not the only one), Træfik also has their own IngressRoute CRD

[nightkr]

bors r+

[nightkr]

bors ping

[bors]

pong

[nightkr]

bors r+

[bors]

👎 Rejected by code reviews

Comments addressed, torch handed over to Sigi

[nightkr]

bors r+

[bors]

Pull request successfully merged into main.

Build succeeded:

• All tests passed

[lfrancke]

I'll need some help on this one.
Do we need to add this to the feature tracker?
It feels like the answer is "yes" but I struggle to come up with a concise name.
It can also be multiple features...

@teozkr @soenkeliebau

[nightkr]

Sort of... I'd say something like "enable configurable listeners" (not happy about this name either) is a feature worth tracking per product operator.

[lfrancke]

Especially for Kafka where listener has another but similar meaning...

[nightkr]

Myeah. "Stackable listeners are used to expose Kafka listeners", but I agree that they need to be distinguished in the feature tracker...

[soenkeliebau]

Maybe something along the lines of "Centrally managed access patterns/methods/..."

Listener is a more technical way of phrasing this I guess, what we actually mean is how stuff can be accessed, no?

[lfrancke]

I'll let it simmer for a few days. I'm not really happy yet but I'll pick something. Thanks for the input.

[soenkeliebau]

Hm... apparantly my comment from this morning never made it here..

I was thinking we don't need to add it to the feature tracker right now, as this PR didn't include the functionality in any operator, it only created the necessary scaffolding that we need in place for this.
So we can punt on the naming until the first operator PR comes around :)

[lfrancke]

Excellent. Let future us deal with that problem then. Happy to move it to done in that case