How to set backend-protocol: "HTTPS"

[samstride]

Hi,

Thanks for maintaining this repo.

I am trying to use caddy ingress to expose the kubernetes dashboard.

With nginx, this annotation is needeed in the ingress:

nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"

Whats the equivalent for Caddy's ingress?

I am on GKE 1.22.8.

My ingress definition looks something like this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: k8s-ingress
  namespace: kubernetes-dashboard
  annotations:
    kubernetes.io/ingress.class: caddy
spec:
  rules:
    - host: k8s.mycompany.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kubernetes-dashboard
                port:
                  number: 443

Thanks.

[Embraser01]

Hi, it's not something possible for now, should be feasible though

The thing I don't know yet, is how to handle TLS verification.
We would have to fill these fields : https://caddyserver.com/docs/json/apps/http/servers/errors/routes/handle/reverse_proxy/transport/http/

[samstride]

@Embraser01 , thanks for clarifying.

Sorry, one more thing. When I install caddy via helm, is there any setting that can be used to select/set a load balancer ip.

The reason I ask is that on GKE, caddy defaulted to using the ephemeral IP of the cluster. I tried editing it to a reserved static IP but that did not work either.

But ya, apart from these 2 issues, it's been pretty awesome so far.

[Embraser01]

The reason I ask is that on GKE, caddy defaulted to using the ephemeral IP of the cluster. I tried editing it to a reserved static IP but that did not work either.

Do you mean the status Ingress resources have? If so, you can find the logic here https://github.com/caddyserver/ingress/blob/master/internal/k8s/pod.go

If you mean the possibility of specifying the loadBalancerIP field in the Service, we could improve the Helm chart to allow adding it https://github.com/caddyserver/ingress/blob/master/charts/caddy-ingress-controller/templates/loadbalancer.yaml

[samstride]

Hmmm, I don't think the IP status of Ephemeral is advertised to Kubernetes. It's more of a cloud provider thing I guess.

I used kubectl edit to manually change the IP to a static one that I reserved but for some reason it kept reverting back.

If we can figure out why manually setting it did not work and solve that first, I can then provide a PR to the template.

I'd also be glad to contribute to the docs around GKE setup too.

[Embraser01]

Looking back at this issue, HTTPS services should kinda work because the 443 port on the service should make caddy use https automatically. It's probably the certificate validity of the kubernetes-dashboard that is self-signed ?

I used kubectl edit to manually change the IP to a static one that I reserved but for some reason it kept reverting back.

If we can figure out why manually setting it did not work and solve that first, I can then provide a PR to the template.
It's weird that service specs were not persisted. Maybe you can't edit a LoadBalancer ips once created on GKE? Didn't find anything related to this in docs.
The caddy controller doesn't patch anything except ingress statuses (with the service endpoint) so it shouldn't impact this.

[samstride]

@Embraser01 , yes, from here

By default self-signed certificates are generated and stored in-memory. In case you would like to use your custom certificates follow the below steps, otherwise skip directly to the Dashboard deploy part.

With NGINX, The annotation nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" solves this.

The alternative is to create a secret using custom certs and modify the deployment, which will be good to document.

[Embraser01]

Ok, thanks, found the Nginx ingress controller config for that: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#backend-certificate-authentication

It will take some work to make it fully work but it should be fairly easy to make it work with verification disabled (default in nginx)

[samstride]

Awesome + Thanks. Keen to try it out when it is ready and also happy to contribute to some GKE specific docs.