Exposing Jaeger UI via Ingress when deploying via Helm

[ValeriiVozniuk]

Hi,
I'm testing linkerd with Jaeger deployment, and can't find the correct setup for Jaeger UI exposure via Ingress.
I'm deploying all the components with the following Helm commands (certificates are generated with a script beforehand):

helm upgrade --install linkerd-crds linkerd/linkerd-crds \
  --namespace=linkerd \
  --create-namespace
helm upgrade --install linkerd-cni linkerd/linkerd2-cni \
  --namespace=linkerd \
  --set destCNINetDir=/var/lib/rancher/k3s/agent/etc/cni/net.d \
  --set destCNIBinDir=/var/lib/rancher/k3s/data/current/bin
# Because of race condition with CNI
sleep 10
helm upgrade --install linkerd-control-plane linkerd/linkerd-control-plane \
  --namespace=linkerd \
  --set-file identityTrustAnchorsPEM=ca.crt \
  --set-file identity.issuer.tls.crtPEM=issuer.crt \
  --set-file identity.issuer.tls.keyPEM=issuer-private.pem \
  --set cniEnabled=true \
  --set prometheusUrl="http://client-prometheus.monitoring.svc:9090" \
  --set podMonitor.enabled=true
helm upgrade --install linkerd-viz linkerd/linkerd-viz \
  --namespace=linkerd-viz \
  --create-namespace \
  --set enforcedHostRegexp=linkerd-viz-my-cluster.domain.com \
  --set prometheusUrl="http://client-prometheus.monitoring.svc:9090" \
  --set jaegerUrl="http://jaeger.linkerd-jaeger.svc:16686" \
  --set prometheus.enabled=false
helm upgrade --install linkerd-jaeger linkerd/linkerd-jaeger \
  --namespace=linkerd-jaeger \
  --create-namespace

After that, I'm creating an ingress for linkerd-viz according to https://linkerd.io/2.15/tasks/exposing-dashboard/#nginx

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-realm: Authentication Required
    nginx.ingress.kubernetes.io/auth-secret: web-ingress-auth
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header Origin "";
      proxy_hide_header l5d-remote-ip;
      proxy_hide_header l5d-server-id;
    nginx.ingress.kubernetes.io/upstream-vhost: $service_name.$namespace.svc.cluster.local:8084
  name: ingress-route-linkerd-viz
  namespace: linkerd-viz
spec:
  ingressClassName: nginx
  rules:
  - host: linkerd-viz-my-cluster.domain.com
    http:
      paths:
      - backend:
          service:
            name: web
            port:
              number: 8084
        path: /
        pathType: Prefix
  tls:
  - secretName: domain.com

So far so good, I'm able to open linkerd-viz-my-cluster.domain.com, most of the stuff is working (tap starts working after all pods in linkerd-viz namespace restart, not sure why). Now I'm trying to access Jaeger UI. linkerd-viz expects it to be on /jaeger subpath of the same domain name. The first issue I've see here, is that Ingress is namespaced object, but Jaeger is deployed in a different namespace than linkerd-viz. Solution I've used: ExternalName service

apiVersion: v1
kind: Service
metadata:
  name: jaeger-external
  namespace: linkerd-viz
spec:
  externalName: jaeger.linkerd-jaeger
  sessionAffinity: None
  type: ExternalName

Now I can add a backed to an existing ingress

      - backend:
          service:
            name: jaeger-external
            port:
              number: 16686
        path: /jaeger
        pathType: Prefix

But here I've got 2 other issues:

1. If I visit https://linkerd-viz-my-cluster.domain.com/jaeger right after install, Jaeger UI loads, but no data is seen in it
   
   Trying to run it with linkerd jaeger dashboard from external node gives "Waiting for linkerd-jaeger to become available" message.
   Aslo, in linkerd UI it shows 0 metrics for Jaeger
   
2. When I do kubectl rollout restart deployment.apps -n linkerd-jaeger, pods are restarting, and now when trying to go to https://linkerd-viz-my-cluster.domain.com/jaeger I'm getting 403 error

[   154.884936s]  INFO ThreadId(01) inbound:server{port=16686}: linkerd_app_inbound::policy::http: Request denied server.group=policy.linkerd.io server.kind=server server.name=jaeger-ui route.group= route.kind=default route.name=default client.tls=None(NoClientHello) client.ip=10.42.1.19
[   154.885018s]  INFO ThreadId(01) inbound:server{port=16686}:rescue{client.addr=10.42.1.19:38118}: linkerd_app_core::errors::respond: HTTP/1.1 request failed error=client 10.42.1.19:38118: server: 10.42.0.50:16686: unauthorized request on route error.sources=[unauthorized request on route]

From the error above I assume that I need to create some Server policy(?) which would allow me to have access via Ingress, but I was not able to find any mentions/examples for it.
Does anyone knows, how I can fix these issues? :)

Additional note: after linkerd-jaeger namespace pods restart, linkerd jaeger dashboard starts working, exposing it on my local machine on http://localhost:16686 address, and traces are present there. Also in linkerd UI metrics are present for Jaeger.