TLS termination at UKS load balancer using multiple domains (PoC)

This is small experiment on how you could serve multiple secured websites using single Kubernetes load balancer service object. Example uses single dynamic certificate bundle to register multiple TLS domains using SAN, but it can be easily extended to support multiple bundles (manual or dynamic).
Cloudflare is used as DNS provider, but any Terraform DNS provider can be used same way.

Requirements

• UpCloud's managed Kubernetes (UKS) cluster running
• Cloudflare API key to modify domain(s) DNS records, defined in app_domains variable
• use terraform.tfvars.skell as template to define variables
• UpCloud credentials defined as environment variables UPCLOUD_USERNAME and UPCLOUD_PASSWORD

Concept

Terminating multiple TLS domains at load balancer in UKS using dynamic certificate bundle isn't totally straightforward thing, because domains DNS records needs to point to towards the load balancer's DNS name using e.g. CNAME record but that name is available only after load balancer is already created. So we need to create load balancer service, use DNS name of newly created service to update domains DNS records and after that apply new TLS config to load balancer.

Resources

kubernetes_namespace
Creates namespace for application.

kubernetes_deployment
Creates deployment that runs pods that can handle multiple domain names (e.g. virtual hosts).

kubernetes_service
Creates load balancer service with initial config. Initial config is applied using special annotations and further modifications to annotations is disabled so that autogenerated annotations are not lost.

kubernetes_annotations
Updates load balancer's config to use certificate bundle when it's available.

cloudflare_record
Updates domain CNAME record to point to LB's domain name.

upcloud_loadbalancer_dynamic_certificate_bundle
Creates dynamic certificate bundle for hosted domains.