Lab 4.2 works, but for the wrong reason(s)

[DonaldKellett]

TL;DR: The web-frontend deployment in chapter 4 should call the ingress gateway instead of the customers service directly, both web-frontend and customers VirtualServices should be modified in terms of host matching, and the traffic weights of the v1/v2 subsets in the customers VirtualService should be made asymmetric (e.g. 80-20 instead of 50-50) to really drive the point home.

Original post forwarded from edX discussions (requires login authentication) quoted below:

Just realized that Lab 4.2 on weighted traffic routing works but for the wrong reason(s). I discovered this by attempting to adjust the traffic weights to 20/80 (or 80/20, etc.) and still getting a (roughly) 50/50 traffic split by observation.

Note: I discovered this using Istio Classic Gateway + VirtualService + DestinationRule, but with Alibaba Cloud's ASM managed service mesh instead of upstream Istio since I was having issues with the latter regarding the IngressGateway Envoy proxy not listening. Regardless, ASM is Istio-compatible so it shouldn't matter.

In Lab 4.2, we have a web-frontend deployment with a corresponding service, a customers-v1 deployment with 1 replica, a customers-v2 deployment with 1 replica and a customers service whose selectors match pods in both customers-v{1,2} deployments. The web-frontend deployment calls the customers service (FQDN customers.default.svc.cluster.local) via an environment variable. We then have a gateway that listens on TCP port 80 (i.e. HTTP) for all hosts '*', a VirtualService web-frontend which routes all hosts '*' to the web-frontend service, a DestinationRule that defines the v1 and v2 subsets for the customers service and a VirtualService customers which routes the host 'customers.default.svc.cluster.local' to the customers service with a 50/50 traffic split.

So, when we perform this lab and visit the web-frontend service which calls the customers service directly without going through the ingress gateway, we observe a roughly 50/50 traffic split between the two versions as expected. But this is because there's 1 replica of customers-v1 and 1 replica of customers-v2 so the customers service targets the v1 pod with 50% probability and the v2 pod with 50% probability - it has nothing to do with the customers VirtualService. Once we adjust the weights of the customers VirtualService to something like 80/20 and repeat the experiment, we still get a roughly 50/50 traffic split instead of the 80/20 split we might be expecting, because the web-frontend is targeting the customers Kubernetes service directly without going through the gateway where the VirtualService applies.

Once we modify the customers VirtualService to target all hosts '*' and modify the web-frontend VirtualService to target a specific host (say, www.example.com), then modify the web-frontend deployment to call the gateway instead (FQDN istio-ingressgateway.istio-system.svc.cluster.local), we can curl the gateway with header "Host: www.example.com" repeatedly and observe the 80/20 traffic split as expected. We can also observe the 80/20 traffic split by visiting the gateway's public IP in a browser, but we get raw JSON instead of a nice HTML page unless using developer tools for HTTP header injection since the browser doesn't automatically set the "Host: www.example.com" request header.

Anyway, I hope this lab gets fixed in the next revision to ensure all the explanations are correct.