You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a pod sends traffic to the exact same pod itself via a service, the traffic will fail.
What happened:
There is repro and log provided. Here is the analysis.
When a pod tries to connect to the pod itself, it's a direct connection and actually mizar is not involved. The connection will succeed because mizar will bypass the traffic.
But a pod may connect to a kubernetes service, and this service may actually pointing to the exact same pod. In this scenario, the pod sends traffic to service ip, and mizar networking is involved to handle the traffic. Then mizar will figure out from service ip to the real destination ip behind. Then mizar will replace the destination ip from the service ip to the destination ip.
In the issue's scenario, the traffic is from a pod, with ip (for example 11.0.0.11). And the service ip will point to the same destination pod, which the ip is 11.0.0.11. Now the packet will look as [source ip: 11.0.0.11, destination ip: 11.0.0.11]. The source and destination ip are the same. I believe mizar didn't aware such packet will occur then has no way to handle it for now.
How to reproduce it (as minimally and precisely as possible):
Start a pod
kubectl run pod1 --image=fwnetworking/testpod
Create a service which pointing to the pod
cat > /home/ubuntu/go/src/k8s.io/mizar/mizar/obj/tests/test_service.yaml
apiVersion: v1
kind: Service
metadata:
name: test-service-3
annotations:
service.beta.kubernetes.io/mizar-scaled-endpoint-type: "scaled-endpoint"
labels:
run: test-service-3
spec:
ports:
kubectl apply -f /home/ubuntu/go/src/k8s.io/mizar/mizar/obj/tests/test_service.yaml
3. Get ip for the pod and the service
kubectl get pods -owide
kubectl get services
5. Go into the pod
kubectl exec -i -t pod1 -- /bin/bash
6. Send traffic
Send traffic to the pod ip directly, succeed.
curl [pod ip]:8000
Send traffic to the service ip, failed.
curl [service ip]: 8000
Anything else we need to know?:
Network tracing log:
root@ip-172-31-20-85:/sys/kernel/tracing# cat trace_pipe | grep -i 0000b9 -A100 -B20
coredns-75061 [005] d.s1 2447.601923: bpf_trace_printk: Modified IP Address, src: 0x55141fac, dst: 0x55141fac, csum: 0x3dfa
When a pod sends traffic to the exact same pod itself via a service, the traffic will fail.
What happened:
There is repro and log provided. Here is the analysis.
When a pod tries to connect to the pod itself, it's a direct connection and actually mizar is not involved. The connection will succeed because mizar will bypass the traffic.
But a pod may connect to a kubernetes service, and this service may actually pointing to the exact same pod. In this scenario, the pod sends traffic to service ip, and mizar networking is involved to handle the traffic. Then mizar will figure out from service ip to the real destination ip behind. Then mizar will replace the destination ip from the service ip to the destination ip.
In the issue's scenario, the traffic is from a pod, with ip (for example 11.0.0.11). And the service ip will point to the same destination pod, which the ip is 11.0.0.11. Now the packet will look as [source ip: 11.0.0.11, destination ip: 11.0.0.11]. The source and destination ip are the same. I believe mizar didn't aware such packet will occur then has no way to handle it for now.
How to reproduce it (as minimally and precisely as possible):
kubectl run pod1 --image=fwnetworking/testpod
cat > /home/ubuntu/go/src/k8s.io/mizar/mizar/obj/tests/test_service.yaml
apiVersion: v1
kind: Service
metadata:
name: test-service-3
annotations:
service.beta.kubernetes.io/mizar-scaled-endpoint-type: "scaled-endpoint"
labels:
run: test-service-3
spec:
ports:
protocol: TCP
port: 8000
targetPort: 8000
selector:
run: pod1
(ctrl+D)
kubectl apply -f /home/ubuntu/go/src/k8s.io/mizar/mizar/obj/tests/test_service.yaml
3. Get ip for the pod and the service
kubectl get pods -owide
kubectl get services
5. Go into the pod
kubectl exec -i -t pod1 -- /bin/bash
6. Send traffic
Send traffic to the pod ip directly, succeed.
curl [pod ip]:8000
Send traffic to the service ip, failed.
curl [service ip]: 8000
Anything else we need to know?:
Network tracing log:
root@ip-172-31-20-85:/sys/kernel/tracing# cat trace_pipe | grep -i 0000b9 -A100 -B20
coredns-75061 [005] d.s1 2447.601923: bpf_trace_printk: Modified IP Address, src: 0x55141fac, dst: 0x55141fac, csum: 0x3dfa
containerd-shim-72090 [007] d.s. 2449.370150: bpf_trace_printk: [Transit:798:0xac1f1455] PASS non-geneve packet
The text was updated successfully, but these errors were encountered: