Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] 针对重启kube-ovn网络插件类型kubernetes集群所有节点场景,存在重启后EIP无法curl通异常(概率性出现,非必现),重启前可以curl通EIP #4684

Open
author970 opened this issue Oct 31, 2024 · 3 comments
Labels
bug Something isn't working eip

Comments

@author970
Copy link

author970 commented Oct 31, 2024

Kube-OVN Version

v1.12.22

Kubernetes Version

v1.27.6

Operation-system/Kernel Version

5.10.0-136.12.0.86.4.hl202.x86_64

Description

重启前可以curl通EIP,重启kube-ovn网络插件类型kubernetes集群所有节点后,待集群所有kube-ovn组件正常running,无法curl通EIP(重启所有节点有一定概率触发此异常,非必现),异常与issue2749一样。

Steps To Reproduce

  1. 重启集群所有节点,重启时间:2024-10-24 09:51:41;

  2. 重启后待集群正常,外部通过EIP curl,发现EIP不通(重启所有节点有一定概率触发此异常,非必现);

  3. kubectl exec进入vpc网关pod vpc-nat-gw容器中,vpc-nat-gw容器中网卡net1@if10上无 多个EIP的弹性IP;
    image

  4. kubectl delete vpc网关pod使pod重新创建后,创建后pod vpc-nat-gw容器中网卡net1@if10上存在多个EIP的弹性IP,正常。
    image

  5. 搜索kube-ovn-controller pod日志"vpc_nat_",显示如下:
    Search "vpc_nat_" (197 hits in 1 file)
    D:\文件资料\xxx\kube-ovn-controller.log.1 (197 hits)
    Line 49474: I1024 10:00:41.530660 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-x3xxhk7u
    Line 49475: I1024 10:00:41.530824 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-7jpvp699
    Line 49476: I1024 10:00:41.530843 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-r9zkzn3v
    Line 49477: I1024 10:00:41.530853 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-7bwa0m5z
    Line 49478: I1024 10:00:41.530860 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-8qfmsbkl
    Line 49479: I1024 10:00:41.530865 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-cu0kjnia
    Line 49480: I1024 10:00:41.530877 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-dvz0m5lr
    Line 49481: I1024 10:00:41.530881 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-pne0nuw5
    Line 49482: I1024 10:00:41.530889 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-yk2obf36
    Line 49483: I1024 10:00:41.531846 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-3wtszss5
    Line 49484: I1024 10:00:41.531868 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-4tyc1gw3
    Line 49485: I1024 10:00:41.531876 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-zzusfbel
    Line 49486: I1024 10:00:41.531885 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-daqwodb7
    Line 49487: I1024 10:00:41.531889 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-g3zbyw8r
    Line 49488: I1024 10:00:41.531901 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-v5w1k2j0
    Line 49494: I1024 10:00:41.531909 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-361x9nh3
    Line 49495: I1024 10:00:41.532043 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-4pjb6i2u
    Line 49496: I1024 10:00:41.532070 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-kia96no8
    Line 49497: I1024 10:00:41.532077 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-lbcwgylw
    Line 49498: I1024 10:00:41.532084 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-qm4lxmov
    Line 49499: I1024 10:00:41.532089 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-108njiom
    Line 49500: I1024 10:00:41.532095 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-a27scv76
    Line 49501: I1024 10:00:41.532101 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-uf3wqr1r
    Line 49502: I1024 10:00:41.532107 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-x8rddy7t
    Line 49503: I1024 10:00:41.532114 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-8t864sku
    Line 49504: I1024 10:00:41.532121 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-phot9clz
    Line 49505: I1024 10:00:41.532126 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-5szww1ks
    Line 49506: I1024 10:00:41.532139 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-86t3yqie
    Line 49507: I1024 10:00:41.532146 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-fzru7uvo
    Line 49510: I1024 10:00:41.532151 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-jvw39tvl
    Line 49511: I1024 10:00:41.532220 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-mqcmv0kp
    Line 49512: I1024 10:00:41.532244 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-pzl2hntv
    Line 49513: I1024 10:00:41.532255 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-27ti4824
    Line 49514: I1024 10:00:41.532269 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-2g9s6j9r
    Line 49515: I1024 10:00:41.532278 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-xs1gvx46
    Line 49517: I1024 10:00:41.532299 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-ujef7cwa
    Line 49518: I1024 10:00:41.532316 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-ummp3hw7
    Line 49520: I1024 10:00:41.532328 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-kwok61je
    Line 49521: I1024 10:00:41.532337 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-s16g1w4y
    Line 49522: I1024 10:00:41.532346 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-tu395xrm
    Line 49523: I1024 10:00:41.532354 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-v2ny3msu
    Line 49524: I1024 10:00:41.532362 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-yrpkufhr
    Line 49525: I1024 10:00:41.532368 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-cmx60tgx
    Line 49526: I1024 10:00:41.532377 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-e2m44kgt
    Line 49527: I1024 10:00:41.532383 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-g22bq8k8
    Line 49528: I1024 10:00:41.532390 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-gn9uve9h
    Line 49529: I1024 10:00:41.532398 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-hdlqbpyr
    Line 49530: I1024 10:00:41.532404 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-jrnpz0ci
    Line 49531: I1024 10:00:41.532410 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-xnqcw8n1
    Line 49532: I1024 10:00:41.532418 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-zlmwpqgp
    Line 49533: I1024 10:00:41.532424 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-5vq54jyx
    Line 49534: I1024 10:00:41.532431 7 vpc_nat_gw_eip.go:30] enqueue add iptables eip eip-9gfybji0
    Line 50605: I1024 10:00:42.151702 7 vpc_nat_gateway.go:235] handle add/update vpc nat gateway eip-gn9uve9h
    Line 50607: E1024 10:00:42.151903 7 vpc_nat_gateway.go:189] process: addOrUpdateVpcNatGateway. err: error syncing 'eip-gn9uve9h': iptables nat gw not enable, requeuing
    Line 50733: I1024 10:00:43.152962 7 vpc_nat_gateway.go:235] handle add/update vpc nat gateway eip-gn9uve9h
    Line 50734: E1024 10:00:43.152994 7 vpc_nat_gateway.go:189] process: addOrUpdateVpcNatGateway. err: error syncing 'eip-gn9uve9h': iptables nat gw not enable, requeuing
    Line 50746: I1024 10:00:45.152506 7 vpc_nat_gateway.go:235] handle add/update vpc nat gateway eip-gn9uve9h
    Line 50747: E1024 10:00:45.152514 7 vpc_nat_gw_nat.go:241] error syncing 'eip-lbcwgylw': iptables nat gw not enable, requeuing
    Line 50748: I1024 10:00:45.152554 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-v5w1k2j0
    Line 50749: I1024 10:00:45.152573 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-yk2obf36
    Line 50751: I1024 10:00:45.152607 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-x3xxhk7u
    Line 50753: I1024 10:00:45.152631 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-7jpvp699
    Line 50754: I1024 10:00:45.152643 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-r9zkzn3v
    Line 50755: I1024 10:00:45.152656 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-7bwa0m5z
    Line 50756: I1024 10:00:45.152667 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-8qfmsbkl
    Line 50757: I1024 10:00:45.152679 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-cu0kjnia
    Line 50758: I1024 10:00:45.152690 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-dvz0m5lr
    Line 50759: I1024 10:00:45.152479 7 vpc_nat_gateway.go:91] finish establishing vpc-nat-gateway
    Line 50760: I1024 10:00:45.152701 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-pne0nuw5
    Line 50761: I1024 10:00:45.152712 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-yk2obf36
    Line 50769: I1024 10:00:45.152723 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-3wtszss5
    Line 50771: I1024 10:00:45.152880 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-4tyc1gw3
    Line 50772: I1024 10:00:45.152898 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-zzusfbel
    Line 50774: I1024 10:00:45.152908 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-daqwodb7
    Line 50775: I1024 10:00:45.152916 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-g3zbyw8r
    Line 50780: I1024 10:00:45.152924 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-v5w1k2j0
    Line 50783: I1024 10:00:45.152583 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-cmx60tgx
    Line 50784: I1024 10:00:45.152979 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-361x9nh3
    Line 50787: I1024 10:00:45.153007 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-cu0kjnia
    Line 50788: I1024 10:00:45.153093 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-dvz0m5lr
    Line 50789: I1024 10:00:45.153109 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-fzru7uvo
    Line 50790: I1024 10:00:45.153125 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-kia96no8
    Line 50791: I1024 10:00:45.153140 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-phot9clz
    Line 50792: I1024 10:00:45.153150 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-pne0nuw5
    Line 50793: I1024 10:00:45.153163 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-s16g1w4y
    Line 50794: I1024 10:00:45.153176 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-361x9nh3
    Line 50795: I1024 10:00:45.153007 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-4pjb6i2u
    Line 50796: I1024 10:00:45.153208 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-kia96no8
    Line 50797: I1024 10:00:45.153223 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-lbcwgylw
    Line 50798: I1024 10:00:45.153235 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-qm4lxmov
    Line 50799: I1024 10:00:45.153245 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-108njiom
    Line 50800: I1024 10:00:45.153254 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-a27scv76
    Line 50804: I1024 10:00:45.153187 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-8t864sku
    Line 50807: I1024 10:00:45.153341 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-9gfybji0
    Line 50810: I1024 10:00:45.153264 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-uf3wqr1r
    Line 50811: I1024 10:00:45.153429 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-g22bq8k8
    Line 50813: I1024 10:00:45.153475 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-x8rddy7t
    Line 50815: I1024 10:00:45.153490 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-8t864sku
    Line 50816: I1024 10:00:45.153500 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-phot9clz
    Line 50818: I1024 10:00:45.153512 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-5szww1ks
    Line 50819: I1024 10:00:45.153471 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-x3xxhk7u
    Line 50820: I1024 10:00:45.153531 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-86t3yqie
    Line 50821: I1024 10:00:45.153545 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-x8rddy7t
    Line 50822: I1024 10:00:45.153558 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-xnqcw8n1
    Line 50823: I1024 10:00:45.153566 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-kwok61je
    Line 50827: I1024 10:00:45.153547 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-fzru7uvo
    Line 50828: I1024 10:00:45.153771 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-jvw39tvl
    Line 50829: I1024 10:00:45.153791 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-mqcmv0kp
    Line 50830: I1024 10:00:45.153799 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-pzl2hntv
    Line 50831: I1024 10:00:45.153808 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-27ti4824
    Line 50832: I1024 10:00:45.153815 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-2g9s6j9r
    Line 50833: I1024 10:00:45.153824 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-xs1gvx46
    Line 50844: I1024 10:00:45.153574 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-ujef7cwa
    Line 50847: I1024 10:00:45.153963 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-v2ny3msu
    Line 50849: I1024 10:00:45.153973 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-27ti4824
    Line 50851: I1024 10:00:45.153983 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-4pjb6i2u
    Line 50852: I1024 10:00:45.153993 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-ujef7cwa
    Line 50853: I1024 10:00:45.153997 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-5szww1ks
    Line 50854: I1024 10:00:45.154003 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-ummp3hw7
    Line 50855: I1024 10:00:45.154018 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-kwok61je
    Line 50856: I1024 10:00:45.154026 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-s16g1w4y
    Line 50857: I1024 10:00:45.154035 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-tu395xrm
    Line 50858: I1024 10:00:45.154042 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-v2ny3msu
    Line 50859: I1024 10:00:45.154047 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-yrpkufhr
    Line 50860: I1024 10:00:45.154054 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-cmx60tgx
    Line 50861: I1024 10:00:45.154060 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-e2m44kgt
    Line 50862: I1024 10:00:45.154066 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-g22bq8k8
    Line 51005: I1024 10:00:45.160532 7 vpc_nat_gateway.go:235] handle add/update vpc nat gateway eip-gn9uve9h
    Line 51031: I1024 10:00:45.163101 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-gn9uve9h
    Line 51032: I1024 10:00:45.163126 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-gn9uve9h
    Line 51033: I1024 10:00:45.163145 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-hdlqbpyr
    Line 51034: I1024 10:00:45.163154 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-jrnpz0ci
    Line 51035: I1024 10:00:45.163163 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-xnqcw8n1
    Line 51036: I1024 10:00:45.163173 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-zlmwpqgp
    Line 51037: I1024 10:00:45.163181 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-5vq54jyx
    Line 51038: I1024 10:00:45.163191 7 vpc_nat_gw_eip.go:208] handle add iptables eip eip-9gfybji0
    Line 51039: I1024 10:00:45.163135 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-7jpvp699
    Line 51040: I1024 10:00:45.163206 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-86t3yqie
    Line 51041: I1024 10:00:45.163214 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-tu395xrm
    Line 51042: I1024 10:00:45.163223 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-daqwodb7
    Line 51043: I1024 10:00:45.163240 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-g3zbyw8r
    Line 51044: I1024 10:00:45.163252 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-jrnpz0ci
    Line 51045: I1024 10:00:45.163260 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-uf3wqr1r
    Line 51046: I1024 10:00:45.163267 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-108njiom
    Line 51047: I1024 10:00:45.163277 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-2g9s6j9r
    Line 51048: I1024 10:00:45.163285 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-4tyc1gw3
    Line 51049: I1024 10:00:45.163294 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-5vq54jyx
    Line 51050: I1024 10:00:45.163304 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-xs1gvx46
    Line 51051: I1024 10:00:45.163314 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-8qfmsbkl
    Line 51052: I1024 10:00:45.163321 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-r9zkzn3v
    Line 51053: I1024 10:00:45.163334 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-zlmwpqgp
    Line 51054: I1024 10:00:45.163341 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-zzusfbel
    Line 51055: I1024 10:00:45.163349 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-ummp3hw7
    Line 51056: I1024 10:00:45.163355 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-yrpkufhr
    Line 51057: I1024 10:00:45.163363 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-3wtszss5
    Line 51058: I1024 10:00:45.163371 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-7bwa0m5z
    Line 51059: I1024 10:00:45.163379 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-jvw39tvl
    Line 51060: I1024 10:00:45.163385 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-mqcmv0kp
    Line 51061: I1024 10:00:45.163392 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-qm4lxmov
    Line 51062: I1024 10:00:45.163399 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-a27scv76
    Line 51063: I1024 10:00:45.163408 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-e2m44kgt
    Line 51064: I1024 10:00:45.163416 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-hdlqbpyr
    Line 51065: I1024 10:00:45.163422 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-pzl2hntv
    Line 51076: I1024 10:00:45.231712 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
    Line 51095: E1024 10:00:45.287490 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51096: E1024 10:00:45.287516 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51097: E1024 10:00:45.287545 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
    Line 51390: I1024 10:00:46.230485 7 vpc_nat_gw_nat.go:486] handle add iptables fip eip-lbcwgylw
    Line 51401: I1024 10:00:46.330325 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
    Line 51425: E1024 10:00:46.430322 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51426: E1024 10:00:46.430338 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51427: E1024 10:00:46.430355 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
    Line 51507: I1024 10:00:48.430937 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
    Line 51508: E1024 10:00:48.457488 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51509: E1024 10:00:48.457513 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51510: E1024 10:00:48.457543 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
    Line 51511: I1024 10:00:52.458104 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
    Line 51512: E1024 10:00:52.479708 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51513: E1024 10:00:52.479724 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51514: E1024 10:00:52.479742 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
    Line 51518: I1024 10:01:00.480109 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
    Line 51519: E1024 10:01:00.501763 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51520: E1024 10:01:00.501800 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51521: E1024 10:01:00.501834 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
    Line 51639: I1024 10:01:16.502776 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
    Line 51641: E1024 10:01:16.538024 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51642: E1024 10:01:16.538051 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 51643: E1024 10:01:16.538077 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
    Line 54122: I1024 10:01:36.538929 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
    Line 54125: E1024 10:01:36.564674 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 54126: E1024 10:01:36.564695 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 54127: E1024 10:01:36.564716 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
    Line 55287: I1024 10:01:56.565251 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h
    Line 55290: E1024 10:01:56.591380 7 vpc_nat_gateway.go:729] unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 55291: E1024 10:01:56.591397 7 vpc_nat_gateway.go:620] failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw")
    Line 55292: E1024 10:01:56.591413 7 vpc_nat_gateway.go:189] process: updateVpcSubnet. err: error syncing 'eip-gn9uve9h': failed to exec nat gateway rule, err: unable to upgrade connection: container not found ("vpc-nat-gw"), requeuing
    Line 56820: I1024 10:02:16.592432 7 vpc_nat_gateway.go:593] handle update subnet route for nat gateway eip-gn9uve9h

Current Behavior

重启集群所有节点后,EIP存在概率无法访问。

Expected Behavior

重启集群所有节点后,EIP可正常访问。

@author970 author970 added the bug Something isn't working label Oct 31, 2024
@dosubot dosubot bot added the eip label Oct 31, 2024
Copy link

dosubot bot commented Oct 31, 2024

Hello, @author970! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together!

I found some similar issues related to EIP connectivity problems after restarting the kube-ovn network plugin:

  1. [BUG] NatGateway ceases working after being restarted: This issue was related to conntrack entries not performing SNAT correctly after a NAT Gateway restart. The suggested solutions were to prevent serving traffic before routes and iptables rules are configured or to flush conntrack entries without SNAT/DNAT after routes and iptables rules are configured [1].

  2. VPC Gateway loses connectivity as soon as an EIP gets attached to it: The workaround for this issue involved disabling ARP on the network interface before and after attaching the EIP, which seemed to address the connectivity problem [2].

  3. EIP NAT not working for few IPs in subnet: It was suggested to use the release-1.12-mc branch or the master branch of kube-ovn, as many bugs related to ovn eip dnat snat fip were fixed in these versions [3].

These issues might provide some insights or potential solutions to the problem you're experiencing with EIP connectivity after a full cluster restart.

To continue talking to Dosu, mention @dosu.

@author970 author970 changed the title [BUG] 针对重启kube-ovn网络插件类型kubernetes集群所有节点场景,存在重启后EIP无法curl通异常,重启前可以curl通EIP [BUG] 针对重启kube-ovn网络插件类型kubernetes集群所有节点场景,存在重启后EIP无法curl通异常(概率性出现,非必现),重启前可以curl通EIP Oct 31, 2024
@bobz965
Copy link
Collaborator

bobz965 commented Oct 31, 2024

一次性重启集群所有节点么?

@author970
Copy link
Author

一次性重启集群所有节点么?

是的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working eip
Projects
None yet
Development

No branches or pull requests

2 participants