chore(bgp): documentation on new bgp nat gw feature

[SkalaNetworks]

No description provided.

[ah8ad3]

Thanks for the docs.
I tried it it seems we have to change default subnet's provider to the nad you created here. I tried without it failed, changed it it can talk to api-server.

Update:
NVM, it still fails i will try to debug it.

[ah8ad3]

It works bug i don't know how and why? Sometimes it can connect to the api-server and sometimes it can not, and i try this by deleting nat-gw and creating it again, maybe there is something wrong with my setup idk.
But thanks @SkalaNetworks

[SkalaNetworks]

You're right about changing the provider of the default subnet, I edited the docs to reflect that.
Can you show errors about failling to connect?

[ah8ad3]

@SkalaNetworks
When it doesn't connect it has something like this:

I0811 07:17:46.898877       1 speaker.go:21] 
-------------------------------------------------------------------------------
Kube-OVN: 
  Version:       v1.13.0
  Build:         2024-08-08_08:00:39
  Commit:        git-5baabfb
  Go Version:    go1.22.6
  Arch:          amd64
-------------------------------------------------------------------------------
I0811 07:17:46.899006       1 speaker.go:24] current capabilities: cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap=ep
I0811 07:17:46.899217       1 config.go:199] no --kubeconfig, use in-cluster kubernetes config
time="2024-08-11T07:17:46Z" level=info msg="Add a peer configuration" Key=10.0.1.1 Topic=Peer
I0811 07:17:46.901328       1 controller.go:51] Creating event broadcaster
I0811 07:17:46.902089       1 reflector.go:296] Starting reflector *v1.Service (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:17:46.902114       1 reflector.go:296] Starting reflector *v1.Pod (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:17:46.902140       1 reflector.go:332] Listing and watching *v1.Service from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:17:46.902145       1 reflector.go:332] Listing and watching *v1.Pod from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:17:46.902156       1 reflector.go:296] Starting reflector *v1.VpcNatGateway (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:17:46.902170       1 reflector.go:332] Listing and watching *v1.VpcNatGateway from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:17:46.902155       1 reflector.go:296] Starting reflector *v1.IptablesEIP (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:17:46.902189       1 reflector.go:332] Listing and watching *v1.IptablesEIP from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:17:46.902090       1 reflector.go:296] Starting reflector *v1.Subnet (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:17:46.902215       1 reflector.go:332] Listing and watching *v1.Subnet from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
W0811 07:17:49.955177       1 reflector.go:547] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232: failed to list *v1.IptablesEIP: Get "https://10.96.0.1:443/apis/kubeovn.io/v1/iptables-eips?allowWatchBookmarks=true&limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: no route to host
W0811 07:17:49.955212       1 reflector.go:547] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232: failed to list *v1.Subnet: Get "https://10.96.0.1:443/apis/kubeovn.io/v1/subnets?allowWatchBookmarks=true&limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: no route to host
W0811 07:17:49.955210       1 reflector.go:547] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?allowWatchBookmarks=true&limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: no route to host
E0811 07:17:49.955329       1 reflector.go:150] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232: Failed to watch *v1.IptablesEIP: failed to list *v1.IptablesEIP: Get "https://10.96.0.1:443/apis/kubeovn.io/v1/iptables-eips?allowWatchBookmarks=true&limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: no route to host
W0811 07:17:49.955278       1 reflector.go:547] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232: failed to list *v1.Pod: Get "https://10.96.0.1:443/api/v1/pods?allowWatchBookmarks=true&limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: no route to host
E0811 07:17:49.955396       1 reflector.go:150] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?allowWatchBookmarks=true&limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: no route to host
E0811 07:17:49.955403       1 reflector.go:150] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232: Failed to watch *v1.Subnet: failed to list *v1.Subnet: Get "https://10.96.0.1:443/apis/kubeovn.io/v1/subnets?allowWatchBookmarks=true&limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: no route to host
W0811 07:17:49.955212       1 reflector.go:547] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232: failed to list *v1.VpcNatGateway: Get "https://10.96.0.1:443/apis/kubeovn.io/v1/vpc-nat-gateways?allowWatchBookmarks=true&limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: no route to host
E0811 07:17:49.955477       1 reflector.go:150] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232: Failed to watch *v1.Pod: failed to list *v1.Pod: Get "https://10.96.0.1:443/api/v1/pods?allowWatchBookmarks=true&limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: no route to host

And when it's ok it will show something like this:

I0811 07:20:59.069487       1 speaker.go:21] 
-------------------------------------------------------------------------------
Kube-OVN: 
  Version:       v1.13.0
  Build:         2024-08-08_08:00:39
  Commit:        git-5baabfb
  Go Version:    go1.22.6
  Arch:          amd64
-------------------------------------------------------------------------------
I0811 07:20:59.070223       1 speaker.go:24] current capabilities: cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap=ep
I0811 07:20:59.070740       1 config.go:199] no --kubeconfig, use in-cluster kubernetes config
time="2024-08-11T07:20:59Z" level=info msg="Add a peer configuration" Key=10.0.1.1 Topic=Peer
I0811 07:20:59.076843       1 controller.go:51] Creating event broadcaster
I0811 07:20:59.077897       1 reflector.go:296] Starting reflector *v1.Service (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.077946       1 reflector.go:332] Listing and watching *v1.Service from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.077938       1 reflector.go:296] Starting reflector *v1.Pod (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.077974       1 reflector.go:332] Listing and watching *v1.Pod from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.077908       1 reflector.go:296] Starting reflector *v1.IptablesEIP (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.078028       1 reflector.go:332] Listing and watching *v1.IptablesEIP from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.078107       1 reflector.go:296] Starting reflector *v1.VpcNatGateway (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.078137       1 reflector.go:332] Listing and watching *v1.VpcNatGateway from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.078053       1 reflector.go:296] Starting reflector *v1.Subnet (0s) from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.078212       1 reflector.go:332] Listing and watching *v1.Subnet from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.089885       1 reflector.go:359] Caches populated for *v1.Service from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.090954       1 reflector.go:359] Caches populated for *v1.IptablesEIP from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.091352       1 reflector.go:359] Caches populated for *v1.VpcNatGateway from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.096991       1 reflector.go:359] Caches populated for *v1.Subnet from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.102591       1 reflector.go:359] Caches populated for *v1.Pod from k8s.io/[email protected]+incompatible/tools/cache/reflector.go:232
I0811 07:20:59.178892       1 controller.go:104] Started workers
I0811 07:20:59.179192       1 bgp.go:78] currently announcing IPv4 routes: []
I0811 07:20:59.179266       1 bgp.go:88] new routes we will announce: []
I0811 07:20:59.179280       1 bgp.go:96] announced routes we will withdraw: []

Both reports generated by just deleting and recreating the same schema.

[ah8ad3]

Also can we log if the connection to the neighbor is not available?
IDK if gobgp can check that and we invoke some function from it?

[SkalaNetworks]

Can you print the YAML of the NAT GW pod when it fails to connect to the API?
Set the extra args to -v5 (with the patch in the PR here kubeovn/kube-ovn#4352) and I added a custom logger to gobgp so that we can get logs about what the speaker is doing.

[ah8ad3]

Can you print the YAML of the NAT GW pod when it fails to connect to the API? Set the extra args to -v5 (with the patch in the PR here kubeovn/kube-ovn#4352) and I added a custom logger to gobgp so that we can get logs about what the speaker is doing.

For sure next that happens i will forward it to you. Maybe my setup is buggy i changed it a lot.