Replace deprecated and obsoleted `IPDB` with `NDB` #758

gab-arrobo · 2024-02-05T23:27:51Z

IPDB is deprecated, and obsolete as stated in IPDB module — pyroute2 0.7.3.post2 documentation. This PR resolves #735 by replacing IPDB with NDB

Run Python Lint and Test locally
Address issue with Python Lint and Test when using GitHub runners
Run an E2E test (UPF in stand-alone mode)
Run an E2E test (UPF as part of AiaB)

gab-arrobo · 2024-02-06T00:29:03Z

I am wondering why the Python Lint and Test action fails... would it require sudo for the tests to pass, which is suspicious because in my local setup, the action successfully completes

[Python Lint and Test/route-control-tests] ⭐ Run Main Run tests
[Python Lint and Test/route-control-tests]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/4] user= workdir=
| 2024-02-05 23:16:15,586 INFO Neighbor entry does not exist, creating modules.
| 2024-02-05 23:16:15,588 INFO Neighbor already exists
| 2024-02-05 23:16:15,589 INFO Route count for 1.2.3.4 decremented to 1
| .2024-02-05 23:16:15,592 INFO Neighbor entry does not exist, creating modules.
| 2024-02-05 23:16:15,593 INFO Module deleted random_interfaceRoutesDstMAC001A2B3C4D5E
| 2024-02-05 23:16:15,593 INFO Deleted item from neighbor cache
| .2024-02-05 23:16:15,596 INFO Neighbor entry does not exist, creating modules.
| 2024-02-05 23:16:15,597 INFO Module deleted random_interfaceRoutesDstMAC001A2B3C4D5E
| 2024-02-05 23:16:15,597 INFO Deleted item from neighbor cache
| ..2024-02-05 23:16:15,599 INFO RTM_DELROUTE netlink event received.
| .2024-02-05 23:16:15,600 INFO RTM_NEWNEIGH netlink event received.
| 2024-02-05 23:16:15,600 ERROR Error parsing route entry message
| Traceback (most recent call last):
|   File "/home/ubuntu/upf-ga/conf/route_control.py", line 659, in _parse_route_entry_msg
|     attr_dict = dict(route_entry["attrs"])
|                      ~~~~~~~~~~~^^^^^^^^^
| TypeError: string indices must be integers, not 'str'
| .2024-02-05 23:16:15,602 INFO RTM_NEWROUTE netlink event received.
| .2024-02-05 23:16:15,604 INFO Mac address not found for 1.2.3.4
| 2024-02-05 23:16:15,604 INFO mac address of the next hop 1.2.3.4 is not stored in ARP table. Probing...
| 2024-02-05 23:16:15,604 INFO Adding entry RouteEntry(next_hop_ip='1.2.3.4', interface='random_interface', dest_prefix='1.1.1.1', prefix_len=24) in arp table by pinging
| 2024-02-05 23:16:15,605 INFO Neighbor entry does not exist, creating modules.
| ..2024-02-05 23:16:15,608 INFO Mac address not found for 1.2.3.4
| 2024-02-05 23:16:15,608 INFO mac address of the next hop 1.2.3.4 is not stored in ARP table. Probing...
| 2024-02-05 23:16:15,608 INFO Adding entry RouteEntry(next_hop_ip='1.2.3.4', interface='random_interface', dest_prefix='1.1.1.1', prefix_len=24) in arp table by pinging
| ...2024-02-05 23:16:15,614 INFO Mac address not found for 1.2.3.4
| 2024-02-05 23:16:15,614 INFO mac address of the next hop 1.2.3.4 is not stored in ARP table. Probing...
| 2024-02-05 23:16:15,614 INFO Adding entry RouteEntry(next_hop_ip='1.2.3.4', interface='random_interface', dest_prefix='1.1.1.1', prefix_len=24) in arp table by pinging
| 2024-02-05 23:16:15,614 INFO Sending ping to 1.2.3.4
| .
| Sent 1 packets.
| .2024-02-05 23:16:15,722 INFO Mac address not found for 1.2.3.4
| 2024-02-05 23:16:15,722 INFO mac address of the next hop 1.2.3.4 is not stored in ARP table. Probing...
| 2024-02-05 23:16:15,722 INFO Adding entry RouteEntry(next_hop_ip='1.2.3.4', interface='random_interface', dest_prefix='1.1.1.1', prefix_len=24) in arp table by pinging
| 2024-02-05 23:16:15,722 INFO Sending ping to 1.2.3.4
| .
| Sent 1 packets.
| ...2024-02-05 23:16:15,797 ERROR The IP address 192.168.300.1 is invalid
| .2024-02-05 23:16:15,797 ERROR The IP address  is invalid
| ..2024-02-05 23:16:15,800 INFO Mac address found for 192.168.1.1, Mac: 00:1a:2b:3c:4d:5e
| .2024-02-05 23:16:15,802 INFO Mac address found for 192.168.1.1, Mac: None
| ....
| ----------------------------------------------------------------------
| Ran 24 tests in 0.220s
|
| OK
[Python Lint and Test/route-control-tests]   ✅  Success - Main Run tests

ghislainbourgeois · 2024-02-07T00:50:56Z

I am wondering why the Python Lint and Test action fails... would it require sudo for the tests to pass, which is suspicious because in my local setup, the action successfully completes

[Python Lint and Test/route-control-tests] ⭐ Run Main Run tests
[Python Lint and Test/route-control-tests]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/4] user= workdir=
| 2024-02-05 23:16:15,586 INFO Neighbor entry does not exist, creating modules.
| 2024-02-05 23:16:15,588 INFO Neighbor already exists
| 2024-02-05 23:16:15,589 INFO Route count for 1.2.3.4 decremented to 1
| .2024-02-05 23:16:15,592 INFO Neighbor entry does not exist, creating modules.
| 2024-02-05 23:16:15,593 INFO Module deleted random_interfaceRoutesDstMAC001A2B3C4D5E
| 2024-02-05 23:16:15,593 INFO Deleted item from neighbor cache
| .2024-02-05 23:16:15,596 INFO Neighbor entry does not exist, creating modules.
| 2024-02-05 23:16:15,597 INFO Module deleted random_interfaceRoutesDstMAC001A2B3C4D5E
| 2024-02-05 23:16:15,597 INFO Deleted item from neighbor cache
| ..2024-02-05 23:16:15,599 INFO RTM_DELROUTE netlink event received.
| .2024-02-05 23:16:15,600 INFO RTM_NEWNEIGH netlink event received.
| 2024-02-05 23:16:15,600 ERROR Error parsing route entry message
| Traceback (most recent call last):
|   File "/home/ubuntu/upf-ga/conf/route_control.py", line 659, in _parse_route_entry_msg
|     attr_dict = dict(route_entry["attrs"])
|                      ~~~~~~~~~~~^^^^^^^^^
| TypeError: string indices must be integers, not 'str'
| .2024-02-05 23:16:15,602 INFO RTM_NEWROUTE netlink event received.
| .2024-02-05 23:16:15,604 INFO Mac address not found for 1.2.3.4
| 2024-02-05 23:16:15,604 INFO mac address of the next hop 1.2.3.4 is not stored in ARP table. Probing...
| 2024-02-05 23:16:15,604 INFO Adding entry RouteEntry(next_hop_ip='1.2.3.4', interface='random_interface', dest_prefix='1.1.1.1', prefix_len=24) in arp table by pinging
| 2024-02-05 23:16:15,605 INFO Neighbor entry does not exist, creating modules.
| ..2024-02-05 23:16:15,608 INFO Mac address not found for 1.2.3.4
| 2024-02-05 23:16:15,608 INFO mac address of the next hop 1.2.3.4 is not stored in ARP table. Probing...
| 2024-02-05 23:16:15,608 INFO Adding entry RouteEntry(next_hop_ip='1.2.3.4', interface='random_interface', dest_prefix='1.1.1.1', prefix_len=24) in arp table by pinging
| ...2024-02-05 23:16:15,614 INFO Mac address not found for 1.2.3.4
| 2024-02-05 23:16:15,614 INFO mac address of the next hop 1.2.3.4 is not stored in ARP table. Probing...
| 2024-02-05 23:16:15,614 INFO Adding entry RouteEntry(next_hop_ip='1.2.3.4', interface='random_interface', dest_prefix='1.1.1.1', prefix_len=24) in arp table by pinging
| 2024-02-05 23:16:15,614 INFO Sending ping to 1.2.3.4
| .
| Sent 1 packets.
| .2024-02-05 23:16:15,722 INFO Mac address not found for 1.2.3.4
| 2024-02-05 23:16:15,722 INFO mac address of the next hop 1.2.3.4 is not stored in ARP table. Probing...
| 2024-02-05 23:16:15,722 INFO Adding entry RouteEntry(next_hop_ip='1.2.3.4', interface='random_interface', dest_prefix='1.1.1.1', prefix_len=24) in arp table by pinging
| 2024-02-05 23:16:15,722 INFO Sending ping to 1.2.3.4
| .
| Sent 1 packets.
| ...2024-02-05 23:16:15,797 ERROR The IP address 192.168.300.1 is invalid
| .2024-02-05 23:16:15,797 ERROR The IP address  is invalid
| ..2024-02-05 23:16:15,800 INFO Mac address found for 192.168.1.1, Mac: 00:1a:2b:3c:4d:5e
| .2024-02-05 23:16:15,802 INFO Mac address found for 192.168.1.1, Mac: None
| ....
| ----------------------------------------------------------------------
| Ran 24 tests in 0.220s
|
| OK
[Python Lint and Test/route-control-tests]   ✅  Success - Main Run tests

Are you running the test in a docker container? In that case, you are probably root inside the container, letting you open a socket. Mocking the send_ping function will let you bypass the actual sending of a ping. You can find an example on line 217 in the test file.

ghislainbourgeois

Overall it looks good, I provided some comments on the tests.

conf/test_route_control.py

gab-arrobo · 2024-02-07T02:09:49Z

Overall it looks good, I provided some comments on the tests.

Thanks for your feedback!

conf/route_control.py

conf/test_route_control.py

Co-authored-by: Ghislain Bourgeois <[email protected]>

gruyaume

Looks good (assuming the lint check gets figured out)

gab-arrobo · 2024-02-07T19:36:07Z

Looks good (assuming the lint check gets figured out)

Investigating the lint issue. However, I am not quite sure why there is an issue with (GO) lint because we are not touching anything related to it.

…ystem for changes

ghislainbourgeois · 2024-02-07T22:22:32Z

conf/route_control.py

        ipr=ipr,
        interfaces=interface_arg,
    )
    route_controller.bootstrap_routes()
-    route_controller.register_callbacks()


This callback was the way that routectl was notified that a new mac address was added to the arp table so that it could configure BESS. I do not see a replacement for this.

When routectl starts, it will look for the mac address of the access gateway and core gateway, and if they are not present, it is sending pings out in the background. The mac address will be added to the arp table in the background. We need to either get notified of the change, or do some kind of polling.

Correct, that is because the way IPDB works. However, with NDB, that should not be needed as indicated here (also below):

The goal of NDB is to provide an easy access to RTNL info and entities via Python objects, like
pyroute2.ndb.objects.interface (see also: [Network interfaces]
(https://docs.pyroute2.org/ndb_interfaces.html#ndbinterfaces)), pyroute2.ndb.objects.route (see also: Routes
management) etc. These objects do not only reflect the
system state for the time of their instantiation, but continuously monitor the system for relevant updates. The
monitoring is done via netlink notifications, thus no polling.

Moreover, as indicated here, NDB does NOT have a register_callback method and when I was trying to use the method indicated in the issue (ndb.register_handler()), I got an error. I am currently running some tests

But will we react to those changes even if they are monitored by NDB?
And if it is the case that we don't need to register callbacks anymore then we probably could remove _netlink_event_listener.

But will we react to those changes even if they are monitored by NDB? And if it is the case that we don't need to register callbacks anymore then we probably could remove _netlink_event_listener.

It looks like the register_handler() is needed because when running the UPF as part of AiaB, the E2E fails. I am planning to add the register_handler() and run some tests.

When using regsiter_handler(), I get an error message saying "AttributeError: 'NDB' object has no attribute 'register_handler'"

Click me to see routectl logs

$ kubectl -n omec logs upf-0 -c routectl
2024-02-08 08:06:09,186 INFO Connected to BESS daemon
2024-02-08 08:06:09,197 INFO Mac address found for 192.168.250.1, Mac: 3e:8d:e3:da:cb:3e
2024-02-08 08:06:09,201 INFO Route entry 0.0.0.0/0 added to coreRoutes
2024-02-08 08:06:09,205 INFO Neighbor entry does not exist, creating modules.
2024-02-08 08:06:09,208 ERROR Module coreDstMAC3E8DE3DACB3E already exists
2024-02-08 08:06:09,215 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection coreRoutes:0->0:coreDstMAC3E8DE3DACB3E failed
2024-02-08 08:06:09,218 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:11,226 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection coreRoutes:0->0:coreDstMAC3E8DE3DACB3E failed
2024-02-08 08:06:11,227 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:13,235 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection coreRoutes:0->0:coreDstMAC3E8DE3DACB3E failed
2024-02-08 08:06:13,236 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:15,245 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection coreRoutes:0->0:coreDstMAC3E8DE3DACB3E failed
2024-02-08 08:06:15,246 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:17,254 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection coreRoutes:0->0:coreDstMAC3E8DE3DACB3E failed
2024-02-08 08:06:17,255 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:19,261 ERROR Error linking module coreDstMAC3E8DE3DACB3E to module coreRoutes
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 490, in _create_module_links
self._bess_controller.link_modules(
File "/opt/bess/bessctl/conf/route_control.py", line 269, in link_modules
raise Exception(
Exception: BESS module connection (coreRoutes:0->0:coreDstMAC3E8DE3DACB3E) failure.
2024-02-08 08:06:19,267 INFO Mac address found for 192.168.252.1, Mac: 26:9f:6a:c1:e4:13
2024-02-08 08:06:19,270 INFO Route entry 192.168.251.0/24 added to accessRoutes
2024-02-08 08:06:19,275 INFO Neighbor entry does not exist, creating modules.
2024-02-08 08:06:19,277 ERROR Module accessDstMAC269F6AC1E413 already exists
2024-02-08 08:06:19,284 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection accessRoutes:0->0:accessDstMAC269F6AC1E413 failed
2024-02-08 08:06:19,285 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:21,294 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection accessRoutes:0->0:accessDstMAC269F6AC1E413 failed
2024-02-08 08:06:21,295 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:23,304 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection accessRoutes:0->0:accessDstMAC269F6AC1E413 failed
2024-02-08 08:06:23,305 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:25,313 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection accessRoutes:0->0:accessDstMAC269F6AC1E413 failed
2024-02-08 08:06:25,314 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:27,323 ERROR Got BESS error
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 239, in link_modules
self._bess.connect_modules(module, next_module, ogate, igate)
File "/opt/bess/pybess/bess.py", line 489, in connect_modules
return self._request('ConnectModules', request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/bess/pybess/bess.py", line 280, in _request
raise self.Error(code, errmsg, query=name, query_arg=req_dict)
pybess.bess.BESS.Error: errno=16 (EBUSY: Device or resource busy), Connection accessRoutes:0->0:accessDstMAC269F6AC1E413 failed
2024-02-08 08:06:27,324 ERROR Got code EBUSY. Retrying in 2 secs...
2024-02-08 08:06:29,330 ERROR Error linking module accessDstMAC269F6AC1E413 to module accessRoutes
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 490, in _create_module_links
self._bess_controller.link_modules(
File "/opt/bess/bessctl/conf/route_control.py", line 269, in link_modules
raise Exception(
Exception: BESS module connection (accessRoutes:0->0:accessDstMAC269F6AC1E413) failure.
2024-02-08 08:06:29,332 INFO Registering netlink event listener handler...
Traceback (most recent call last):
File "/opt/bess/bessctl/conf/route_control.py", line 810, in
route_controller.register_handlers()
File "/opt/bess/bessctl/conf/route_control.py", line 352, in register_handlers
self._event_callback = self._ndb.register_handler(ifinfmsg,
^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NDB' object has no attribute 'register_handler'

It seems like register_handler is now part of the class TaskManager. Here.
And I see it being called in some places of the ndb code as self.ndb.task_manager.register_handler(<event>, <handler>)
Like here.
And it seems to me as if register_handler has been wrapped into TmpHandler
And it is used here in ndb objects to register handlers too.
I didn't go into details to understand how it works so I'm not sure if we can leverage TmpHandler or ndb objects, or if we should use the function directly and for some reason I'm unable to access the docs on https://docs.pyroute2.org.

I took a look at it and it seems really poorly documented. TaskManager and TmpHandler are not mentioned at all in the documentation. TaskManager is a public attribute of NDB, so I think it would be reasonable to use it, but we would have to test it out.

Another approach we could take is to create a polling thread in routectl (or reuse the pinging thread) to trigger the BESS configuration when the MAC address is present.

Using the task_manager, then the register_handler works as expected and the pipeline for the N3 (access) interface is completed. However, the pipeline for the N6 (core) interface is still incomplete.

$ kubectl -n omec logs upf-0 -c routectl 2024-02-22 05:02:28,909 INFO Connected to BESS daemon 2024-02-22 05:02:28,921 INFO Mac address not found for 192.168.250.1 2024-02-22 05:02:28,921 INFO mac address of the next hop 192.168.250.1 is not stored in ARP table. Probing... 2024-02-22 05:02:28,921 INFO Adding entry RouteEntry(next_hop_ip='192.168.250.1', interface='core', dest_prefix='0.0.0.0', prefix_len=0) in arp table by pinging 2024-02-22 05:02:28,921 INFO Sending ping to 192.168.250.1 . Sent 1 packets. 2024-02-22 05:02:29,102 INFO Mac address found for 192.168.252.1, Mac: ca:14:78:27:df:ba 2024-02-22 05:02:29,106 INFO Route entry 192.168.251.0/24 added to accessRoutes 2024-02-22 05:02:29,111 INFO Neighbor entry does not exist, creating modules. 2024-02-22 05:02:29,114 INFO Add Update module accessDstMACCA147827DFBA successfully 2024-02-22 05:02:29,121 INFO Module accessRoutes:0->0/accessDstMACCA147827DFBA linked 2024-02-22 05:02:29,128 INFO Module accessDstMACCA147827DFBA:0->0/accessMerge linked 2024-02-22 05:02:29,133 INFO Registering netlink event listener handler... 2024-02-22 05:02:29,134 INFO Missing ARP entries: ['192.168.250.1'] 2024-02-22 05:02:29,134 INFO Sending ping to 192.168.250.1 2024-02-22 05:02:29,136 INFO Ping missing entries thread started 2024-02-22 05:02:29,136 INFO Registering signals handlers. 2024-02-22 05:02:29,136 INFO Sleep until a signal is received 2024-02-22 05:02:29,141 INFO {'family': 0, '__align': (), 'ifi_type': 1, 'index': 5, 'flags': 69699, 'change': 256, 'attrs': [('IFLA_IFNAME', 'core'), ('IFLA_TXQLEN', 0), ('IFLA_OPERSTATE', 'UP'), ('IFLA_LINKMODE', 0), ('IFLA_MTU', 1500), ('IFLA_MIN_MTU', 68), ('IFLA_MAX_MTU', 0), ('IFLA_GROUP', 0), ('IFLA_PROMISCUITY', 1), ('IFLA_NUM_TX_QUEUES', 1), ('IFLA_GSO_MAX_SEGS', 65535), ('IFLA_GSO_MAX_SIZE', 65536), ('IFLA_NUM_RX_QUEUES', 1), ('IFLA_CARRIER', 1), ('IFLA_QDISC', 'noqueue'), ('IFLA_CARRIER_CHANGES', 0), ('IFLA_CARRIER_UP_COUNT', 0), ('IFLA_CARRIER_DOWN_COUNT', 0), ('IFLA_PROTO_DOWN', 0), ('IFLA_MAP', {'mem_start': 0, 'mem_end': 0, 'base_addr': 0, 'irq': 0, 'dma': 0, 'port': 0}), ('IFLA_ADDRESS', '5a:a5:0e:4c:e1:0c'), ('IFLA_BROADCAST', 'ff:ff:ff:ff:ff:ff'), ('IFLA_STATS64', {'rx_packets': 7, 'tx_packets': 13, 'rx_bytes': 336, 'tx_bytes': 962, 'rx_errors': 0, 'tx_errors': 0, 'rx_dropped': 0, 'tx_dropped': 0, 'multicast': 3, 'collisions': 0, 'rx_length_errors': 0, 'rx_over_errors': 0, 'rx_crc_errors': 0, 'rx_frame_errors': 0, 'rx_fifo_errors': 0, 'rx_missed_errors': 0, 'tx_aborted_errors': 0, 'tx_carrier_errors': 0, 'tx_fifo_errors': 0, 'tx_heartbeat_errors': 0, 'tx_window_errors': 0, 'rx_compressed': 0, 'tx_compressed': 0}), ('IFLA_STATS', {'rx_packets': 7, 'tx_packets': 13, 'rx_bytes': 336, 'tx_bytes': 962, 'rx_errors': 0, 'tx_errors': 0, 'rx_dropped': 0, 'tx_dropped': 0, 'multicast': 3, 'collisions': 0, 'rx_length_errors': 0, 'rx_over_errors': 0, 'rx_crc_errors': 0, 'rx_frame_errors': 0, 'rx_fifo_errors': 0, 'rx_missed_errors': 0, 'tx_aborted_errors': 0, 'tx_carrier_errors': 0, 'tx_fifo_errors': 0, 'tx_heartbeat_errors': 0, 'tx_window_errors': 0, 'rx_compressed': 0, 'tx_compressed': 0}), ('IFLA_XDP', {'attrs': [('IFLA_XDP_ATTACHED', None)]}), ('IFLA_LINKINFO', {'attrs': [('IFLA_INFO_KIND', 'macvlan'), ('IFLA_INFO_DATA', {'attrs': [('IFLA_MACVLAN_MODE', 'bridge'), ('IFLA_MACVLAN_FLAGS', 'none'), ('IFLA_MACVLAN_MACADDR_COUNT', 0), ('UNKNOWN', {'header': {'length': 8, 'type': 7}}), ('UNKNOWN', {'header': {'length': 8, 'type': 8}})], 'header': {}, 'index': 5})]}), ('IFLA_LINK_NETNSID', 0), ('IFLA_LINK', 43), ('IFLA_AF_SPEC', {'attrs': [('AF_INET', {'dummy': 65668, 'forwarding': 0, 'mc_forwarding': 0, 'proxy_arp': 0, 'accept_redirects': 1, 'secure_redirects': 1, 'send_redirects': 1, 'shared_media': 1, 'rp_filter': 2, 'accept_source_route': 0, 'bootp_relay': 0, 'log_martians': 0, 'tag': 0, 'arpfilter': 0, 'medium_id': 0, 'noxfrm': 0, 'nopolicy': 0, 'force_igmp_version': 0, 'arp_announce': 0, 'arp_ignore': 0, 'promote_secondaries': 1, 'arp_accept': 0, 'arp_notify': 0, 'accept_local': 0, 'src_vmark': 0, 'proxy_arp_pvlan': 0, 'route_localnet': 0, 'igmpv2_unsolicited_report_interval': 10000, 'igmpv3_unsolicited_report_interval': 1000}), ('AF_INET6', {'attrs': [('IFLA_INET6_FLAGS', 2147483664), ('IFLA_INET6_CACHEINFO', {'max_reasm_len': 65535, 'tstamp': 2631746, 'reachable_time': 34204, 'retrans_time': 1000}), ('IFLA_INET6_CONF', {'forwarding': 0, 'hop_limit': 64, 'mtu': 1500, 'accept_ra': 1, 'accept_redirects': 1, 'autoconf': 1, 'dad_transmits': 1, 'router_solicitations': 4294967295, 'router_solicitation_interval': 4000, 'router_solicitation_delay': 1000, 'use_tempaddr': 0, 'temp_valid_lft': 604800, 'temp_preferred_lft': 86400, 'regen_max_retry': 3, 'max_desync_factor': 600, 'max_addresses': 16, 'force_mld_version': 0, 'accept_ra_defrtr': 1, 'accept_ra_pinfo': 1, 'accept_ra_rtr_pref': 1, 'router_probe_interval': 60000, 'accept_ra_rt_info_max_plen': 0, 'proxy_ndp': 0, 'optimistic_dad': 0, 'accept_source_route': 0, 'mc_forwarding': 0, 'disable_ipv6': 0, 'accept_dad': 1, 'force_tllao': 0, 'ndisc_notify': 0}), ('IFLA_INET6_STATS', {'num': 37, 'inpkts': 0, 'inoctets': 0, 'indelivers': 0, 'outforwdatagrams': 0, 'outpkts': 10, 'outoctets': 696, 'inhdrerrors': 0, 'intoobigerrors': 0, 'innoroutes': 0, 'inaddrerrors': 0, 'inunknownprotos': 0, 'intruncatedpkts': 0, 'indiscards': 0, 'outdiscards': 0, 'outnoroutes': 0, 'reasmtimeout': 0, 'reasmreqds': 0, 'reasmoks': 0, 'reasmfails': 0, 'fragoks': 0, 'fragfails': 0, 'fragcreates': 0, 'inmcastpkts': 0, 'outmcastpkts': 10, 'inbcastpkts': 0, 'outbcastpkts': 0, 'inmcastoctets': 0, 'outmcastoctets': 696, 'inbcastoctets': 0, 'outbcastoctets': 0, 'csumerrors': 0, 'noectpkts': 0, 'ect1pkts': 0, 'ect0pkts': 0, 'cepkts': 0}), ('IFLA_INET6_ICMP6STATS', {'num': 6, 'inmsgs': 0, 'inerrors': 0, 'outmsgs': 10, 'outerrors': 0, 'csumerrors': 0}), ('IFLA_INET6_TOKEN', '::'), ('IFLA_INET6_ADDR_GEN_MODE', 0)]})]})], 'header': {'length': 1428, 'type': 16, 'flags': 0, 'sequence_number': 0, 'pid': 0, 'error': None, 'target': 'localhost', 'stats': Stats(qsize=0, delta=0, delay=0)}, 'state': 'up', 'event': 'RTM_NEWLINK', 'alt_ifname_list': []} netlink event received. 2024-02-22 05:02:29,143 ERROR Error parsing route entry message Traceback (most recent call last): File "/opt/bess/bessctl/conf/route_control.py", line 659, in _parse_route_entry_msg attr_dict = dict(route_entry["attrs"]) ~~~~~~~~~~~^^^^^^^^^ TypeError: string indices must be integers, not 'str' . Sent 1 packets. 2024-02-22 05:02:29,161 INFO {'family': 0, '__align': (), 'ifi_type': 1, 'index': 5, 'flags': 69699, 'change': 256, 'attrs': [('IFLA_IFNAME', 'core'), ('IFLA_TXQLEN', 0), ('IFLA_OPERSTATE', 'UP'), ('IFLA_LINKMODE', 0), ('IFLA_MTU', 1500), ('IFLA_MIN_MTU', 68), ('IFLA_MAX_MTU', 0), ('IFLA_GROUP', 0), ('IFLA_PROMISCUITY', 0), ('IFLA_NUM_TX_QUEUES', 1), ('IFLA_GSO_MAX_SEGS', 65535), ('IFLA_GSO_MAX_SIZE', 65536), ('IFLA_NUM_RX_QUEUES', 1), ('IFLA_CARRIER', 1), ('IFLA_QDISC', 'noqueue'), ('IFLA_CARRIER_CHANGES', 0), ('IFLA_CARRIER_UP_COUNT', 0), ('IFLA_CARRIER_DOWN_COUNT', 0), ('IFLA_PROTO_DOWN', 0), ('IFLA_MAP', {'mem_start': 0, 'mem_end': 0, 'base_addr': 0, 'irq': 0, 'dma': 0, 'port': 0}), ('IFLA_ADDRESS', '5a:a5:0e:4c:e1:0c'), ('IFLA_BROADCAST', 'ff:ff:ff:ff:ff:ff'), ('IFLA_STATS64', {'rx_packets': 8, 'tx_packets': 14, 'rx_bytes': 378, 'tx_bytes': 1004, 'rx_errors': 0, 'tx_errors': 0, 'rx_dropped': 0, 'tx_dropped': 0, 'multicast': 3, 'collisions': 0, 'rx_length_errors': 0, 'rx_over_errors': 0, 'rx_crc_errors': 0, 'rx_frame_errors': 0, 'rx_fifo_errors': 0, 'rx_missed_errors': 0, 'tx_aborted_errors': 0, 'tx_carrier_errors': 0, 'tx_fifo_errors': 0, 'tx_heartbeat_errors': 0, 'tx_window_errors': 0, 'rx_compressed': 0, 'tx_compressed': 0}), ('IFLA_STATS', {'rx_packets': 8, 'tx_packets': 14, 'rx_bytes': 378, 'tx_bytes': 1004, 'rx_errors': 0, 'tx_errors': 0, 'rx_dropped': 0, 'tx_dropped': 0, 'multicast': 3, 'collisions': 0, 'rx_length_errors': 0, 'rx_over_errors': 0, 'rx_crc_errors': 0, 'rx_frame_errors': 0, 'rx_fifo_errors': 0, 'rx_missed_errors': 0, 'tx_aborted_errors': 0, 'tx_carrier_errors': 0, 'tx_fifo_errors': 0, 'tx_heartbeat_errors': 0, 'tx_window_errors': 0, 'rx_compressed': 0, 'tx_compressed': 0}), ('IFLA_XDP', {'attrs': [('IFLA_XDP_ATTACHED', None)]}), ('IFLA_LINKINFO', {'attrs': [('IFLA_INFO_KIND', 'macvlan'), ('IFLA_INFO_DATA', {'attrs': [('IFLA_MACVLAN_MODE', 'bridge'), ('IFLA_MACVLAN_FLAGS', 'none'), ('IFLA_MACVLAN_MACADDR_COUNT', 0), ('UNKNOWN', {'header': {'length': 8, 'type': 7}}), ('UNKNOWN', {'header': {'length': 8, 'type': 8}})], 'header': {}, 'index': 5})]}), ('IFLA_LINK_NETNSID', 0), ('IFLA_LINK', 43), ('IFLA_AF_SPEC', {'attrs': [('AF_INET', {'dummy': 65668, 'forwarding': 0, 'mc_forwarding': 0, 'proxy_arp': 0, 'accept_redirects': 1, 'secure_redirects': 1, 'send_redirects': 1, 'shared_media': 1, 'rp_filter': 2, 'accept_source_route': 0, 'bootp_relay': 0, 'log_martians': 0, 'tag': 0, 'arpfilter': 0, 'medium_id': 0, 'noxfrm': 0, 'nopolicy': 0, 'force_igmp_version': 0, 'arp_announce': 0, 'arp_ignore': 0, 'promote_secondaries': 1, 'arp_accept': 0, 'arp_notify': 0, 'accept_local': 0, 'src_vmark': 0, 'proxy_arp_pvlan': 0, 'route_localnet': 0, 'igmpv2_unsolicited_report_interval': 10000, 'igmpv3_unsolicited_report_interval': 1000}), ('AF_INET6', {'attrs': [('IFLA_INET6_FLAGS', 2147483664), ('IFLA_INET6_CACHEINFO', {'max_reasm_len': 65535, 'tstamp': 2631746, 'reachable_time': 34204, 'retrans_time': 1000}), ('IFLA_INET6_CONF', {'forwarding': 0, 'hop_limit': 64, 'mtu': 1500, 'accept_ra': 1, 'accept_redirects': 1, 'autoconf': 1, 'dad_transmits': 1, 'router_solicitations': 4294967295, 'router_solicitation_interval': 4000, 'router_solicitation_delay': 1000, 'use_tempaddr': 0, 'temp_valid_lft': 604800, 'temp_preferred_lft': 86400, 'regen_max_retry': 3, 'max_desync_factor': 600, 'max_addresses': 16, 'force_mld_version': 0, 'accept_ra_defrtr': 1, 'accept_ra_pinfo': 1, 'accept_ra_rtr_pref': 1, 'router_probe_interval': 60000, 'accept_ra_rt_info_max_plen': 0, 'proxy_ndp': 0, 'optimistic_dad': 0, 'accept_source_route': 0, 'mc_forwarding': 0, 'disable_ipv6': 0, 'accept_dad': 1, 'force_tllao': 0, 'ndisc_notify': 0}), ('IFLA_INET6_STATS', {'num': 37, 'inpkts': 0, 'inoctets': 0, 'indelivers': 0, 'outforwdatagrams': 0, 'outpkts': 10, 'outoctets': 696, 'inhdrerrors': 0, 'intoobigerrors': 0, 'innoroutes': 0, 'inaddrerrors': 0, 'inunknownprotos': 0, 'intruncatedpkts': 0, 'indiscards': 0, 'outdiscards': 0, 'outnoroutes': 0, 'reasmtimeout': 0, 'reasmreqds': 0, 'reasmoks': 0, 'reasmfails': 0, 'fragoks': 0, 'fragfails': 0, 'fragcreates': 0, 'inmcastpkts': 0, 'outmcastpkts': 10, 'inbcastpkts': 0, 'outbcastpkts': 0, 'inmcastoctets': 0, 'outmcastoctets': 696, 'inbcastoctets': 0, 'outbcastoctets': 0, 'csumerrors': 0, 'noectpkts': 0, 'ect1pkts': 0, 'ect0pkts': 0, 'cepkts': 0}), ('IFLA_INET6_ICMP6STATS', {'num': 6, 'inmsgs': 0, 'inerrors': 0, 'outmsgs': 10, 'outerrors': 0, 'csumerrors': 0}), ('IFLA_INET6_TOKEN', '::'), ('IFLA_INET6_ADDR_GEN_MODE', 0)]})]})], 'header': {'length': 1428, 'type': 16, 'flags': 0, 'sequence_number': 0, 'pid': 0, 'error': None, 'target': 'localhost', 'stats': Stats(qsize=0, delta=0, delay=0)}, 'state': 'up', 'event': 'RTM_NEWLINK', 'alt_ifname_list': []} netlink event received. 2024-02-22 05:02:29,163 ERROR Error parsing route entry message Traceback (most recent call last): File "/opt/bess/bessctl/conf/route_control.py", line 659, in _parse_route_entry_msg attr_dict = dict(route_entry["attrs"]) ~~~~~~~~~~~^^^^^^^^^ TypeError: string indices must be integers, not 'str'

…interface

Replace deprecated IPDB with NDB

3a22664

gab-arrobo mentioned this pull request Feb 5, 2024

Replace IPDB with NDB in UPF #735

Closed

gab-arrobo force-pushed the replace-deprecated-ipdb branch from 055eec5 to 3a22664 Compare February 6, 2024 00:28

ghislainbourgeois suggested changes Feb 7, 2024

View reviewed changes

gab-arrobo added 2 commits February 6, 2024 18:09

Merge branch 'master' into replace-deprecated-ipdb

16cc53a

Update test file

5664b7b

gruyaume reviewed Feb 7, 2024

View reviewed changes

conf/route_control.py Outdated Show resolved Hide resolved

ghislainbourgeois suggested changes Feb 7, 2024

View reviewed changes

conf/test_route_control.py Outdated Show resolved Hide resolved

conf/test_route_control.py Outdated Show resolved Hide resolved

gab-arrobo changed the title ~~Replace deprecated IPDB with NDB~~ Replace deprecated and obsoleted IPDB with NDB Feb 7, 2024

gab-arrobo and others added 2 commits February 7, 2024 12:39

Apply suggestions from code review

1a4a15f

Co-authored-by: Ghislain Bourgeois <[email protected]>

Use get method

526d168

gruyaume approved these changes Feb 7, 2024

View reviewed changes

gab-arrobo added 2 commits February 7, 2024 12:15

Merge branch 'master' into replace-deprecated-ipdb

e656a2e

Merge branch 'omec-project:master' into replace-deprecated-ipdb

7c696da

gab-arrobo requested a review from ghislainbourgeois February 7, 2024 21:46

No need to register callbacks because NDB continuously monitors the s…

16f68db

…ystem for changes

gab-arrobo requested a review from gruyaume February 7, 2024 22:02

ghislainbourgeois suggested changes Feb 7, 2024

View reviewed changes

gab-arrobo added 3 commits February 15, 2024 15:25

Merge branch 'omec-project:master' into replace-deprecated-ipdb

55de595

Merge branch 'omec-project:master' into replace-deprecated-ipdb

2e05e4f

Partially address issue with registrer_handler

094e687

gab-arrobo force-pushed the replace-deprecated-ipdb branch from 058935a to 094e687 Compare February 22, 2024 04:56

Update route control script. Still need to address issue with core …

56b95ba

…interface

gab-arrobo force-pushed the replace-deprecated-ipdb branch from c486faf to 56b95ba Compare February 22, 2024 17:30

gab-arrobo added 2 commits February 27, 2024 15:26

Merge branch 'omec-project:master' into replace-deprecated-ipdb

0982fb7

Merge branch 'omec-project:master' into replace-deprecated-ipdb

25b1a1d

gab-arrobo added 13 commits March 6, 2024 12:13

Merge branch 'omec-project:master' into replace-deprecated-ipdb

1da2ecd

Merge branch 'omec-project:master' into replace-deprecated-ipdb

a390989

Merge branch 'omec-project:master' into replace-deprecated-ipdb

824ed04

Merge branch 'omec-project:master' into replace-deprecated-ipdb

6b7b490

Merge branch 'omec-project:master' into replace-deprecated-ipdb

33c5cbd

Merge branch 'omec-project:master' into replace-deprecated-ipdb

5ff4806

Merge branch 'omec-project:master' into replace-deprecated-ipdb

011dd93

Merge branch 'omec-project:master' into replace-deprecated-ipdb

a31d6fa

Merge branch 'omec-project:master' into replace-deprecated-ipdb

68e9893

Merge branch 'omec-project:master' into replace-deprecated-ipdb

690531e

Merge branch 'omec-project:master' into replace-deprecated-ipdb

748412e

Merge branch 'omec-project:master' into replace-deprecated-ipdb

c035ec0

Merge branch 'omec-project:master' into replace-deprecated-ipdb

5cde3ce

gatici mentioned this pull request Jul 16, 2024

fix: Use ndb and arrange the methods #830

Merged

gab-arrobo closed this Jul 19, 2024

gab-arrobo deleted the replace-deprecated-ipdb branch July 24, 2024 23:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace deprecated and obsoleted `IPDB` with `NDB` #758

Replace deprecated and obsoleted `IPDB` with `NDB` #758

gab-arrobo commented Feb 5, 2024 •

edited

Loading

gab-arrobo commented Feb 6, 2024 •

edited

Loading

ghislainbourgeois commented Feb 7, 2024

ghislainbourgeois left a comment

gab-arrobo commented Feb 7, 2024

gruyaume left a comment

gab-arrobo commented Feb 7, 2024

ghislainbourgeois Feb 7, 2024

gab-arrobo Feb 8, 2024

saltiyazan Feb 8, 2024

gab-arrobo Feb 8, 2024

gab-arrobo Feb 8, 2024

saltiyazan Feb 9, 2024

ghislainbourgeois Feb 9, 2024

gab-arrobo Feb 22, 2024

Replace deprecated and obsoleted IPDB with NDB #758

Replace deprecated and obsoleted IPDB with NDB #758

Conversation

gab-arrobo commented Feb 5, 2024 • edited Loading

gab-arrobo commented Feb 6, 2024 • edited Loading

ghislainbourgeois commented Feb 7, 2024

ghislainbourgeois left a comment

Choose a reason for hiding this comment

gab-arrobo commented Feb 7, 2024

gruyaume left a comment

Choose a reason for hiding this comment

gab-arrobo commented Feb 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Replace deprecated and obsoleted `IPDB` with `NDB` #758

Replace deprecated and obsoleted `IPDB` with `NDB` #758

gab-arrobo commented Feb 5, 2024 •

edited

Loading

gab-arrobo commented Feb 6, 2024 •

edited

Loading