Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Removing ip address from interface and immediately adding same interface under Portchannel cause setPortPvid error #16633

Open
kbabujp opened this issue Jan 22, 2025 · 0 comments
Open

[Bug]: Removing ip address from interface and immediately adding same interface under Portchannel cause setPortPvid error #16633

kbabujp opened this issue Jan 22, 2025 · 0 comments

Comments

@kbabujp
Copy link
Contributor

kbabujp commented Jan 22, 2025 •
edited
Loading

Issue Description

In sub_port_interfaces/test_sub_port_interfaces.py::TestSubPorts::test_routing_between_sub_ports_and_port, it is removing ip addresses from interface and adding the port to port channel immediately. By the time L3 related config removed from the given port, its added to Portchannel and setPortPvid is failing.
 
Sub port module tests are reporting this error:
========================================================================================================= short test summary info =========================================================================================================PASSED sub_port_interfaces/test_sub_port_interfaces.py::TestSubPorts::test_routing_between_sub_ports_and_port[port_in_lag-svi-TCP-UDP-ICMP]
ERROR sub_port_interfaces/test_sub_port_interfaces.py::TestSubPorts::test_routing_between_sub_ports_and_port[port_in_lag-svi-TCP-UDP-ICMP] - Failed: Processes "['analyze_logs--']" failed with exit code "1"
 
 
If i add a delay between removing ip address and adding the interface to Portchannel this error not seen.
The ideal way is to wait till all L3 related config should be removed before adding it to Portchannel/vlan.
 
syslog.7.gz:2025 Jan 20 09:13:05.720443 sonic-dut ERR swss#orchagent: :- setPortPvid: pvid setting for router interface Ethernet0 is not allowed
 
sub_port_interfaces/sub_ports_helpers.py  635         if lag_port not in portchannels:  636             port_name = port_list[port_list_idx]  637             remove_ip_from_port(duthost, port_name)  638             #time.sleep(5)  639             remove_member_from_vlan(duthost, '1000', port_name)  640             cmds.append('config portchannel add {}'.format(lag_port))  641             cmds.append('config portchannel member add {} {}'.format(lag_port, port_name))  642             lag_port_map[portchannel_idx] = lag_port  643             port_list_idx += 1

Results you see

PASSED sub_port_interfaces/test_sub_port_interfaces.py::TestSubPorts::test_routing_between_sub_ports_and_port[port_in_lag-svi-TCP-UDP-ICMP]
ERROR sub_port_interfaces/test_sub_port_interfaces.py::TestSubPorts::test_routing_between_sub_ports_and_port[port_in_lag-svi-TCP-UDP-ICMP] - Failed: Processes "['analyze_logs--']" failed with exit code "1"

Results you expected to see

Error should not be seen while adding interface to Portchannel.

Is it platform specific

generic

Relevant log output

Syslog:
2025 Jan 20 14:09:03.497375 sonic-dut INFO python[221767]: ansible-ansible.legacy.command Invoked with _raw_params=config interface ip remove Ethernet16 10.0.0.4/31 _uses_shell=True warn=False stdin_add_newline=True strip_empty_ends=True argv=None chdir=None executable=None creates=None removes=None stdin=None
2025 Jan 20 14:09:04.019621 sonic-dut DEBUG bgp#bgpcfgd: Received message : '('Ethernet16|10.0.0.4/31', 'DEL', ())'
2025 Jan 20 14:09:04.019657 sonic-dut DEBUG bgp#staticroutebfd: Received message : '('Ethernet16|10.0.0.4/31', 'DEL', ())'
2025 Jan 20 14:09:04.019851 sonic-dut DEBUG bgp#staticroutebfd: get ip from intf key: valid True is_ipv4 True, if_name Ethernet16 ip 10.0.0.4
2025 Jan 20 14:09:04.023041 sonic-dut DEBUG bgp#bgpcfgd: Received message : '('Ethernet16|10.0.0.4/31', 'DEL', ())'
2025 Jan 20 14:09:04.023090 sonic-dut WARNING bgp#bgpcfgd: Delete key 'Ethernet16|10.0.0.4/31' is not supported for 'zebra set src' templates
2025 Jan 20 14:09:04.024728 sonic-dut NOTICE swss#orchagent: :- removeIp2MeRoute: Remove packet action trap route ip:10.0.0.4
2025 Jan 20 14:09:04.032387 sonic-dut NOTICE bgp#fpmsyncd: :- onRouteMsg: RouteTable del msg for route with only one nh on eth0/docker0: fd00::/80 :: docker0
2025 Jan 20 14:09:04.238418 sonic-dut NOTICE swss#orchagent: :- addNextHopGroup: Create next hop group 10.0.0.7@Ethernet24,10.0.0.9@Ethernet32,10.0.0.11@Ethernet40,10.0.0.13@Ethernet48,10.0.0.15@Ethernet56,10.0.0.17@Ethernet64,10.0.0.19@Ethernet72,10.0.0.21@Ethernet80,10.0.0.23@Ethernet88,10.0.0.25@Ethernet96,10.0.0.27@Ethernet104,10.0.0.29@Ethernet112,10.0.0.31@Ethernet120
2025 Jan 20 14:09:04.839230 sonic-dut INFO python[221779]: ansible-ansible.legacy.command Invoked with _raw_params=config interface ip remove Ethernet16 FC00::9/126 _uses_shell=True warn=False stdin_add_newline=True strip_empty_ends=True argv=None chdir=None executable=None creates=None removes=None stdin=None
2025 Jan 20 14:09:05.770826 sonic-dut DEBUG bgp#bgpcfgd: Received message : '('Ethernet16|FC00::9/126', 'DEL', ())'
2025 Jan 20 14:09:05.770887 sonic-dut DEBUG bgp#staticroutebfd: Received message : '('Ethernet16|FC00::9/126', 'DEL', ())'
2025 Jan 20 14:09:05.770887 sonic-dut DEBUG bgp#staticroutebfd: get ip from intf key: valid True is_ipv4 False, if_name Ethernet16 ip FC00::9
2025 Jan 20 14:09:05.776020 sonic-dut DEBUG bgp#bgpcfgd: Received message : '('Ethernet16|FC00::9/126', 'DEL', ())'
2025 Jan 20 14:09:05.776053 sonic-dut WARNING bgp#bgpcfgd: Delete key 'Ethernet16|FC00::9/126' is not supported for 'zebra set src' templates
2025 Jan 20 14:09:05.778455 sonic-dut DEBUG bgp#bgpcfgd: Received message : '('Ethernet16', 'DEL', ())'
2025 Jan 20 14:09:05.778496 sonic-dut DEBUG bgp#staticroutebfd: Received message : '('Ethernet16', 'DEL', ())'
2025 Jan 20 14:09:05.783408 sonic-dut NOTICE bgp#fpmsyncd: :- onRouteMsg: RouteTable del msg for route with only one nh on eth0/docker0: fd00::/80 :: docker0
2025 Jan 20 14:09:05.787851 sonic-dut DEBUG bgp#bgpcfgd: Received message : '('Ethernet16', 'DEL', ())'
2025 Jan 20 14:09:05.787851 sonic-dut WARNING bgp#bgpcfgd: Delete key 'Ethernet16' is not supported for 'zebra set src' templates
2025 Jan 20 14:09:05.907070 sonic-dut NOTICE swss#orchagent: :- removeIp2MeRoute: Remove packet action trap route ip:fc00::9
2025 Jan 20 14:09:05.907070 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 5
2025 Jan 20 14:09:06.531638 sonic-dut INFO python[221856]: ansible-ansible.legacy.command Invoked with _raw_params=redis-cli -n 4 keys "VLAN_MEMBER|Vlan1000|Ethernet16" _uses_shell=True warn=False stdin_add_newline=True strip_empty_ends=True argv=None chdir=None executable=None creates=None removes=None stdin=None
2025 Jan 20 14:09:06.909253 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 5
2025 Jan 20 14:09:06.909253 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4
2025 Jan 20 14:09:07.082062 sonic-dut NOTICE swss#orchagent: message repeated 4 times: [ :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4]
2025 Jan 20 14:09:07.082062 sonic-dut NOTICE swss#orchagent: :- addNextHopGroup: Create next hop group fc00::e@Ethernet24,fc00::12@Ethernet32,fc00::16@Ethernet40,fc00::1a@Ethernet48,fc00::1e@Ethernet56,fc00::22@Ethernet64,fc00::26@Ethernet72,fc00::2a@Ethernet80,fc00::2e@Ethernet88,fc00::32@Ethernet96,fc00::36@Ethernet104,fc00::3a@Ethernet112,fc00::3e@Ethernet120
2025 Jan 20 14:09:07.229558 sonic-dut NOTICE syncd#syncd: message repeated 7 times: [ :- updateSupportedCounters: Ignore checking of supported counters]
2025 Jan 20 14:09:07.229558 sonic-dut NOTICE syncd#syncd: :- inc: 10173 (calls 36) Syncd::syncUpdateRedisBulkQuadEvent op took: 438 ms
2025 Jan 20 14:09:07.415921 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4
2025 Jan 20 14:09:09.279092 sonic-dut INFO python[221890]: ansible-shell_cmds Invoked with cmds=['config portchannel add PortChannel3', 'config portchannel member add PortChannel3 Ethernet16'] continue_on_fail=True timeout=0
2025 Jan 20 14:09:09.915765 sonic-dut INFO teamd#supervisord: teammgrd Using team device "PortChannel3".
2025 Jan 20 14:09:09.915765 sonic-dut INFO teamd#supervisord: teammgrd Using PID file "/var/run/teamd/PortChannel3.pid"
2025 Jan 20 14:09:09.915801 sonic-dut INFO teamd#supervisord: teammgrd This program is not intended to be run as root.
2025 Jan 20 14:09:09.919685 sonic-dut NOTICE swss#orchagent: message repeated 9 times: [ :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4]
2025 Jan 20 14:09:09.919685 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:PortChannel3 admin:0 oper:0 addr:da:f0:b2:23:00:c6 ifindex:948 master:0 type:team
2025 Jan 20 14:09:09.923857 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:PortChannel3 admin:0 oper:0 addr:da:f0:b2:23:00:c6 ifindex:948 master:0 type:team
2025 Jan 20 14:09:09.923857 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:PortChannel3 admin:0 oper:0 addr:00:e0:4b:77:0d:63 ifindex:948 master:0 type:team
2025 Jan 20 14:09:09.928954 sonic-dut INFO kernel: [24996.064441] PortChannel3: Mode changed to "loadbalance"
2025 Jan 20 14:09:09.931556 sonic-dut NOTICE bgp#fpmsyncd: :- onRouteMsg: RouteTable del msg for route with only one nh on eth0/docker0: fd00::/80 :: docker0
2025 Jan 20 14:09:09.932183 sonic-dut INFO (udev-worker)[221900]: Network interface NamePolicy= disabled on kernel command line.
2025 Jan 20 14:09:09.933268 sonic-dut NOTICE teamd#teammgrd: :- addLag: Start port channel PortChannel3 with teamd
2025 Jan 20 14:09:09.936244 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:PortChannel3 admin:1 oper:0 addr:00:e0:4b:77:0d:63 ifindex:948 master:0 type:team
2025 Jan 20 14:09:09.936957 sonic-dut INFO kernel: [24996.074966] 8021q: adding VLAN 0 to HW filter on device PortChannel3
2025 Jan 20 14:09:09.943343 sonic-dut NOTICE teamd#teammgrd: :- setLagAdminStatus: Set port channel PortChannel3 admin status to up
2025 Jan 20 14:09:09.943343 sonic-dut NOTICE teamd#tlm_teamd: :- try_add_lag: The LAG 'PortChannel3' has been added.
2025 Jan 20 14:09:09.951913 sonic-dut NOTICE teamd#teammgrd: :- setLagMtu: Set port channel PortChannel3 MTU to 9100
2025 Jan 20 14:09:09.953755 sonic-dut NOTICE bgp#fpmsyncd: :- onRouteMsg: RouteTable del msg for route with only one nh on eth0/docker0: fd00::/80 :: docker0
2025 Jan 20 14:09:10.086317 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:PortChannel3 admin:1 oper:0 addr:00:e0:4b:77:0d:63 ifindex:948 master:0 type:team
2025 Jan 20 14:09:10.086317 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4
2025 Jan 20 14:09:10.087141 sonic-dut INFO swss#supervisord: orchagent
2025 Jan 20 14:09:10.088496 sonic-dut NOTICE swss#orchagent: :- addLag: Create an empty LAG PortChannel3 lid:2000000000e0a
2025 Jan 20 14:09:10.088860 sonic-dut NOTICE swss#orchagent: :- updatePortOperStatus: Port PortChannel3 oper state set from unknown to down
2025 Jan 20 14:09:10.088878 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4
2025 Jan 20 14:09:10.786126 sonic-dut NOTICE swss#orchagent: message repeated 3 times: [ :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4]
2025 Jan 20 14:09:10.786126 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:Ethernet16 admin:0 oper:0 addr:00:e0:4b:77:0d:63 ifindex:890 master:0
2025 Jan 20 14:09:10.786500 sonic-dut NOTICE swss#portsyncd: :- onMsg: Publish Ethernet16(ok:down) to state db
2025 Jan 20 14:09:10.796723 sonic-dut NOTICE bgp#fpmsyncd: :- onRouteMsg: RouteTable del msg for route with only one nh on eth0/docker0: fe80::/64 :: eth0
2025 Jan 20 14:09:10.796911 sonic-dut NOTICE bgp#fpmsyncd: :- onRouteMsg: RouteTable del msg for route with only one nh on eth0/docker0: fd00::/80 :: docker0
2025 Jan 20 14:09:10.893856 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:Ethernet16 admin:1 oper:1 addr:00:e0:4b:77:0d:63 ifindex:890 master:0
2025 Jan 20 14:09:10.894151 sonic-dut NOTICE swss#portsyncd: :- onMsg: Publish Ethernet16(ok:up) to state db
2025 Jan 20 14:09:10.895905 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:Ethernet16 admin:1 oper:1 addr:00:e0:4b:77:0d:63 ifindex:890 master:948
2025 Jan 20 14:09:10.896387 sonic-dut NOTICE swss#portsyncd: :- onMsg: Publish Ethernet16(ok:up) to state db
2025 Jan 20 14:09:10.896522 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:Ethernet16 admin:1 oper:1 addr:00:e0:4b:77:0d:63 ifindex:890 master:948
2025 Jan 20 14:09:10.897533 sonic-dut INFO kernel: [24997.034234] PortChannel3: Port device Ethernet16 added
2025 Jan 20 14:09:10.898086 sonic-dut NOTICE swss#portsyncd: :- onMsg: Publish Ethernet16(ok:up) to state db
2025 Jan 20 14:09:10.898086 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:Ethernet16 admin:1 oper:1 addr:00:e0:4b:77:0d:63 ifindex:890 master:948
2025 Jan 20 14:09:10.898361 sonic-dut NOTICE swss#portsyncd: :- onMsg: Publish Ethernet16(ok:up) to state db
2025 Jan 20 14:09:10.898539 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:PortChannel3 admin:1 oper:0 addr:00:e0:4b:77:0d:63 ifindex:948 master:0 type:team
2025 Jan 20 14:09:10.903114 sonic-dut NOTICE teamd#teammgrd: :- addLagMember: Add Ethernet16 to port channel PortChannel3
2025 Jan 20 14:09:10.904582 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:Ethernet16 admin:1 oper:1 addr:00:e0:4b:77:0d:63 ifindex:890 master:948
2025 Jan 20 14:09:10.906173 sonic-dut NOTICE swss#portsyncd: :- onMsg: Publish Ethernet16(ok:up) to state db
2025 Jan 20 14:09:10.906173 sonic-dut NOTICE swss#portsyncd: :- onMsg: nlmsg type:16 key:Ethernet16 admin:1 oper:1 addr:00:e0:4b:77:0d:63 ifindex:890 master:948
2025 Jan 20 14:09:10.906173 sonic-dut NOTICE swss#portsyncd: :- onMsg: Publish Ethernet16(ok:up) to state db
2025 Jan 20 14:09:10.911787 sonic-dut NOTICE bgp#fpmsyncd: :- onRouteMsg: RouteTable del msg for route with only one nh on eth0/docker0: fe80::/64 :: eth0
2025 Jan 20 14:09:10.912539 sonic-dut NOTICE bgp#fpmsyncd: :- onRouteMsg: RouteTable del msg for route with only one nh on eth0/docker0: fd00::/80 :: docker0
2025 Jan 20 14:09:11.099517 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4
2025 Jan 20 14:09:11.101556 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4
2025 Jan 20 14:09:11.101556 sonic-dut ERR swss#orchagent: :- setPortPvid: pvid setting for router interface Ethernet16 is not allowed
2025 Jan 20 14:09:11.103461 sonic-dut NOTICE swss#orchagent: :- addLagMember: Add member Ethernet16 to LAG PortChannel3 lid:2000000000e0a pid:1000000000259
2025 Jan 20 14:09:11.104148 sonic-dut NOTICE swss#orchagent: :- setDistributionOnLagMember: Disable distribution on LAG member Ethernet16
2025 Jan 20 14:09:11.104798 sonic-dut NOTICE swss#orchagent: :- setCollectionOnLagMember: Disable collection on LAG member Ethernet16
2025 Jan 20 14:09:11.104863 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4
2025 Jan 20 14:09:11.106222 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4
2025 Jan 20 14:09:11.106222 sonic-dut NOTICE swss#orchagent: :- updatePortOperStatus: Port PortChannel3 oper state set from down to down
2025 Jan 20 14:09:11.106275 sonic-dut NOTICE swss#orchagent: :- removeRouterIntfs: Router interface Ethernet16 is still referenced with ref count 4

Output of show version

SONiC Software Version: SONiC.202405.0-dirty-20241128.225021
SONiC OS Version: 12
Distribution: Debian 12.8
Kernel: 6.1.0-22-2-amd64
Build commit: 29cbc5423
Build date: Fri Nov 29 06:51:17 UTC 2024

Attach files (if any)

No response
@kbabujp kbabujp changed the title [Bug]: Removing ip address from interface and immediately adding same interface under vlan cause setPortPvid error [Bug]: Removing ip address from interface and immediately adding same interface under Portchannel cause setPortPvid error Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kbabujp