Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VXLAN tunnel bogus error unexpected type: SAI_OBJECT_TYPE_TUNNEL #1467

Closed
wants to merge 1 commit into from

Conversation

bradh352
Copy link

When trying to debug VXLAN tunnels it is bad to have errors output in the logs that are false positives. This can lead users to go down a rabbit hole when the error has nothing to do with the issue at hand.

Here is the error sequence:

Aug 16 03:14:19.373668 Odin-ec58a ERR swss#orchagent: :- meta_sai_on_port_state_change_single: data.port_id oid:0x2a0000000009bf has unexpected type: SAI_OBJECT_TYPE_TUNNEL, expected PORT, BRIDGE_PORT or LAG
Aug 16 03:14:19.375495 Odin-ec58a NOTICE swss#orchagent: :- doTask: Get port state change notification id:2a0000000009bf status:1
Aug 16 03:14:19.375495 Odin-ec58a ERR swss#orchagent: :- doTask: Failed to get port object for port id 0x2a0000000009bf

In this case, as reported in sonic-net/sonic-buildimage#10004 the fix is to simply add the SAI_OBJECT_TYPE_TUNNEL to the switch statement.

This is a trivial correction for an incorrect log entry. It doesn't appear to alter behavior in any way.

It has been confirmed via my own Dell S5248F switches to resolve this false-positive error message.

Signed-off-by: Brad House (@bradh352)

@bradh352 bradh352 changed the title VXLAN tunnel bogus error correction VXLAN tunnel bogus error unexpected type: SAI_OBJECT_TYPE_TUNNEL Nov 20, 2024
@kcudnik
Copy link
Collaborator

kcudnik commented Nov 21, 2024

tunnel object is not expected there: https://github.com/opencomputeproject/SAI/blob/master/inc/saiport.h#L137
according to latest SAI on master only 3 objects are expected at that point, if tunnel is als oexpected teher, please bring that issue on SAI comunity meeting and post PR on SAI repo to update that metadata

Copy link
Collaborator

@kcudnik kcudnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to comments

@@ -6702,6 +6702,7 @@ void Meta::meta_sai_on_port_state_change_single(
case SAI_OBJECT_TYPE_PORT:
case SAI_OBJECT_TYPE_BRIDGE_PORT:
case SAI_OBJECT_TYPE_LAG:
case SAI_OBJECT_TYPE_TUNNEL:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this code should be based on sai metadata, and was written when there was no metadata for those objecs yet, if you want to fix it, please add it using metadata checks

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i fixed it here: #1469

@bradh352
Copy link
Author

@kcudnik

I don't intend on adding a new metadata port type to track by any means. I was just trying to silence that warning as it inhibits the ability of someone attempting to debug VXLAN issues. I guess I didn't fully evaluate what that warning was about.

So it sounds like the better thing to do is to find where that event originates and filter out SAI_OBJECT_TYPE_TUNNEL so it never tries to send a port metadata update using that object type.

Any idea off the top of your head where to do that? I've only been playing with SONiC for about 2 weeks, so I'm still quite green.

@kcudnik
Copy link
Collaborator

kcudnik commented Nov 21, 2024

if onject type tunnel is send in notification and by current SAI headers this is not allowed, then this seems like vendor bug, otherwise SAI metadat should be updated, but not in this PR, im meking generic fix for this and i will post it shortly

@kcudnik
Copy link
Collaborator

kcudnik commented Nov 21, 2024

if this iss issue for you you probably can post new issue on SAI https://github.com/opencomputeproject/SAI/issues and explain the origin

@bradh352
Copy link
Author

call chain appears to be Sai::handle_notification() -> RedisRemoteSaiInterface::syncProcessNotification() -> NotificationPortStateChange::processMetadata() -> Meta::meta_sai_on_port_state_change() -> Meta::meta_sai_on_port_state_change_single()

Sai::apiInitialize() makes a context which calls this notification callback.

The chain is a little harder to follow upstream from there. I'm assuming its being called from a different project, maybe swss->orchagent->portsarch ?

@bradh352
Copy link
Author

ah, i see you said you'll make a generic fix for that. Thanks.

And I also see you say its probably a vendor bug, so I guess that's bcm sai sending something it shouldn't is what you're saying.

@kcudnik
Copy link
Collaborator

kcudnik commented Nov 21, 2024

maybe tunnel is also valid for that notification and metadata was not updated, that i dont know

fixed this part to use metadata: #1469

@@ -6702,6 +6702,7 @@ void Meta::meta_sai_on_port_state_change_single(
case SAI_OBJECT_TYPE_PORT:
case SAI_OBJECT_TYPE_BRIDGE_PORT:
case SAI_OBJECT_TYPE_LAG:
case SAI_OBJECT_TYPE_TUNNEL:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i fixed it here: #1469

@kcudnik
Copy link
Collaborator

kcudnik commented Nov 21, 2024

will not merge this becaue of above fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants