Skip to content

Conversation

5ur3
Copy link
Member

@5ur3 5ur3 commented Jul 8, 2025

Description

This issue arose during the implementation of a RED FEC decoder interceptor. A sender can announce both an original codec (Opus, in this example) and RED in the SDP as follows:

a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10;useinbandfec=1;usedtx=1
a=rtpmap:63 red/48000/2
a=fmtp:63 111/111

Following this announcement, the sender may or may not use RED for audio transmission. Furthermore, the sender can dynamically switch between unprotected Opus and RED-protected streams during a session.

Currently, the receiver's interceptor has no way to determine the MIME type of a received packet. The StreamInfo structure only contains the capabilities of the first negotiated codec for the media section, which is not always the codec being used for transmission.

This PR resolves this ambiguity by adding a PayloadToMimeType map to StreamInfo. This allows decoder interceptors to match a received packet's payload type to its corresponding MIME type. For instance, with this change, the RED FEC decoder interceptor can now reliably differentiate between unprotected audio packets and RED-protected audio packets.

Implementation PR

The PayloadToMimeType field is set within the pion/webrtc repository. The work is handled by the following pull request: pion/webrtc#3175

This corresponding PR is currently a draft because it depends on the changes from this PR. Once this is merged, the pion/webrtc PR will be updated with a commit that bumps its dependency on this repository to include these changes, and it can then be finalized for review.

Copy link

codecov bot commented Jul 8, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.46%. Comparing base (7be3ed2) to head (0b40e43).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #346      +/-   ##
==========================================
+ Coverage   76.00%   78.46%   +2.45%     
==========================================
  Files          81       81              
  Lines        5130     5130              
==========================================
+ Hits         3899     4025     +126     
+ Misses       1042      930     -112     
+ Partials      189      175      -14     
Flag Coverage Δ
go 78.46% <ø> (?)
wasm 76.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JoeTurki
Copy link
Member

JoeTurki commented Jul 8, 2025

Hello @5ur3 Thank you for the work you do, Let me know when you need review for this PR, also I invited you to the org so you can make branches directly, and run the CI without waiting for approvals.

@5ur3
Copy link
Member Author

5ur3 commented Jul 8, 2025

@JoeTurki Thanks! This PR is ready to be reviewed

Copy link
Member

@JoeTurki JoeTurki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, I added a comment in the other PR, maybe there is a way to handle this using another fix? pion/webrtc#3175 (review)

@5ur3
Copy link
Member Author

5ur3 commented Jul 8, 2025

Let's continue the discussion in pion/webrtc#3175 (review)

@5ur3 5ur3 changed the title Add negotiated codec parameters to StreamInfo Add PayloadToMimeType map to StreamInfo Jul 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants