Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix addCodec to return error if payload type exists in codecs list #3016

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

itzmanish
Copy link
Contributor

Description

This PR adds changes to return an error if the codec already exists on the codecs list. This way now the implementing party should take care of duplicate codecs addition if the payload type is the same.

Reference issue

Fixes #3015

Copy link

codecov bot commented Jan 22, 2025

Codecov Report

Attention: Patch coverage is 81.48148% with 5 lines in your changes missing coverage. Please review.

Project coverage is 78.22%. Comparing base (44062a7) to head (a34cc7e).

Files with missing lines Patch % Lines
mediaengine.go 81.48% 3 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3016      +/-   ##
==========================================
- Coverage   78.41%   78.22%   -0.20%     
==========================================
  Files          91       91              
  Lines       11251    11262      +11     
==========================================
- Hits         8823     8810      -13     
- Misses       1938     1956      +18     
- Partials      490      496       +6     
Flag Coverage Δ
go 80.08% <81.48%> (-0.22%) ⬇️
wasm 63.44% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@itzmanish
Copy link
Contributor Author

While this PR fixes the issue of duplicate payload type codec entry on RegisterCodec, There is still one issue that this PR doesn't cover.

What if we have some registered codecs and the client sends the codecs with different payload types and one of the payload types the client is using clashes with our registered codecs, in that case, the client's codec won't be added to the negotiated codecs list. That's an issue because the server will now be missing that codec.

The flow -

Pion server codec list - []{..,PayloadType: 96, Mime: "video/VP8"..}
Browser sends codecs on sdp which triggers `updateFromRemoteDescription` - []{..., PayloadType: 96, Mime: "audio/OPUS"...}

This PR will make the media engine discard the opus codec. However, the issue remains even without this PR,

Pion server codec list - []{..,PayloadType: 96, Mime: "video/VP9", "profile-id=0"..}
Browser sends codecs on sdp which triggers `updateFromRemoteDescription` - []{..., PayloadType: 96, Mime: "video/VP9", "profile-id=1"...}

Again the client-given codec won't be added and most probably the browser will use "profile-id=0" one only (I am not sure about specs here, so please correct me here).

@JoeTurki
Copy link
Member

JoeTurki commented Jan 22, 2025

@itzmanish Thank you so much, Will you be able to fix the lint issues (commit and code?).

What if we have some registered codecs and the client sends the codecs with different payload types and one of the payload types the client is using clashes with our registered codecs, in that case, the client's codec won't be added to the negotiated codecs list. That's an issue because the server will now be missing that codec.

For your use cause (SFU) you will need to manage separate payload type mappings for the send and receive directions in certain cases, as outlined in rfc3264

For recvonly RTP streams, the payload type numbers indicate the value
of the payload type field in RTP packets the offerer is expecting to
receive for that codec. For sendonly RTP streams, the payload type
numbers indicate the value of the payload type field in RTP packets
the offerer is planning to send for that codec. For sendrecv RTP
streams, the payload type numbers indicate the value of the payload
type field the offerer expects to receive, and would prefer to send.
However, for sendonly and sendrecv streams, the answer might indicate
different payload type numbers for the same codecs, in which case,
the offerer MUST send with the payload type numbers from the answer.
Different payload type numbers may be needed in each direction
because of interoperability concerns with H.323.

Worth mentioning rfc3551

This profile reserves payload type numbers in the range 96-127
exclusively for dynamic assignment. Applications SHOULD first use
values in this range for dynamic payload types. Those applications
which need to define more than 32 dynamic payload types MAY bind
codes below 96, in which case it is RECOMMENDED that unassigned
payload type numbers be used first. However, the statically assigned
payload types are default bindings and MAY be dynamically bound to
new encodings if needed. Redefining payload types below 96 may cause
incorrect operation if an attempt is made to join a session without
obtaining session description information that defines the dynamic
payload types.

Dynamic payload types SHOULD NOT be used without a well-defined
mechanism to indicate the mapping. Systems that expect to
interoperate with others operating under this profile SHOULD NOT make
their own assignments of proprietary encodings to particular, fixed
payload types.

Maybe we can make changes to accommodate this.
Pion supports dynamic payload types.

@itzmanish itzmanish changed the title fix: return err if codec with same payload type exists in addCodec fix: addCodec returns error if payload type exists in codecs list Jan 22, 2025
@itzmanish itzmanish changed the title fix: addCodec returns error if payload type exists in codecs list Fix addCodec to return error if payload type exists in codecs list Jan 22, 2025
@itzmanish
Copy link
Contributor Author

itzmanish commented Jan 22, 2025

Will you be able to fix the lint issues (commit and code?).

Maybe need to re-trigger the commit linter to run. I have updated the PR title and last commit message to pass, but it's not getting triggered for newer commits. @JoeTurki could you please re-trigger the gh action for lint metadata?

@itzmanish
Copy link
Contributor Author

itzmanish commented Jan 22, 2025

However, for sendonly and sendrecv streams, the answer might indicate
different payload type numbers for the same codecs, in which case,
the offerer MUST send with the payload type numbers from the answer.

This line. let's say, the server is an offerer and generates SDP with PT 96, mime video/VP8 and PT 97, mime video/VP9. Could it be possible for the client to generate and answer SDP with PT 96, mime video/VP9 and PT 97, mime video/VP8? If yes, according to how we update negotiatedCodecs, it won't get updated.
But the spec says the offerer MUST send with the payload type numbers from the answer, With what PT number the server will send the packet?

@JoeTurki
Copy link
Member

Maybe need to re-trigger the commit linter to run. I have updated the PR title and last commit message to pass, but it's not getting triggered for newer commits. @JoeTurki could you please re-trigger the gh action for lint metadata?

PR title doesn't affect it, You'll need to reword your old commits, but since we only allow for one commit per PR it's better if you squash your commits into a single one and fix the lint issues :)

Thanks :)

@JoeTurki
Copy link
Member

This line. let's say, the server is an offerer and generates SDP with PT 96, mime video/VP8 and PT 97, mime video/VP9. Could it be possible for the client to generate and answer SDP with PT 96, mime video/VP9 and PT 97, mime video/VP8? If yes, according to how we update negotiatedCodecs, it won't get updated.
But the spec says the offerer MUST send with the payload type numbers from the answer, With what PT number the server will send the packet?

Is this what you mean?

t.Run("Change Payload Type", func(t *testing.T) {

@itzmanish
Copy link
Contributor Author

itzmanish commented Jan 22, 2025

yes. I get it now. The negotiated codecs list always respects the answer's codec preferences. Thanks :)

@Sean-Der Sean-Der added this to the V4.1.0 milestone Feb 13, 2025
@itzmanish itzmanish force-pushed the fix-mediaengine-manish branch 3 times, most recently from f76eee2 to 5ff1c97 Compare February 25, 2025 11:27
@itzmanish itzmanish force-pushed the fix-mediaengine-manish branch from 5ff1c97 to a34cc7e Compare February 25, 2025 11:31
@itzmanish itzmanish requested a review from JoeTurki February 25, 2025 11:36
@itzmanish
Copy link
Contributor Author

Hi @JoeTurki, Sorry for the delay on my end. Please re-review the PR, I have updated it to propagate the error on the updateRemoteDescription flow as well.

Copy link
Member

@JoeTurki JoeTurki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Manish :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

RegisterCodec can register two different codecs with same payload type
3 participants