ExoPlayer playback fails when attempting to play multiple audio tracks, transcoding issues

[osus222]

Describe the bug

Upgraded to v2.4, seems that some H264/DTS5.1 media are not playing. Another case is, it will buffer for a second and get stuck at pause state and can never be played.

If these media is then stuck at pause state or not playing, changing the audio/subtitle will lead for the app to crash.

There are no errors stated in the app, it is just stuck at pause state (black screen).

Here are some media that experienced this issue so far:

• Pirates of the Caribbean The Curse of the Black Pearl (2003) [tmdb-22][Bluray-1080p][8bit][x264][DTS 5.1]-TayTO.mkv
• Pirates of the Caribbean Dead Mans Chest (2006) [tmdb-58][Bluray-1080p][8bit][x264][DTS 5.1]-Z0N3.mkv
• Pirates of the Caribbean On Stranger Tides (2011) [tmdb-1865][Bluray-1080p][8bit][x264][DTS 5.1]-TayTO.mkv

In v2.3.3 these files are playing properly wherein their audio are transcoded into AAC.
Other H264/DTS5.1 media plays properly in v2.4.

Logs

logcat.txt

Application version

2.4.0

Where did you install the app from?

Google Play

Device information

Samsung A10, LG G6

Android version

Android 9, Android 8

Jellyfin server version

10.7.7

Which video player implementations does this bug apply to?

• Web player (default)
• Integrated player (ExoPlayer)
• External player (VLC, mpv, MX Player)

[CarlosOlivo]

...
11-06 00:06:12.641 18798 18798 E EventLogger: playerFailed [eventTime=1.91, mediaPos=0.00, window=0, period=0, errorCode=ERROR_CODE_UNSPECIFIED
11-06 00:06:12.641 18798 18798 E EventLogger:   com.google.android.exoplayer2.ExoPlaybackException: Unexpected runtime error
11-06 00:06:12.641 18798 18798 E EventLogger:       at com.google.android.exoplayer2.DefaultMediaClock.onRendererEnabled(DefaultMediaClock.java:96)
11-06 00:06:12.641 18798 18798 E EventLogger:       at com.google.android.exoplayer2.ExoPlayerImplInternal.enableRenderer(ExoPlayerImplInternal.java:2447)
11-06 00:06:12.641 18798 18798 E EventLogger:       at com.google.android.exoplayer2.ExoPlayerImplInternal.enableRenderers(ExoPlayerImplInternal.java:2395)
11-06 00:06:12.641 18798 18798 E EventLogger:       at com.google.android.exoplayer2.ExoPlayerImplInternal.reselectTracksInternal(ExoPlayerImplInternal.java:1702)
11-06 00:06:12.641 18798 18798 E EventLogger:       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:507)
11-06 00:06:12.641 18798 18798 E EventLogger:       at android.os.Handler.dispatchMessage(Handler.java:102)
11-06 00:06:12.641 18798 18798 E EventLogger:       at android.os.Looper.loop(Looper.java:214)
11-06 00:06:12.641 18798 18798 E EventLogger:       at android.os.HandlerThread.run(HandlerThread.java:65)
11-06 00:06:12.641 18798 18798 E EventLogger:   Caused by: java.lang.IllegalStateException: Multiple renderer media clocks enabled.
11-06 00:06:12.641 18798 18798 E EventLogger:       ... 8 more
11-06 00:06:12.641 18798 18798 E EventLogger: ]
...
11-06 00:06:12.642 18798 18798 D EventLogger:   MediaCodecAudioRenderer [
11-06 00:06:12.642 18798 18798 D EventLogger:     Group:0, adaptive_supported=N/A [
11-06 00:06:12.642 18798 18798 D EventLogger:       [X] Track:0, id=3, mimeType=audio/ac3, channels=2, sample_rate=48000, language=en, label=Commentary with Writers Ted Elliott and Terry Rossio, supported=YES
11-06 00:06:12.642 18798 18798 D EventLogger:     ]
11-06 00:06:12.642 18798 18798 D EventLogger:   ]
...
11-06 00:06:12.642 18798 18798 D EventLogger:   FfmpegAudioRenderer [
11-06 00:06:12.642 18798 18798 D EventLogger:     Group:0, adaptive_supported=N/A [
11-06 00:06:12.643 18798 18798 D EventLogger:       [X] Track:0, id=2, mimeType=audio/vnd.dts, channels=6, sample_rate=48000, language=en, label=D-Z0N3 HD AUDIO @ DTS 1510 Kbps Core English DTSHD-MA, supported=YES
11-06 00:06:12.643 18798 18798 D EventLogger:     ]
11-06 00:06:12.643 18798 18798 D EventLogger:   ]
...

The problem is that your device tries to play 2 different audios at the same time with 2 different audio decoders, which is not allowed in ExoPlayer, and even if it was, it should not happen in Jellyfin.

However, I don't quite understand how the extension should be used correctly, so I do not know how to fix it, so I leave the comment with my findings in case it helps the devs.

Relevant links:

• https://github.com/google/ExoPlayer/tree/release-v2/extensions/ffmpeg#using-the-module
• DefaultTrackSelector should avoid enabling multiple renderers implementing MediaClock by default google/ExoPlayer#2618

[Maxr1998]

Thanks for the investigation Carlos, this is indeed quite weird. I'm surprised your phone supports ac3 natively (without the extension), I don't think that's very common.
Regardless, I don't know either what makes ExoPlayer enable both audio streams at the same time, but I'd definitely appreciate a mediainfo of the file(s) you were playing. Maybe the metadata reveal something notable so that I can investigate this further.

[osus222]

Hello! Attached are the MediaInfo of the files correspondingly. I also tested some various scenario where I change the audio before playing the videos and concluded with this results. Please let me know if you need to see the logcat. Thank you so much.

Perfect Blue (1997) Remastered [tmdb-10494][Remux-1080p][8bit][x264][DTS-HD MA 5.1][JA+EN]-ALiEN.mkv
MediaInfo_PerfectBlue.txt

before playing

• choosing Audio#2 (eng)(AC3) & any subtitile (PGS,SRT) or off will play the media but the audio playing is Audio#1 (jap)(DTS XLL) [directplay]
  changing the subtitle while playing = will not crash
  changing the audio while playing = pause state, will not play
• choosing Audio#1 (jap)(DTS XLL) & any subtitile or off = pause state, will not play
  changing the subtitle/audio while playing = pause state, will not play and sometimes will crash

Pirates of the Caribbean The Curse of the Black Pearl (2003) [tmdb-22][Bluray-1080p][8bit][x264][DTS 5.1]-TayTO.mkv
MediaInfo_POTC(2003).txt

before playing

• choosing Audio#3 (eng)(AC3) & subtitle off or any subtitle will play the media properly [directplay]
  changing the subtitle while playing = will not crash, still playing
  changing the audio while playing = pause state, will not play
• choosing Audio#1 (eng)(DTS) & subtitle (off) or any subtitle = pause state, will not play
  changing subtitle/audio while playing = pause state, will not play and sometimes will crash

Pirates of the Caribbean Dead Mans Chest (2006) [tmdb-58][Bluray-1080p][8bit][x264][DTS 5.1]-Z0N3.mkv
MediaInfo_POTC(2006).txt

before playing

• choosing Audio#2 (eng)(AC3) & subtitle off or any subtitle will play the media but the audio playing is Audio#1 (eng)(DTS) [directplay]
  changing subtitle while playing = will not crash, still playing
  changing audio while playing = pause state, will not play
  bug: slider disappeared, skipping will cause the media to go back to 00:00
• choosing Audio#1 (eng)(DTS) & subtitle off or any subtitle = pause state, will not play
  changing subtitle/audio while playing = pause state, will not play

Pirates of the Caribbean On Stranger Tides (2011) [tmdb-1865][Bluray-1080p][8bit][x264][DTS 5.1]-TayTO.mkv
MediaInfo_POTC(2011).txt

before playing

• choosing Audio#3 (eng)(AC3) & subtitle off or any subtitle will play the media properly [directplay]
  changing subtitle while playing = will not crash, still playing
  changing audio while playing = pause state, will not play
• choosing Audio#1 (eng)(DTS) & subtitle off or any subtitle = pause state, will not play
  changing subtitle/audio while playing = pause state, will not play

[Maxr1998]

Awesome, thanks for the details! I'll look into them and try to fix those issues when I find the time.

[Maxr1998]

This seems to have been an ExoPlayer issue that was fixed in the latest version, see google/ExoPlayer#9675.

We'll include this in the next update.

[gauth-fr]

I'm not sure the issue is (totally?) fixed.

I have some files many files with multiples tracks, and when one of these tracks is AC3/EAC3 and the second one is AAC, then I still have the issue.

Here is an example where the files has 2 audio tracks. First is AC3 (french) and second is AAC (english). If i play the movie using the AC3 track (first track), it fails (media looks paused).
logcat_2tracks.txt
However, from the movie page, if I choose the AAC tracks (and this one is the last track) and play the movie, it works but the it's actually the french track which is playing...so the first track... weird....
logcat_2tracks_wrong_track_playing.txt

Then I have another files with four audio tracks. First is AC3, then AAC, then AC3, then AAC.
Selecting any of the 2 first tracks won't play the media
logcat_4tracks.txt

But selecting any of the 2 last ones will play the movie but the actual audio tracks playing is always the first one.
logcat_4tracks_select_3rd_track.txt
logcat_4tracks_select_4rd_track.txt

I know it's probably an issue with exoplayer, but since their repo seems above my level (i'm not a developer), I prefer to post here and, maybe, let you relay the issue and you may speak the same language as the devs there.

Note that the issue seems really related to the mix of (E)AC3 and AAC.
I have some other media with AC3, EAC3 or DTS mixed, they seems to work fine.

Hope this helps!
And of course, thanks for all the hard work!

[gauth-fr]

Hi @Maxr1998,
Any news about this issue?
I don't know if it can be of any help but I found
google/ExoPlayer#9753

And if ever you need testers... I can help :)

Anyway thanks for the great work!

[Maxr1998]

Haven't had the chance to look into this yet. Exams are coming up too, so I doubt I'll fix this soon. But it's definitely still on the radar, and some of the code will be rewritten for the transcoding selector, that might already fix it.

[jellyfin-bot]

This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.

If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or master branch, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.

This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.

[gauth-fr]

Just to unstale the issue as it still exists :)

If ever it's relevant, since 10.8 is now GA, i spinned an instance to try to if it would do any good, but no :)
No pressure, no hurry, i'm still using @CarlosOlivo mpv PR build (and i preciously keep the apk :)) ), which works like a charm ^^

[osus222]

this issue still exists

using 10.8.1

[jellyfin-bot]

This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.

If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or master branch, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.

This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.

[osus222]

this issue still exist

using jf server 10.8.7 and jf android app 2.4.4

[Maxr1998]

Could you try the latest 2.5.0 beta?

[gauth-fr]

Hi @Maxr1998 ,
I've just tried, and unfortunately similar behaviour.
Still with my 2 tracks video.
If i choose the first one (French AC3), it won't play and ends with Unexpected Runtime error message.
logcat__first_tracks.txt

If i choose the second one (English AAC), it works but it the first track playing (the french one).
logcat__second_tracks.txt

Anyway, thanks for the effort! I keep faith!

[Maxr1998]

I see, looks like the bug in ExoPlayer still exists - or we're using it incorrectly. Need to investigate.

[satmandu]

I think I might be seeing the same exo player issue?

I get an unexpected runtime error playing with integrated player, but the web player plays just fine.

This is also for a 1080p h264 file with multiple audio tracks. (TrueHD, DTS, DTSHD).

This is with the current beta 2.5.0, and the previous version. Playing with mpv works fine.

[EvanAllan]

I am having the same issue with movies containing multiple audio tracks when using Exoplayer.

The web player had a different issue (black screen when playback controls were hidden, requiring constant touching on screen to keep video alive), but switching to the beta version of Android Webview solved that.

Hoping the Exoplayer bug gets figured out as that would be a better experience on mobile.