feat: implement enterPictureInPictureOnLeave prop for both platform(Android, iOS) #3385

YangJonghun · 2023-11-26T20:46:49Z

Update the changelog

feat(android): implement pictureInPicture prop and onPictureInPictureStatusChanged event

Describe the changes

(Breaking Change) Remove pictureInPicture boolean prop (iOS)
Add enterPictureInPictureOnLeave boolean prop (Android, iOS)
Add enterPictureInPicture method (Android, iOS)
Add exitPictureInPicture method (Android, iOS)
Add onPictureInPictureStatusChanged event listener (Android)
Resolve issue React Native Video Picture In Picture for Android #2621

memo)
As I know Activity's onUserLeaveHint is acceptable lifecycle method for enter PIP. but it's impossible to detect that method within this library. so, I implement enterPicturInPicture within onHostPause.
If someone doesn't like this approach, please leave comment in this PR, I'll give you intent style code.

freeboub · 2023-11-26T20:50:00Z

Thank you for the PR, can you open a PR in draft, it will allow to comment on it.

public void onHostPause() {
isInBackground = true;
if (pictureInPictureEnabled) {
enterPictureInPictureMode();
return;
}
You cannot do like this, when another application will go over the app, it will automatically switch to Pip ...

freeboub · 2023-11-26T20:51:13Z

I don't understand why you use a dedicated fragment : ReactExoplayerFragment can you explain why you had to use it please ? (point to doc or other sample is enough)

freeboub · 2023-11-26T20:51:30Z

can you enable pip in sample to be able to test easily please

YangJonghun · 2023-11-26T20:52:11Z

can you enable pip in sample to be able to test easily please

      <Video
        style={{width: '100%', height: 300, backgroundColor: 'black'}}
        resizeMode={ResizeMode.CONTAIN}
        source={{
          uri: 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-en.ism/.mpd',
        }}
        selectedTextTrack={{
          type: SelectedTrackType.INDEX,
          value: 0,
        }}
        fullscreen={false}
        controls
        pictureInPicture
        playWhenInactive
        playInBackground={false}
        debug={{enable: true, thread: true}}
        onPictureInPictureStatusChanged={({isActive}) => {
          console.log(isActive, 'isActive');
        }}
      />

YangJonghun · 2023-11-26T20:57:44Z

@freeboub

I don't understand why you use a dedicated fragment : ReactExoplayerFragment can you explain why you had to use it please ? (point to doc or other sample is enough)

To implement PIP, we need to use the onUserLeaveHint and onPictureInPictureModeChanged methods of Activity. However, this requires forcing the user to modify their native code.
To avoid this, I tricky replaced onUserLeaveHint with onPause in the Fragment and onPictureInPictureModeChanged with Fragment's onPictureInPictureModeChanged in the Fragment.

freeboub · 2023-11-26T20:59:11Z

can you enable pip in sample to be able to test easily please

      <Video
        style={{width: '100%', height: 300, backgroundColor: 'black'}}
        resizeMode={ResizeMode.CONTAIN}
        source={{
          uri: 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-en.ism/.mpd',
        }}
        selectedTextTrack={{
          type: SelectedTrackType.INDEX,
          value: 0,
        }}
        fullscreen={false}
        controls
        pictureInPicture
        playWhenInactive
        playInBackground={false}
        debug={{enable: true, thread: true}}
        onPictureInPictureStatusChanged={({isActive}) => {
          console.log(isActive, 'isActive');
        }}
      />

Ok, but will it work without native controls ?

YangJonghun · 2023-11-26T21:00:49Z

can you enable pip in sample to be able to test easily please

      <Video
        style={{width: '100%', height: 300, backgroundColor: 'black'}}
        resizeMode={ResizeMode.CONTAIN}
        source={{
          uri: 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-en.ism/.mpd',
        }}
        selectedTextTrack={{
          type: SelectedTrackType.INDEX,
          value: 0,
        }}
        fullscreen={false}
        controls
        pictureInPicture
        playWhenInactive
        playInBackground={false}
        debug={{enable: true, thread: true}}
        onPictureInPictureStatusChanged={({isActive}) => {
          console.log(isActive, 'isActive');
        }}
      />

Ok, but will it work without native controls ?

Yes, it's same like iOS and there are no restrictions.

YangJonghun · 2023-11-26T21:06:08Z

Thank you for the PR, can you open a PR in draft, it will allow to comment on it.

public void onHostPause() { isInBackground = true; if (pictureInPictureEnabled) { enterPictureInPictureMode(); return; } You cannot do like this, when another application will go over the app, it will automatically switch to Pip ...

I'm not sure, but as I know onUserLeaveHint is also called when another app is turned on.
Actually I wanted to detect and implement the home key event, but the API was deprecated and I replaced it with onPause.

YangJonghun · 2024-01-11T04:08:26Z

@nick-mccomb-easygo
Unfortunately, I've been so busy lately that I haven't been able to keep up with this work, but I'm going to finish it.
You're welcome to implement it if you'd like.

- activity provide helper method for same purpose, but this flag makes code simple

YangJonghun · 2024-01-19T17:44:53Z

@freeboub @KrzysztofMoch

Hi. I'd like to let you know this PR is ready for review 🙌
please review when you possible. :)

FYI,
this PR depends on PR #3489
If doesn't merge, PIP's action button doesn't work properly

I think this code is too mush complicate to understand.
but I couldn't rewrite FullScreenPlayerView code in this PR.
It would be nice if you(or someone) could refactor it later.

android/src/main/java/com/brentvatne/exoplayer/PictureInPictureUtil.kt

KrzysztofMoch

LGTM 👍, thank you for the PR!
Tested it on Android 14, and 10 (simulators) and it worked for me. Will test it on device later.
Lets @freeboub review and for me we can merge

tiplefbxx · 2024-09-09T13:36:46Z

I have no any idea how to reproduce it, but it might help you
@YangJonghun

Positivo TL10 (TL10) Android 13 (SDK 33)
Type
java.lang.IndexOutOfBoundsException
Exception java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
  at jdk.internal.util.Preconditions.outOfBounds (Preconditions.java:64)
  at jdk.internal.util.Preconditions.outOfBoundsCheckIndex (Preconditions.java:70)
  at jdk.internal.util.Preconditions.checkIndex (Preconditions.java:266)
  at java.util.Objects.checkIndex (Objects.java:359)
  at java.util.ArrayList.get (ArrayList.java:434)
  at com.brentvatne.exoplayer.ReactExoplayerView.setIsInPictureInPicture (ReactExoplayerView.java:2192)
  at com.brentvatne.exoplayer.ReactExoplayerFragment.onPictureInPictureModeChanged (ReactExoplayerFragment.kt:35)
  at androidx.fragment.app.Fragment.performPictureInPictureModeChanged (Fragment.java:3104)
  at androidx.fragment.app.FragmentManager.dispatchPictureInPictureModeChanged (FragmentManager.java:2863)
  at androidx.fragment.app.FragmentController.dispatchPictureInPictureModeChanged (FragmentController.java:367)
  at androidx.fragment.app.FragmentActivity.onPictureInPictureModeChanged (FragmentActivity.java:227)
  at android.app.Activity.onPictureInPictureModeChanged (Activity.java:2828)
  at androidx.activity.ComponentActivity.onPictureInPictureModeChanged (ComponentActivity.java:1064)
  at android.app.Activity.dispatchPictureInPictureModeChanged (Activity.java:8619)
  at android.app.ActivityThread.handleWindowingModeChangeIfNeeded (ActivityThread.java:6181)
  at android.app.ActivityThread.performActivityConfigurationChanged (ActivityThread.java:6051)
  at android.app.ActivityThread.performConfigurationChangedForActivity (ActivityThread.java:6028)
  at android.app.ActivityThread.handleActivityConfigurationChanged (ActivityThread.java:6314)
  at android.app.servertransaction.ActivityConfigurationChangeItem.execute (ActivityConfigurationChangeItem.java:53)
  at android.app.servertransaction.ActivityTransactionItem.execute (ActivityTransactionItem.java:45)
  at android.app.servertransaction.TransactionExecutor.executeCallbacks (TransactionExecutor.java:137)
  at android.app.servertransaction.TransactionExecutor.execute (TransactionExecutor.java:97)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:2393)
  at android.os.Handler.dispatchMessage (Handler.java:106)
  at android.os.Looper.loopOnce (Looper.java:201)
  at android.os.Looper.loop (Looper.java:288)
  at android.app.ActivityThread.main (ActivityThread.java:8100)
  at java.lang.reflect.Method.invoke
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:703)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:911)

YangJonghun · 2024-09-09T14:34:17Z

@tiplefbxx
Thanks! Error you mentioned occurs in code below.

https://github.com/TheWidlarzGroup/react-native-video/pull/3385/files#diff-a35ae7e6ae6efee3b22522a836bcbcdd9d59a214f087df454e4ccbb8cd5c0e76R2177-R2209

To be precise, it seems to happen in the part where you call rootViewChildrenOriginalVisibility.get(i) when isInPictureInPicture is false.
The role of the code above is to save the visibility of all views except Player when activate PIP and make them invisible, and then restore the visibility of the view normally after PIP close.
It seems that the value of rootView.getChildCount() is different when PIP is on and off, which is causing the error, but debugging it would require reproducible code.🥲

tiplefbxx · 2024-09-10T11:35:53Z

@YangJonghun

Actually I was able to reproduce the crash, not sure that exactly this one, as it not thrown any errors
Steps to reproduce

Open any media in PiP
Being in PiP navigate to the same screen with another props (autoplay next video feature)
PiP started playing new media, but sometimes you can see black '5-10px' line in the top of PiP View
Press expand or close button from PiP View
Instantly crashed without any error

tiplefbxx · 2024-09-10T11:45:37Z

@YangJonghun
upd: I checked from logcat, that is error that a pasted above

YangJonghun · 2024-09-10T12:05:16Z

@tiplefbxx
Thank you for letting me know how to reproduce it. I'll check it out when I have time. 👍
And If you have a little more time, please provide me any small reproduction code🙏 it would be very helpful to me

tiplefbxx · 2024-09-11T14:39:38Z

@YangJonghun Sorry cannot paste simple example in nearest time, but what I found
It seems the most close way to reproduce the issue, is mount and then unmount player while you in PiP

I tested case when I changing source while playing in PiP and it works fine
case when player component mounted and then unmounted while in PiP makes that error

tiplefbxx · 2024-09-15T11:46:40Z

@YangJonghun
Here is simply playground for testing https://github.com/tiplefbxx/React-native-video-PiP-crash
BTW: Don't forget to update doc! It needs to add android:supportsPictureInPicture="true" to main activity, otherwise it will not work

tiplefbxx · 2024-09-26T13:10:10Z

It seems one more issue for ios
When you change source being in background with PiP, it will close the pip and do not notify onPictureInPictureStatusChanged, closing manually also do not help videoRef.current?.exitPictureInPicture

Release v6.6.2

YangJonghun · 2024-09-29T13:18:40Z

@tiplefbxx
Thank you for providing sample code.
Previous code does not assume source or mount changes from entering PIP to leaving PIP, as it is common to utilize the existing view when playing other videos.

Fixed Code has been applied with 362459d to force PIP to exit when the video component is unmounted within PIP

BTW: Don't forget to update doc! It needs to add android:supportsPictureInPicture="true" to main activity, otherwise it will not work

I've already stated this in this PR. Thanks for letting me know. :)

It seems one more issue for ios
When you change source being in background with PiP, it will close the pip and do not notify onPictureInPictureStatusChanged, closing manually also do not help videoRef.current?.exitPictureInPicture

onPictureInPictureStatusChanged is an API provided by iOS native, and it's not clear what the intent of the prop is. To implement the feature, I have to forced call onPictureInPictureStatusChanged for JS, which may be outside the intent of the existing prop.🤔

tiplefbxx · 2024-09-29T14:28:02Z

Might this ios thing is not related to you pr, but if you started changes for PiP it could be interesting for you
Not crucial issue but
When you go to PiP and return to foreground by clicking app icon from HomeScreen it will return to app, but the PiP will be not resolved, player will be minimised in PiP, so it needs to manually call method to close PiP

It's might be not issue at all as PiP usually works for ios in that way always, but could be surprising behaviour for developers who are used to see closed pip after returning to foreground as it usually works for cross button and app selector

Release v6.6.3

YangJonghun · 2024-10-07T15:25:51Z

@tiplefbxx
Thanks for suggestion, but for now I don't want this PR to grow any bigger, so I'll figure out it when this PR is merged🥲

KrzysztofMoch · 2024-10-10T15:20:16Z

Anyone interested in this feature, please see this discussion

Picture in Picture on Android - We need your help! #4228

freeboub · 2024-10-10T21:44:36Z

I just fix conflict. The only comments I have are in the discussion

android/src/main/java/com/brentvatne/react/VideoManagerModule.kt

KrzysztofMoch · 2024-10-11T10:16:15Z

I have created 6.7.0-rc.0 release that contain this PR - as we will get feedback and do some changes I will push more RCs. For now let's see how it will go 🙌

KrzysztofMoch · 2024-10-26T15:09:40Z

I have released next rc version 6.8.0-rc.0 - if there will be no reported issues, this PR should be merged 🙌

tungmin97 · 2024-11-10T16:03:36Z

Hi @YangJonghun, I'm encountering the same issue as @tiplefbxx mentioned where my app would always crash upon return from PIP.

For a little bit more context, I have a separated Video component outside my navigator stack, and in my testing unload the video component below when having a floating player eliminates this behavior.

Your app just crashed. See the error below.
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
  java.util.ArrayList.get(ArrayList.java:437)
  com.brentvatne.exoplayer.ReactExoplayerView.setIsInPictureInPicture(ReactExoplayerView.java:2305)
  com.brentvatne.exoplayer.ReactExoplayerFragment.onPictureInPictureModeChanged(ReactExoplayerFragment.kt:35)
  androidx.fragment.app.Fragment.performPictureInPictureModeChanged(Fragment.java:3221)
  androidx.fragment.app.FragmentManager.dispatchPictureInPictureModeChanged(FragmentManager.java:3017)
  androidx.fragment.app.FragmentManager.lambda$new$3$androidx-fragment-app-FragmentManager(FragmentManager.java:468)
  androidx.fragment.app.FragmentManager$$ExternalSyntheticLambda3.accept(Unknown Source:4)
  androidx.activity.ComponentActivity.onPictureInPictureModeChanged(ComponentActivity.java:1097)
  android.app.Activity.dispatchPictureInPictureModeChanged(Unknown Source:23)
  android.app.ActivityThread.handleWindowingModeChangeIfNeeded(Unknown Source:40)
  android.app.ActivityThread.performActivityConfigurationChanged(Unknown Source:12)
  android.app.ActivityThread.performConfigurationChangedForActivity(Unknown Source:28)
  android.app.ActivityThread.handleActivityConfigurationChanged(Unknown Source:273)
  android.app.ActivityThread$ActivityClientRecord$1.onConfigurationChanged(Unknown Source:14)
  android.view.ViewRootImpl.performConfigurationChange(Unknown Source:81)
  android.view.ViewRootImpl.handleResized(Unknown Source:112)
  android.view.ViewRootImpl.-$$Nest$mhandleResized(Unknown Source:0)
  android.view.ViewRootImpl$ViewRootHandler.handleMessageImpl(Unknown Source:658)
  android.view.ViewRootImpl$ViewRootHandler.handleMessage(Unknown Source:15)
  android.os.Handler.dispatchMessage(Unknown Source:19)
  android.os.Looper.loopOnce(Unknown Source:182)
  android.os.Looper.loop(Unknown Source:82)
  android.app.ActivityThread.main(Unknown Source:123)
  java.lang.reflect.Method.invoke(Native Method)
  com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(Unknown Source:11)
  com.android.internal.os.ZygoteInit.main(Unknown Source:312)

YangJonghun · 2024-11-11T05:38:18Z

@tungmin97
Thanks for reporting me. but it's hard to me to reproduce what you're describing🥲
If you can, please provide me any small reproduction code🙏

YangJonghun added 3 commits November 27, 2023 05:03

docs: enable Android PIP

29c35e9

chore: change comments

e5fd0c7

feat(android): implement Android PictureInPicture

b554034

YangJonghun mentioned this pull request Nov 26, 2023

React Native Video Picture In Picture for Android #2621

Open

YangJonghun added 6 commits January 18, 2024 21:46

Merge branch 'master' into feat/android-pip

e6797fd

refactor: minor refactor code and apply lint

3ba6a30

fix: rewrite pip action intent code for Android14

4938bd0

fix: remove redundant codes

6e8307b

feat: add isInPictureInPicture flag for lifecycle handling

0c06c2f

- activity provide helper method for same purpose, but this flag makes code simple

feat: add pipFullscreenPlayerView for makes PIP include video only

f3066a4

YangJonghun marked this pull request as ready for review January 19, 2024 17:40

Merge branch 'master' into feat/android-pip

60657eb

KrzysztofMoch reviewed Jan 20, 2024

View reviewed changes

android/src/main/java/com/brentvatne/exoplayer/PictureInPictureUtil.kt Outdated Show resolved Hide resolved

YangJonghun added 3 commits January 21, 2024 00:20

fix: add manifest value checker for prevent crash

2872689

docs: add pictureInPicture prop's Android guide

9723313

fix: sync controller visibility

8269546

KrzysztofMoch requested a review from freeboub January 22, 2024 15:43

KrzysztofMoch approved these changes Jan 22, 2024

View reviewed changes

refactor: refining variable name

d10f624

YangJonghun force-pushed the feat/android-pip branch from 4621774 to 1b3a0b2 Compare January 23, 2024 16:43

YangJonghun added 2 commits September 29, 2024 17:21

Merge tag 'v6.6.2' into feat/android-pip

f376b53

Release v6.6.2

fix(android): fix crash issue when unmounting in PIP mode

362459d

fix(android): fix lint error

7e8a0b8

Merge tag 'v6.6.3' into feat/android-pip

2c78d41

Release v6.6.3

Merge branch 'master' into feat/android-pip

74abd7f

freeboub reviewed Oct 10, 2024

View reviewed changes

android/src/main/java/com/brentvatne/react/VideoManagerModule.kt Show resolved Hide resolved

freeboub and others added 4 commits October 10, 2024 23:46

Update android/src/main/java/com/brentvatne/react/VideoManagerModule.kt

9c805d9

Merge branch 'master' into feat/android-pip

8e132c3

fix(android): fix lint error

d712690

fix(ios): fix lint error

441fb56

YangJonghun added 3 commits October 13, 2024 11:46

Merge branch 'master' into feat/android-pip

c431403

fix(ios): fix lint error

42b2477

feat(expo): add android picture in picture config within expo plugin

a4381e4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement enterPictureInPictureOnLeave prop for both platform(Android, iOS) #3385

feat: implement enterPictureInPictureOnLeave prop for both platform(Android, iOS) #3385

YangJonghun commented Nov 26, 2023 •

edited

Loading

freeboub commented Nov 26, 2023 •

edited

Loading

freeboub commented Nov 26, 2023

freeboub commented Nov 26, 2023

YangJonghun commented Nov 26, 2023 •

edited

Loading

YangJonghun commented Nov 26, 2023 •

edited

Loading

freeboub commented Nov 26, 2023

YangJonghun commented Nov 26, 2023

YangJonghun commented Nov 26, 2023 •

edited

Loading

YangJonghun commented Jan 11, 2024

YangJonghun commented Jan 19, 2024 •

edited

Loading

KrzysztofMoch left a comment

tiplefbxx commented Sep 9, 2024 •

edited

Loading

YangJonghun commented Sep 9, 2024

tiplefbxx commented Sep 10, 2024 •

edited

Loading

tiplefbxx commented Sep 10, 2024

YangJonghun commented Sep 10, 2024

tiplefbxx commented Sep 11, 2024 •

edited

Loading

tiplefbxx commented Sep 15, 2024 •

edited

Loading

tiplefbxx commented Sep 26, 2024

YangJonghun commented Sep 29, 2024

tiplefbxx commented Sep 29, 2024

YangJonghun commented Oct 7, 2024

KrzysztofMoch commented Oct 10, 2024

freeboub commented Oct 10, 2024

KrzysztofMoch commented Oct 11, 2024

KrzysztofMoch commented Oct 26, 2024

tungmin97 commented Nov 10, 2024

YangJonghun commented Nov 11, 2024

feat: implement enterPictureInPictureOnLeave prop for both platform(Android, iOS) #3385

Are you sure you want to change the base?

feat: implement enterPictureInPictureOnLeave prop for both platform(Android, iOS) #3385

Conversation

YangJonghun commented Nov 26, 2023 • edited Loading

Update the changelog

Describe the changes

freeboub commented Nov 26, 2023 • edited Loading

freeboub commented Nov 26, 2023

freeboub commented Nov 26, 2023

YangJonghun commented Nov 26, 2023 • edited Loading

YangJonghun commented Nov 26, 2023 • edited Loading

freeboub commented Nov 26, 2023

YangJonghun commented Nov 26, 2023

YangJonghun commented Nov 26, 2023 • edited Loading

YangJonghun commented Jan 11, 2024

YangJonghun commented Jan 19, 2024 • edited Loading

KrzysztofMoch left a comment

Choose a reason for hiding this comment

tiplefbxx commented Sep 9, 2024 • edited Loading

YangJonghun commented Sep 9, 2024

tiplefbxx commented Sep 10, 2024 • edited Loading

tiplefbxx commented Sep 10, 2024

YangJonghun commented Sep 10, 2024

tiplefbxx commented Sep 11, 2024 • edited Loading

tiplefbxx commented Sep 15, 2024 • edited Loading

tiplefbxx commented Sep 26, 2024

YangJonghun commented Sep 29, 2024

tiplefbxx commented Sep 29, 2024

YangJonghun commented Oct 7, 2024

KrzysztofMoch commented Oct 10, 2024

freeboub commented Oct 10, 2024

KrzysztofMoch commented Oct 11, 2024

KrzysztofMoch commented Oct 26, 2024

tungmin97 commented Nov 10, 2024

YangJonghun commented Nov 11, 2024

YangJonghun commented Nov 26, 2023 •

edited

Loading

freeboub commented Nov 26, 2023 •

edited

Loading

YangJonghun commented Nov 26, 2023 •

edited

Loading

YangJonghun commented Nov 26, 2023 •

edited

Loading

YangJonghun commented Nov 26, 2023 •

edited

Loading

YangJonghun commented Jan 19, 2024 •

edited

Loading

tiplefbxx commented Sep 9, 2024 •

edited

Loading

tiplefbxx commented Sep 10, 2024 •

edited

Loading

tiplefbxx commented Sep 11, 2024 •

edited

Loading

tiplefbxx commented Sep 15, 2024 •

edited

Loading