[Bug] MediaElement XCT v2.0.6 not working with UWP

[bhughes1123]

Description

When creating a ContentPage to modally display videos taken in our application the MediaElement does not display the loaded video or controls, only a box with the set background color and width/height for UWP (tested on local machine and various Surface tablets with no luck). The MediaElement works fine on Android and I have tested with all version of Xamarin Community Toolkit from v1.0.3 through 2.0.6 with various combos of Min Version and Targeting Version without any success. The only way in which I can get the videos to play on a UWP device is by disabling the 'Compile with .Net native tool chain' in properties and debug but the app certification will not pass without that option enabled. I have taken the basic implementation from this repo using a static web source and still get the black box, same when setting the source to use our Azure's video stream service. I have also wired up the MediaOpened, MediaFailed, and MediaEnded but these methods only fire when running on Android and there are no errors reported in the output for UWP.

UI Implementation:

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml" 
             x:Class="InfoCap.Views.MediaViewerPageView"
             xmlns:vm="clr-namespace:InfoCap.ViewModels"
             xmlns:toolkit="http://xamarin.com/schemas/2020/toolkit">

    <ContentPage.BindingContext>
        <vm:MediaViewerPageViewModel />
    </ContentPage.BindingContext>

    <ContentPage.Resources>
        <toolkit:InvertedBoolConverter x:Key="InvertedBoolConverter" />
    </ContentPage.Resources>

    <AbsoluteLayout HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand">
        <Grid Margin="15,50,15,25" AbsoluteLayout.LayoutBounds="0,0,1,1" AbsoluteLayout.LayoutFlags="SizeProportional">
            <Grid.ColumnDefinitions>
                <ColumnDefinition>
                    <ColumnDefinition.Width>
                        <OnIdiom Phone="15*" Tablet="10*" Desktop="10*"/>
                    </ColumnDefinition.Width>
                </ColumnDefinition>
                <ColumnDefinition>
                    <ColumnDefinition.Width>
                        <OnIdiom Phone="35*" Tablet="40*" Desktop="40*"/>
                    </ColumnDefinition.Width>
                </ColumnDefinition>
                <ColumnDefinition>
                    <ColumnDefinition.Width>
                        <OnIdiom Phone="35*" Tablet="40*" Desktop="40*" />
                    </ColumnDefinition.Width>
                </ColumnDefinition>
                <ColumnDefinition>
                    <ColumnDefinition.Width>
                        <OnIdiom Phone="15*" Tablet="10*" Desktop="10*"/>
                    </ColumnDefinition.Width>
                </ColumnDefinition>
            </Grid.ColumnDefinitions>
            <Grid.RowDefinitions>
                <RowDefinition Height="Auto"/>
                <RowDefinition Height="350"/>
                <RowDefinition Height="50"/>
            </Grid.RowDefinitions>

            <Label Grid.Column="0" Grid.Row="0" Grid.ColumnSpan="4" Text="{Binding CurrentMedia.Name}" Margin="15,0,15,0" FontSize="Subtitle" />

            <Button TextColor="White" BackgroundColor="{Binding HeaderBackgroundColor}" Grid.Column="0" Grid.Row="1" Text="&lt;" Style="{DynamicResource ButtonStyle}" Clicked="PreviousMedia_Clicked" Margin="5,0,5,0" IsVisible="{Binding IsNavVisible}" CornerRadius="40" HeightRequest="50">
                <Button.FontSize>
                    <OnIdiom Phone="14" Tablet="20" Desktop="20"/>
                </Button.FontSize>
            </Button>
            <toolkit:MediaElement Grid.Column="1" Grid.Row="1" Grid.ColumnSpan="2" IsVisible="{Binding CurrentMedia.IsVideo}" AutoPlay="True" KeepScreenOn="True" WidthRequest="350" HeightRequest="350"  
                                  ShowsPlaybackControls="True" BackgroundColor="Black" Margin="0,0,0,25" x:Name="VideoPlayer" Aspect="AspectFit" />
            <Image Grid.Column="1" Grid.Row="1" Grid.ColumnSpan="2" x:Name="photoImage" Aspect="AspectFit" Source="{Binding CurrentMedia.PhotoSource}" WidthRequest="350" IsVisible="{Binding CurrentMedia.IsVideo, Converter={StaticResource InvertedBoolConverter}}" />
            <Button TextColor="White" BackgroundColor="{Binding HeaderBackgroundColor}" Grid.Column="3" Grid.Row="1" Text="&gt;" Style="{DynamicResource ButtonStyle}" Clicked="NextMedia_Clicked" Margin="5,0,5,0" IsVisible="{Binding IsNavVisible}" CornerRadius="40" HeightRequest="50">
                <Button.FontSize>
                    <OnIdiom Phone="14" Tablet="20" Desktop="20"/>
                </Button.FontSize>
            </Button>

            <Button TextColor="White" Style="{DynamicResource ButtonStyle}" BackgroundColor="{Binding HeaderBackgroundColor}" Grid.Column="1" Grid.Row="2"  Text="Delete"   x:Name="btnDelete"  Clicked="BtnDelete_Clicked" CornerRadius="40" HeightRequest="50">
                <Button.FontSize>
                    <OnIdiom Phone="14" Tablet="20" Desktop="20"/>
                </Button.FontSize>
            </Button>
            <Button TextColor="White" Style="{DynamicResource ButtonStyle}" BackgroundColor="{Binding HeaderBackgroundColor}" Grid.Column="2" Grid.Row="2"  Text="Done"     x:Name="btnDone"    Clicked="BtnDone_Clicked"   CornerRadius="40" HeightRequest="50">
                <Button.FontSize>
                    <OnIdiom Phone="14" Tablet="20" Desktop="20"/>
                </Button.FontSize>
            </Button>
        </Grid>

        <!--Spinner-->
        <ActivityIndicator IsRunning="{Binding IsWorking}" Color="{Binding HeaderBackgroundColor}" Scale="3" AbsoluteLayout.LayoutFlags="PositionProportional" AbsoluteLayout.LayoutBounds="0.5,0.5,-1,-1"/>

    </AbsoluteLayout>
</ContentPage>

Relevant code behind:

public MediaViewerPageView(ObservableCollection<UserMedia> userMedia)
        {
            try
            {
                InitializeComponent();
                if (BindingContext is MediaViewerPageViewModel vm)
                {
                    viewModel = vm;
                    viewModel.UserMedia = userMedia;
                    if (viewModel.UserMedia.Count > 0)
                    {
                        viewModel.CurrentMedia = viewModel.UserMedia[_currentIndex];
                    }
                }
            }
            catch (Exception ex)
            {
                TelemetryErrorHandling.TrackException(ex);
            }
        }

protected override void OnAppearing()
        {
            try
            {
                base.OnAppearing();
                if (viewModel.CurrentMedia.IsVideo)
                {
                    TryToDownloadCurrentMedia();
                }
            }
            catch (Exception ex)
            {
                TelemetryErrorHandling.TrackException(ex);
            }
        }

protected async void TryToDownloadCurrentMedia()
        {
            try
            {
                viewModel.IsWorking = true;

                if (viewModel.CurrentMedia.Data == null)
                {
                    InspectionVideo video = await clsWebService.GetVideoBlob(viewModel.CurrentMedia.Name);
                    if (video != null)
                    {
                        viewModel.CurrentMedia.Data = Convert.FromBase64String(video.Base64VideoString);
                    }
                }

                if (viewModel.CurrentMedia.Data != null)
                {
                    Stream videoStream = new MemoryStream(viewModel.CurrentMedia.Data);

                    string videoPath = string.Empty;

                    if (Device.RuntimePlatform == Device.UWP)
                    {
                        videoPath = await App.FileService.SaveVideo("temp.mp4", videoStream);
                    }
                    else if (Device.RuntimePlatform == Device.Android)
                    {
                        videoPath = await DependencyService.Get<IFileService>().SaveVideo("temp.mp4", videoStream);
                    }

                    viewModel.VideoPath = MediaSource.FromFile(videoPath);
                }
                else
                {
                    await DisplayAlert("Could not load video", "Error in downloading the video, it may have been deleted or renamed", "OK");
                    _ = Navigation.PopModalAsync();
                }
            }
            catch (Exception ex)
            {
                TelemetryErrorHandling.TrackException(ex);
            }
            finally
            {
                viewModel.IsWorking = false;
            }
        }

Basic Information

• Version with issue: 2.0.6
• Last known good version: ???
• IDE: Visual Studio Professional 2022
• Platform Target Frameworks:
  • Android: min 5.1, target 12.1
  • UWP: min 1809 (build 17763), target 2004 (build 19041)
• Nuget Packages: Xamarin.Community.Toolkit
• Affected Devices: all UWP devices

Reproduction imagery

[jasells]

The only way in which I can get the videos to play on a UWP device is by disabling the 'Compile with .Net native tool chain' in properties and debug but the app certification will not pass without that option enabled.

I have built a custom renderer package in the past that performed this functionality for UWP/Android. I was never able to get the native compile option to work, but I was able to build in release without the native compile flag. I don't think the native compile is required for store, just a little performance hit at load time.

Have you tried release build without the native compile flag?

I can't remember details at this point, but it is possible that some required files are missing in the package for UWP... there are some non-binary resources needed in the package for UWP that may be the issue here, if you can't build release without native compile flag.

[bhughes1123]

@jasells thank you for that input, I've been looking at creating a custom renderer and I'm thinking that is the direction to go in. Just to clarify though the build works fine in Debug and Release with or without the 'Compile with .Net native tool chain', it's only the very last step in submitting it to the MS Store that the option becomes mandatory, I've tried several times without the .net native compile option and every time the automated certification process fails and won't generate the needed app package to upload to the store.

[jasells]

I've been looking at creating a custom renderer and I'm thinking that is the direction to go in.

I assume that's what is being done in here in the Community TK code base as well... So, I'm not sure if there's anything that can be done about the native compile?

The issue is that the .Net MediaElement on UWP is itself a .Net wrapper around a native component, and the native compile fails to pull in that native layer, i think? My project was for ad-hoc distribution only, so I never had to tackle the native compile problem.

I'd be interested to know if you resolve it. I'd guess you need to look into UWP forums for help on this (compiling UWP to native with a .Net->native-interop dependency)? Hopefully someone is still around that knows something about that platform. Maybe the Windows store has support that could help?

What I can remember without going back and looking at code is that the actual codec/runtime is the same native code that Windows Media player uses and that WPF has a MediaElement that is very similar, though less of the native functionality is exposed. I think you can only open files, not streams.

One thought I just had is to try to write a pure UWP app (as simple as possible) that has the video player capabilities you need and see if you can get the native compile to work there?

Not sure that helps...

[bhughes1123]

@jasells thanks for the info, I'll dig into the native compile some more and see what I can find but for now my manager thought of a work around and so far so good (using a WebView pointed to a page with an html video player that gets feed an encrypted string so it streams directly from azure). Kind of crazy we had to resort to this but this was getting to be a major issue for our product.