Background audio streaming with CommunityToolkit.Maui.MediaElement (Preview) for Android and iOS

This article is part of the .NET MAUI Advent Calender. Find all contributions here.

So people have been wondering since the very first previews of .NET MAUI how they will be supposed to play media files such as videos and mp3 files for instance. While they have always been able to utilize the Xamarin CommunityToolkit - which includes the well known MediaElement control - this only worked for iOS when trying to import it into a MAUI solution. So that you can play or stream media files on Android, you had to directly install ExoPlayer2. However, since MediaElement also has a reference to the ExoPlayer, things got really nasty. Either you found out how to implement platform dependent controls yourself to seperate these components from eachother, or you had to clutter your code with lots of precompiler directives to ensure that Android based code does not interfere with iOS based code and vice versa.

Gerald Versluis to the rescue

Not long ago, Gerald became active on this topic and started to implement Media Plugin .NET MAUI which is loosely based on the work of James Montemagno and his .NET Podcast App. The first iteration only showed how to play local mediafiles and was hosted in his private GitHub repository.

However, things kinda went viral and so he decided to put a little more effort into this component. The result is an active Pull Request on the official .NET MAUI CommunityToolkit!

First things first

As of the time of writing, you may or may not run into some version conflicts when trying out preview libraries such as the CommunityToolkit.Maui.MediaElement. In this case this could be due to the library referencing a newer version of the WindowsAppSDK compared to what your version of Visual Studio has installed. If this is the case, then you could skip targetting Windows for now and get back to this later when the library is finally released.

UPDATE
Microsoft released a new version of Visual Studio that also includes support for WindowsAppSDK 1.2.221109.1 which should render the last paragraph obsolete. More about it here.

Play with it

Before I show you how you can make the component stream audio content even when your app is not in the foreground ("Background Streaming"), let's dive into how to include the MediaElement in the first place, since it is still in preview (and actually still "only" a Pull Request - but don't let that frighten you).

Detailed instructions on how to NuGet install MediaElement into your .NET MAUI project can be found on Geralds blog here.

The shortcut is this:

1. Go to Tools > NuGet Package Manager > Package Manager Settings
2. Go to Package Sources
3. Hit the Plus Icon
4. Create a new Package Source

Name the new Package Source "Microsoft CommunityToolkit PR" and let it point to this URL: "https://pkgs.dev.azure.com/dotnet/CommunityToolkit/_packaging/CommunityToolkit-PullRequests/nuget/v3/index.json"

The magic is, that the CommunityToolkit GitHub repo automatically creates NuGet packages for Pull Request. Nice, eh?

Install the NuGet Package

Next step is installing the NuGet Package into your project. I assume you already know how to install NuGet Packages, so I'll ommit detailed steps. However, this is the Package you want to install: CommunityToolkit.Maui.MediaElement. (Remember to tick the preview checkbox).

As of the time of writing this, the Package is labeled as version 1.0.0-build-667.84118. 667 refers to the Pull Request id.

It also depends on the WindowsAppSDK Version 1.2.221116.1 which you might not have installed yet. As mentioned above this could result in errors and the easiest way to deal with this at the moment is removing the Windows target out of your .csproj file.

Initialize and configure the MediaElement

Next thing you need to do is registering all necessary handlers. Luckily, there is an extension you can call from within your MauiProgram.cs:CreateMauiApp method (UseMauiCommunityToolkitMediaElement) like this:

builder
    .UseMauiApp<App>()
    **.UseMauiCommunityToolkitMediaElement()**
    .ConfigureFonts(fonts =>
    {
        fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");
    fonts.AddFont("OpenSans-Semibold.ttf", "OpenSansSemibold");
    });

With this being more or less it, you could theoretically try to use the MediaElement right now.

Open your MainPage.xaml and add the following namespace to it:

xmlns:media="clr-namespace:CommunityToolkit.Maui.MediaElement;assembly=CommunityToolkit.Maui.MediaElement"

Then you can add the MediaElement itself to your page like this:

<media:MediaElement
    x:Name="mediaElement"    
    AutoPlay="True"  
    HeightRequest="100"
    Source="http://ais-sa5.cdnstream1.com/b55667_128mp3" />        

Android Settings to consider

If you tried that right now, you may have run into an issue with ExoPlayer telling you that it cannot stream from an HTTP address. This is due to some security constraints. You can easily get around this problem by adding a network_security_config.xml file to your Platforms/Android/Resources/xml folder (create one if it does not exist yet) with the following content:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <domain-config cleartextTrafficPermitted="true">
        <domain includeSubdomains="true">cdnstream1.com</domain>
    </domain-config>
</network-security-config>

More info on that can be found here. Finally you need to reference this configuration directly from within your AndroidManifest.xml file like so:

<application **android:networkSecurityConfig="@xml/network_security_config"** android:allowBackup="true" android:icon="@mipmap/appicon" android:roundIcon="@mipmap/appicon_round" android:supportsRtl="true"></application>

iOS Settings to consider

You also might have noticed that on Android, the app is already capable of streaming in the background. On iOS this does not work, yet. Also, if your iPhone has the ability to enter a silent mode with a hardware button, you might have noticed, that the playback stops when the device is in silent mode.

So that on iOS your app is able to stream even when being in background mode, you need to do two things.

Step 1: Add the Background Audio Mode capability to your Info.plist file like so:

<key>UIBackgroundModes</key>
<array>
    <string>audio</string>
</array>

Step 2: You need to register a lifecycle event so that the AVAudioSession is set as a shared instance to the device.

This is done in your CreateMauiApp method and needs to be restricted to the iOS target like so:

builder
    .UseMauiApp<App>()
    .UseMauiCommunityToolkit()
    .UseMauiCommunityToolkitMediaElement()              
    .ConfigureLifecycleEvents(events =>
    {
#if IOS
        events.AddiOS(ios => ios
            .FinishedLaunching((app, options) =>
            {
                AVFoundation.AVAudioSession session = AVFoundation.AVAudioSession.SharedInstance();
session.SetCategory(AVFoundation.AVAudioSessionCategory.Playback);
                session.SetActive(true);
                return true;
            }));
#endif
    })  

If everything works as expected, you should now be able to background stream audio on iOS as well. Also, the stream does not stop when the device is set to silent mode.

Interesting to know

There is so much more to know. For instance you could make the MediaElement itself invisible and add an own ImageButton to control the state of the player. This could be something you might want to consider when building a streaming client for an online radio. Because in this case you won't need a visible MediaElement - in contrast to a case scenario in which you stream a video for example.

I assume the most interesting parts in this article might be the hint about how to configure networking security for Android so that you can also stream from non encrypted HTTP addresses and of course the code snippet that allows you to register the MediaElements' iOS implementation as a shared instance so that you can stream in the background on your iPhone.

Shoutout goes to Gerald for his awesome work and now have fun building awesome apps with it!