DEV Community: Meher

Building a Scalable UI for Your Flutter Application Using Agora

Meher — Tue, 12 Oct 2021 15:48:13 +0000

One of the biggest challenges facing any developer is building applications that can scale. With a video-conferencing application using Agora, the main scaling issue is the bandwidth of your local device, especially if there are many incoming video streams. And as the number of participants rises, it becomes increasingly difficult to make sure that your application can keep up.
In this tutorial, you will see how to use Agora to build an application that can scale to up to 17 users by optimizing the bandwidth usage of the incoming video streams.

Pre-requisites

An Agora developer account (see How to Get Started with Agora)
Flutter SDK
Editor of your choice
Android or iOS device for testing

Project Setup

In your project directory, open a terminal window and create a Flutter project by running this command:
flutter create agora_scalable_ui
Open the newly created project in the editor of your choice.
Navigate to pubspec.yaml and add the latest version of the Agora Flutter SDK as a dependency.
Add relevant permission in your AndroidManifest.xml file (Android) or Info.plist file (iOS):

Android (in your AndroidManifest.xml):

<uses-permission android:name="android.permission.INTERNET"/> 
<uses-permission android:name="android.permission.RECORD_AUDIO"/> 
<uses-permission android:name="android.permission.CAMERA"/>

iOS (in your info.plist): Add both NSCameraUsageDescription and NSMicrophoneUsageDescription to Info.plist along with text descriptions. See here for more information on these, along with code samples of how to request and get authorization states.

All done! Let's start building the app.

Build

We will stick with a very simple grid view. Our goal is to scale this application as the user joins. To do this, we will optimize our bandwidth usage to ensure all the streams work concurrently without any difficulty.

Building a Home Page

Our home page is responsible for prompting the user for the name of the channel they want to join and requesting camera and microphone permission.

To request user permission, we are using a plug-in called permission_handler.

Building the video call page

Adding all the required variables:

Initializing the Agora methods:

In the above code snippet, we set up the Agora engine with dual-stream mode enabled. Dual-stream is an Agora feature that allows a client to publish two streams at the same time. One stream is for a higher resolution and bitrate, and the other stream is for a lower resolution and bitrate. With this dual-stream setup, when a remote client subscribes to your stream, they can switch to a lower stream based on their bandwidth requirements.

Initializing the Agora SDK:

The _initAgoraRtcEngine function creates an object of the AgoraRtcEngine, which we use to refer to all the methods provided in the Agora SDK.

Adding Agora event handlers:

In the userJoined callback, we are doing the following:
Adding the UID of the remote video to a list of UIDs (_users) that we use as the dataset for all the users' UIDs present in the channel
Checking if the number of remote users has grown to five to know whether to switch to using the lower stream in dual-stream mode

In the userOffline callback, we are doing the following:
Removing the UID from the _users list
Switching back to the higher stream in the dual-stream mode if the number of remote users shrinks to three

Creating the grid view for local and remote user video

Here, we simply pass the UID present in the _users list to generate a list of views. These views are then used to generate a grid that scales automatically as the user joins.

Cleaning Up

Finally, it is time to write the dispose method. We clean up all the resources relevant to Agora here:

Testing

To test the application, you will need five or more users such that when they join the channel the video quality optimizes automatically in order to provide the best video calling experience.

To run the app, enter this command in your terminal:

flutter run

Conclusion

You now have a video chat application that can scale to up to 17 users by optimizing settings for incoming streams.

You can find a complete application using all of the above code on GitHub

Other Resources

To learn more about the Agora Flutter SDK and other use cases, see the developer guide here.
You can also have a look at the complete documentation for the functions discussed above and many more here.
And I invite you to join the Agora.io Developer Slack Community.

1-to-1 Video Chat App on Android Using the Agora 4.x SDK Preview

Meher — Fri, 17 Sep 2021 21:40:59 +0000

In this post, we’ll build a basic video chat app in 10 easy steps, using the Agora Video 4.0 SDK Preview for Android.

Prerequisites

Android Studio
(https://developer.android.com/studio)
Basic knowledge of Java and the Android SDK
(https://developer.android.com/training/basics/firstapp)
Agora.io developer account
(see: How To Get Started with Agora)

Step 1: Getting your App ID

Once you finish the sign-up process, you are redirected to the Agora Console. Open the Projects tab in the left-hand nav to see your default project’s App ID.

Step2: JCenter Integration

Add the following line in your project level /app/build.gradle:

allprojects {
    repositories {    ...**
        ***// Jit pack
        *maven { url ‘[https://www.jitpack.io'](https://www.jitpack.io%27/) }
    }
}

Add the following line in the /app/build.gradle file of your project:

dependencies {
    ...**
    ***//Agora RTC 4.0 SDK for video call
    *implementation "com.github.agorabuilder:agora-full-preview:4.0.0"**
**}

Step 3: Setting up the Agora App ID

Now it’s time to add your Agora App ID (see Step 1) to the Android project’s Strings.xml (app/src/main/res/values/Strings.xml).

<resources>
    <string name="app_name">Agora-Android-Video-Tutorial</string>
    <string name="agora_app_id">#ADD YOUR APP ID HERE#</string>
</resources>

Note: This guide does not implement token authentication which is recommended for all RTE apps running in production environments.
For more information about token based authentication within the Agora platform please refer to this guide: https://bit.ly/3sNiFRs

The next step is to add the appropriate permissions within Manifest.xml

The final step is to prevent obfuscation of the Agora classes. This might sound complex, but it’s really very simple. In the proguard-rules.pro file, add a
-keep class configuration for the Agora SDK.

-keep class io.agora.**{*;}

This prevents obfuscation of the Agora SDK public class names.

Note: **Ensure that the Android NDK plug-in is installed and set up for this project.

Step 4: Setting Up Views

Now that we have the Agora SDK integrated, let’s set up our UI. I will breeze through this portion since we are using standard UI elements.

In the example, I chose to use ImageView instead of Button for the various UI elements. Either works. The important part is to note that there are functions that we link to using the onClick property.

Step 5: Checking Permissions

I know what you must be thinking: “Didn’t we already set up the permissions?” Earlier, we let the application’s Manifest know which permissions our app plans to use, but we still have to explicitly request the user to grant these permissions. Don’t worry — this is the final step in getting the boilerplate project running, and it’s painless.

First, let’s declare which permissions we want to request:

// Permissions
private static final int *PERMISSION_REQ_ID *= 22;
private static final String[] *REQUESTED_PERMISSIONS *= {Manifest.permission.*RECORD_AUDIO*, Manifest.permission.*CAMERA*};

Next, we set up a couple of functions to help us. First, we have a method that requests permissions for a given permission string and code.

Next, we have a callback method that gets called after the user has responded to the permissions request prompt.

Last, in our class’s onCreate we check if our permissions were granted. If they were not, the above methods handle the requests

Step 6: Initializing the Agora SDK

Now that we have our view, we are ready to initialize the Agora SDK, set up the user profile, and set the video quality settings.

In the previous step, you may have noticed a couple of places that I call initAgoraEngine(). Before we can dive into the initialization, we need to make sure that our activity has access to an instance of the Agora RtcEngine.

In our MainActivity class, we need to declare a class property to store our instance of RtcEngine:

private RtcEngine mRtcEngine;

Now it’s time to initialize. After all the boilerplate setup, we are finally at the step where we can start playing with the Agora engine!

Go ahead and declare your initAgoraEngine() method in your class. In this function, we will create a new instance of the RtcEngine using the baseContext, the Agora App ID (declared above), and an instance of the RtcEngineEventHandler (we’ll get into this a little later).

Once we have our new instance, it’s time to set up our user’s session. Here we can set the Channel Profile to Communication, since this is a video chat and not a broadcasting application. This is also where we configure our video encoder settings.

Note: At this phase of the build, we can use the emulator to test our work, but I am choosing to use a mobile device.

Step 7: Connecting the Video Streams

Before we can join a call we need to be able to present the local video stream to the user via the UI elements we set up earlier (step 4). For reference, below is the UI element’s ID that our local video will render into:

android:id="@+id/floating_video_container"

In the first line, we declare a *SurfaceView that renders the stream from the camera. The second step is to set up the layout and link it with our local or remote video container. We then add a view to our container where we add our SurfaceView. This lets us play the camera feed in that particular container. Last, we pass the localView or remoteView, which is the object of our SurfaceView, to the engine as part of a VideoCanvas object. We leave the *uid parameter blank so that the SDK can handle creating a dynamic ID for each user.

Now, that we have our local video feed set up, we need to use a similar function to connect our remote video stream.

The main difference between the remote video and the local video is the user ID parameter that gets passed to the engine as part of the VideoCanvas object. The last line sets the fallback option: If the video degrades, the engine reverts to a low-quality video stream.

Step 8: Setting Up the SDK Event Handler

Earlier, I made a reference to the RtcEngineEventHandler, and now it’s time to declare it as a property of our MainActivity class. The engine will call these methods from the RtcEngineEventHandler.

Each event triggers some fairly straightforward functions, including one we wrote in the previous step. In the interest of brevity, I provide the code below, but I don’t give an in-depth breakdown.

Feel free to leave a comment if anything is unclear.

Step 9: Joining and Leaving Channels

I know what you’re thinking, step 9 ?! Don’t sweat it — the next two steps are very simple. Let’s start by joining a call.

Note: if you do not specify the uid when joining the channel, the engine will assign one

As you can see from the first line, the Agora SDK makes it simple. The engine calls joinChannel, passing in the channel name followed by the call to set up our local video stream (Step 7).

Note: With the 4.0 SDK you need to add the StartPreview() method after setting up your local video in order to start your camera feed.

Note: The remove video method is the same used in step 8

Leaving the channel is even simpler: The engine calls leaveChannel. Above, you’ll notice there are a few lines to remove the video stream subviews from each UI element.

Note: It is also important to use the stopPreview() method before leaving the channel in order to stop the camera feed.

Step 10: Adding Other Functionality

The last remaining parts are related to connecting the UI elements for toggling the microphone and video streams on the local device. Let’s start with the audio toggle.

First, we get the reference to our button and then check if it has been toggled on or off using isSelected(). Once we have updated the UI element state, we pass the button’s updated state to the RtcEngine.

Moving on to the video toggle:

As with the audio toggle, we check or update the button’s state using isSelected() and then pass that to the RtcEngine. To give a better visual representation of the video being muted, we hide or show the videoSurface.

I hope you enjoyed reading along and working together on creating a 1-to-1 video chat Android app using the Agora 4.0 SDK.

Full source code: **https://github.com/digitallysavvy/android-video-chat-demo/tree/ngsdk-update

If you have any questions or comments, I invite you to join the Agora Developer Slack community.

Add Live Streaming to your Android App using Agora 4.0 SDK Preview

Meher — Fri, 17 Sep 2021 20:54:36 +0000

“You’re live!” — that’s a phrase that used to be limited to celebrities and other famous people. But now we see everyone going live and sharing their life stories. We have seen the market evolve so much that live streaming has become one of the biggest sources of real-time engagement (RTE).

Building your own live streaming application can be difficult especially if scalability is a concern. Agora makes our lives much easier by providing an easy-to-use SDK that also gives developers low-level control of exactly how we want the streaming service to work. While using the Agora platform, you don’t have to worry about scalability, call drops, or latency.

In this tutorial, we will walk through how to add live streaming to an Android application with multiple broadcasters and audiences who can subscribe to their stream.

So let’s get started!

Requirements

An Agora developer account (see How to get started with Agora)
Android Studio
A basic understanding of Android development

Project Setup

Though I won’t be going over the UI building part of the app, I have created a simple UI for our application. You can head over to GitHub to get the design used in this tutorial: https://github.com/Meherdeep/agora-android-live-streaming/tree/design

Navigate to the strings.xml file, and add your App ID to the private_app_id resource. Make sure to then sync your Gradle files.

MainActivity.java

Now, let’s create a file called MainActivity.java. This file contains the logic of what the users see when they first open the app. The main UI elements include a radio button and a text box. The radio button allows a user to decide whether they want to be a host or an audience for the live stream. The text box allows the user to input the channel name they wish to join.

In our onCreate method, we add the user permission so that the application asks for camera and microphone permissions as soon as the page is built.

We’ll be adding two methods to our class:

The first method will be called when the user chooses an option from the radio buttons. We will be setting a variable accordingly. We will be setting it to a value that will determine whether the user is a broadcaster or an audience.

We then implement a function that is called when the user submits the details. Here we will be getting all the details we need and sending them over to the next activity.

VideoActivity.java

By default, audiences have their cameras disabled and microphones muted, but the broadcasters have complete access. So when users join as a broadcaster, their videos are automatically added to the interface.

So first we are adding some variables that we will be using throughout the class. We then modify the onCreate method to get the data that we had passed in from the previous activity.

We declare a method called initAgoraEngineAndJoinChannel that will call all the other methods needed in the life cycle of the broadcast. We also define the event handler that will decide what methods to call when a remote user joins or leaves or when the remote feed is muted.

Let’s take a deeper look into the initAgoraEngineAndJoinChannel() method and all the other methods referred in it.

initializeAgoraEngine() : This is a method used to initialize the RtcEngine.

setClientRole(): This method sets the role of a user to host or an audience (default). This method should be called before joining the channel. You can call it again after joining the channel to switch the client role.
setupLocalVideo(): This function is used to refer to the setupLocalVideo method from our AgoraRtcEngine, through which we set a surface view for our local user that is used in the live stream.

startPreview: This method allows you to add your local stream before joining a channel. It is important that before calling this method we have enabled the video and set up our local user video.
joinChannel(): A channel acts as a common room for people to be in the same video call. The joinChannel() method can be called like this:

This method takes four parameters to run successfully:

Token: Token authentication is recommended for all RTE apps running in production environments. For more information about token-based authentication in the Agora platform, see https://docs.agora.io/en/Video/token?platform=All%20Platforms.
Channel Name: It takes a string input to put users into a common video call.
Optional Info: This is an optional field through which you can pass additional information about the channel.
uid: It is the unique ID for every user who joins the channel. If you pass 0 or the null value, then Agora automatically assigns a uid for every user.

Note: This project is meant for reference purposes and development environments, it is not intended for production environments. Token authentication is recommended for all RTE apps running in production environments. For more information about token based authentication within the Agora platform please refer to this guide: https://bit.ly/3sNiFRs

In our initialize function, we passed mRtcEventHandler as one of the parameters to the create method. This sets up a series of callback events that are triggered whenever a user joins a channel or leaves a channel.

Using these callbacks we are calling functions as a user joins or leaves the channel.

setupRemoteVideo : When a remote user joins the call the remote container is selected to display the video of the remote user. Here we will be using FrameLayout to display the user video.

onRemoteUserLeft : When a remote user leaves a channel this function removes all the views from the container.

We then add the onDestroy method for releasing the resources that we have used.

The leaveChannel function here calls the stopPreview and leaveChannel methods.

You can also add features like switching the camera by calling the switchCamera() method using our RtcEngine’s object:

mRtcEngine.switchCamera()

Or muting the microphone by calling the muteLocalAudioStream() method using our RtcEngine’s object:

mRtcEngine.muteLocalAudioStream(true)

With this, you’re ready to go and build your own live streaming application.

Testing

Before you build the app, make sure that you did the following:

You added your App ID while initializing RtcEngine.
You added your channel name for both RtcEngine and RtcChannel.
If your project has App Certificate enabled, you used your token in the token variable.
You followed all the instructions correctly, such as adding the correct dependencies to the build.gradle file, adding the permissions to your manifest, and so on.

After you build the app, you should see something like this :

Here I have added two broadcaster streams while multiple audiences have subscribed to it.

Conclusion

Congratulations! You have implemented your own live video streaming application built using the Agora Android SDK.

You can get the complete code for this application here.

Other Resources

For more information about Agora.io applications, take a look at the Agora Video Call Quickstart Guide and Agora API Reference.

And take a look at the complete documentation for the functions discussed above and many more here.

I also invite you to join the Agora.io Developer Slack community.