Adding Video Communication to A Multiplayer Unity Game

Do you ever imagine when you play against your friends on a multiplayer game on your lovely mobile device, you want to see each other’s facial expression or tease each other with jokes and funny faces? You’ve found a solution here without leaving the game itself to another chat App. In this tutorial we are going to take Unity’s popular Tanks game to the next level and make it into a game with live video chats!

Before we get started, there are a few prerequisites for anyone reading this article.

Prerequisites

• Unity (2018) and a Unity Developer account

• Knowledge of how to build your Unity project for iOS and Android

• A cross-platform mobile multiplayer Unity game (I chose to use Tanks!!!)

• An understanding of C# and scripting within Unity

• An Agora.io developer account

• At least two mobile devices (one iOS & one Android is ideal)

Project Setup

If you plan to use your own existing Unity project, go ahead and open it now and skip down to “Integrating Group Video Chat”.

For those readers that don’t have an existing project, keep reading; the next few sections are for you. (Note, this step is exactly the same to the Project Setup you may find in or have done most of the set up by following Hermes’ “Adding Voice Chat to a Multiplayer Cross-Platform Unity game” tutorial.)

New Unity Project

Please bear with me as the basic setup has a few steps and I’ll do my best to cover it swiftly with lots of images. Let’s start by opening Unity, creating a blank project. I recommend starting this project with the latest Unity 2018 LTS version. Note that the networking module UNet has been deprecated in 2019. So 2018 LTS is essentially the best option here.

Create a new project from Unity Hub

Download and import the “Tanks!!! Reference Project” from the Unity Store:

Searched by “Tanks Reference” and download this asset

When Unity prompts for if you want to overwrite the existing project with the new asset, click Yes. Furthermore, accept the API update prompt that will come up next.

Theres a couple more steps to getting the Tanks!!! reference project ready for building on mobile. First we need to enable Unity Live Mode for the project through the Unity dashboard. (select project → Multiplayer → Unet Config).

Set max players to 6 even though Tanks!!! limits the game to 4 players and click save

Once Unity Live Mode is enabled

Building for iOS

Now that we have Unity’s multiplayer enable, we are ready to build the iOS version. Let’s start by opening our Build Settings and switch our platform to iOS and build the project for testing.

Update the Bundle id and Usage Descriptions

Please note: you need to have Xcode installed and setup before attempting to build the project for iOS.

When building for the first time, create a new folder “Builds” and save the build as iOS

After the project has successfully built for iOS, we will see the project in the **Builds* folder*

Let’s open Unity-iPhone.xcodeproj, sign, and build / run on our test device.

Enable automatic signing to simplify the signing process. Remove In-App-Purchase if it shows up.

Don’t start celebrating just yet. Now that we have a working iOS build we still need to get the Android build running.

Building for Android

Android is a bit simpler than iOS since Unity can build, sign, and deploy to Android without the need to open Android Studio. For this section I’m going to assume everyone reading this has already linked Unity with their Android SDK folder. Let’s start by opening our Build Settings and switching our platform to Android.

Before we try to “Build and Run” the project on Android we need to make a couple adjustments to the code. Don’t worry this part is really simple, we only need to comment out a few lines of code, add a simple return statement, and replace one file.

**Some background:* the Tanks!!! Android build contains the Everyplay plugin for screen recording and sharing your game session. Unfortunately Everyplay shutdown in October 2018 and the plugin contains some issues that if not addressed will cause the project to fail to compile and to quit unexpectedly once it compiles.*

The first change we need to make is to correct a mistake in the syntax within the Everplay plugin’s build.gradle file. Start by navigating to our project’s Plugins folder and click into the Android folder and then go into the everyplay folder and open the build.gradle file in your favorite code editor.

Now that we have the Gradle file open, select all and replace it with the code below. The team that built Tanks!!! updated the code on GitHub but for some reason it didn’t make its way into the Unity Store plugin.

	// UNITY EXPORT COMPATIBLE
	apply plugin: 'com.android.library'
	repositories {
	mavenCentral()
	}
	buildscript {
	repositories {
	mavenCentral()
	}
	dependencies {
	classpath 'com.android.tools.build:gradle:1.0.0'
	}
	}
	dependencies {
	compile fileTree(dir: 'libs', include: ['*.jar'])
	}
	android {
	compileSdkVersion 23
	buildToolsVersion "25.0.3"
	defaultPublishConfig "release"
	defaultConfig {
	versionCode 1600
	versionName "1.6.0"
	minSdkVersion 16
	}
	buildTypes {
	debug {
	debuggable true
	minifyEnabled false
	}
	release {
	debuggable false
	minifyEnabled true
	proguardFile getDefaultProguardFile('proguard-android.txt')
	proguardFile 'proguard-project.txt'
	}
	}
	sourceSets {
	main {
	manifest.srcFile 'AndroidManifest.xml'
	java.srcDirs = ['src']
	aidl.srcDirs = ['src']
	renderscript.srcDirs = ['src']
	res.srcDirs = ['res']
	jniLibs.srcDirs = ['libs']
	}
	}
	lintOptions {
	abortOnError false
	}
	}

view raw build.gradle hosted with ❤ by GitHub

The last change we need to make is to disable EveryPlay. Why would we want to disable EveryPlay, you may ask. That’s because when the plugin tries to initialize itself it causes the Android app to crash. The fastest way I found was to update a couple lines within the EveryPlaySettings.cs, (Assets → Plugins → EveryPlay → Scripts) so that whenever EveryPlay attempts to check if it’s supported or enabled, we return false.

	public class EveryplaySettings : ScriptableObject
	{
	public string clientId;
	public string clientSecret;
	public string redirectURI = "https://m.everyplay.com/auth";
	public bool iosSupportEnabled;
	public bool tvosSupportEnabled;
	public bool androidSupportEnabled;
	public bool standaloneSupportEnabled;
	public bool testButtonsEnabled;
	public bool earlyInitializerEnabled = true;
	public bool IsEnabled
	{
	get
	{
	return false;
	}
	}
	#if UNITY_EDITOR
	public bool IsBuildTargetEnabled
	{
	get
	{
	return false;
	}
	}
	#endif
	public bool IsValid
	{
	get
	{
	return false;
	}
	}
	}

view raw EveryPlaySettings.cs hosted with ❤ by GitHub

Now we are finally ready to build the project for Android! Within Unity open the Build Settings (File > Build Settings), select Android from the Platform list and click Switch Platform. Once Unity finishes its setup process, open the Player Settings. We need to make sure our Android app also has a unique Package Name, I chose com.agora.tanks.videodemo.

You may also need to create a key store for the Android app. See this section of the PlayerSettings in Unity Editor:

Android KeyStore setting

Integrating Video Chat

For this project Agora.io Video SDK for Unity was chosen, because it makes implementation into our cross-platform mobile project, really simple.

Let’s open up the Unity Store and search for “Agora Video SDK”.

You only download the asset once, and then you can import it to different projects.

Once the plugin page has loaded, go ahead and click **Download. *Once the download is complete, click and *Import **the assets into your project.

Uncheck the last four items before import

You should then open the Lobby as your main scene. The following shows the also how the service page would look like for the multiplayer settings:

Discussion: in the following sections we will go through how the project to be updated with new code and prefabs changes. For those just want to quickly try out everything. Here is a plugin file to import all the changes. You will just need to enter the AppId to the GameSettings object as described after importing.

Modify the Tank Prefab

Let add a plane on to the top of the tank to render the video display. Find the CompleteTank prefab from the project. Add a 3D object Plane to the prefab. Make sure the follow values are updated for the best result:

• Y =8 for position; Scale to 0.7. Rotate -45 degrees on X, 45 degrees on Y.

• Do not cast shadow

• Disable the Mesh Collider script

Plane Prefab values

Attach VideoSurface.cs script from the Agora SDK to the Plane game object.

Save the change, and test the prefab in the game by going to Training to see the outcome. You should see a tank similar to the following screen:

Tank with Plane attached

Create UI for Mic/Camera Controls

Next, open the GameManager prefab and create a container game object and add three toggles under it:

• Mic On

• Cam On

• FrontCam

GameManager UI Canvas

That’s basically all the UI changes we need for this project. The controller script will be added to the prefab later in the following sections.

Controller Scripts

Next we will go over some scripts to make the video chat to work for this game. Before adding new scripts, we will modify a script to allow the input for the Agora AppId.

GameSettings

Two updates to the scripts to make the game to work with Agora SDK.

(1) Add a SerializedField here for the AppId.

Go to your Agora developer’s account and get the AppId (you may need to following the instruction to create the project first):

Agora AppId

In the Lobby scene of the Unity Editor, paste the value of the App ID there and save:

Set the App ID for Agora API

(2) Add support for Android devices by asking for Microphone and Camera permissions.

	using System;
	using UnityEngine;
	using Tanks.Map;
	using Tanks.Rules;
	using Tanks.Networking;
	using Tanks.Utilities;
	#if PLATFORM_ANDROID
	using UnityEngine.Android;
	#endif
	namespace Tanks
	{
	/// <summary>
	/// Persistent singleton for handling the game settings
	/// </summary>
	public class GameSettings : PersistentSingleton<GameSettings>
	{
	public event Action<MapDetails> mapChanged;
	public event Action<ModeDetails> modeChanged;
	[SerializeField]
	protected MapList m_MapList;
	[SerializeField]
	protected SinglePlayerMapList m_SinglePlayerMapList;
	[SerializeField]
	protected ModeList m_ModeList;
	[SerializeField]
	protected string mAgoraAppId;
	public string AgoraAppId
	{
	get { return mAgoraAppId; }
	}
	public MapDetails map
	{
	get;
	private set;
	}
	public int mapIndex
	{
	get;
	private set;
	}
	public ModeDetails mode
	{
	get;
	private set;
	}
	public int modeIndex
	{
	get;
	private set;
	}
	public int scoreTarget
	{
	get;
	private set;
	}
	public bool isSinglePlayer
	{
	get { return NetworkManager.s_Instance.isSinglePlayer; }
	}
	/// <summary>
	/// Sets the index of the map.
	/// </summary>
	/// <param name="index">Index.</param>
	public void SetMapIndex(int index)
	{
	map = m_MapList[index];
	mapIndex = index;
	if (mapChanged != null)
	{
	mapChanged(map);
	}
	}
	/// <summary>
	/// Sets the index of the mode.
	/// </summary>
	/// <param name="index">Index.</param>
	public void SetModeIndex(int index)
	{
	SetMode(m_ModeList[index], index);
	}
	/// <summary>
	/// Sets up single player
	/// </summary>
	/// <param name="mapIndex">Map index.</param>
	/// <param name="modeDetails">Mode details.</param>
	public void SetupSinglePlayer(int mapIndex, ModeDetails modeDetails)
	{
	this.map = m_SinglePlayerMapList[mapIndex];
	this.mapIndex = mapIndex;
	if (mapChanged != null)
	{
	mapChanged(map);
	}
	SetMode(modeDetails, -1);
	}
	/// <summary>
	/// Sets up single player
	/// </summary>
	/// <param name="map">Map.</param>
	/// <param name="modeDetails">Mode details.</param>
	public void SetupSinglePlayer(MapDetails map, ModeDetails modeDetails)
	{
	this.map = map;
	this.mapIndex = -1;
	if (mapChanged != null)
	{
	mapChanged(map);
	}
	SetMode(modeDetails, -1);
	}
	/// <summary>
	/// Sets the mode.
	/// </summary>
	/// <param name="mode">Mode.</param>
	/// <param name="modeIndex">Mode index.</param>
	private void SetMode(ModeDetails mode, int modeIndex)
	{
	this.mode = mode;
	this.modeIndex = modeIndex;
	if (modeChanged != null)
	{
	modeChanged(mode);
	}
	mode.rulesProcessor.GetColorProvider().Reset();
	scoreTarget = mode.rulesProcessor.scoreTarget;
	}
	protected override void Awake()
	{
	base.Awake();
	Debug.Assert(!string.IsNullOrEmpty(mAgoraAppId), "Agora AppId needs to be assigned in GameSettings!");
	Screen.sleepTimeout = SleepTimeout.NeverSleep;
	}
	#if PLATFORM_ANDROID
	string[] mPermissions = { Permission.Camera, Permission.Microphone };
	void Start()
	{
	foreach (var permit in mPermissions)
	{
	if (!Permission.HasUserAuthorizedPermission(permit))
	{
	Debug.LogWarning("Start: requesting permission " + permit);
	Permission.RequestUserPermission(permit);
	}
	}
	}
	#endif
	}
	}

view raw GameSettings.cs hosted with ❤ by GitHub

Agora Controller Scripts

Before jump right into the code, let’s understand what capabilities are needed. They are:

• Interface to the Agora Video SDK to do join channel, show video, mute microphone, flip camera, etc.

• Actual implementation for the Agora SDK event callbacks.

• Mapping from the Unity Multiplayer’s id to Agora user’s id.

• A manager to respond to the UI Toggle actions that we created earlier.

The following script capture shows the corresponding scripts hierarchy. A discussion about the four classes follows.

Controller Script Hierarchy

AgoraApiHandlerImpl: this class implements most of the Agora video SDK event callbacks. Many of them are placeholders. To support the minimum capability in this game, the following handlers are of the most interest:

• JoinChannelSuccessHandler — when local user joins a channel, this will corrospond to create a game and start a server. The server name is the same as the channel number for Agora SDK.

• UserJoinedHandler — when a remote user joins the game.

• UserOfflineHandler — when a remote user leaves the game.

SDKWarningHandler is commented out to reduce the noise in the debugging log. But it is recommended to enable it for an actual project.

	using UnityEngine;
	using agora_gaming_rtc;
	public class AgoraApiHandlersImpl
	{
	private IRtcEngine mRtcEngine;
	public AgoraApiHandlersImpl(IRtcEngine engine)
	{
	mRtcEngine = engine;
	BindEngineCalls();
	}
	~AgoraApiHandlersImpl()
	{
	mRtcEngine?.LeaveChannel();
	}
	protected void JoinChannelSuccessHandler(string channelName, uint uid, int elapsed)
	{
	string joinSuccessMessage = string.Format("joinChannel callback uid: {0}, channel: {1}, version: {2}",
	uid, channelName, IRtcEngine.GetSdkVersion());
	Debug_Log(joinSuccessMessage);
	AgoraPlayerController.instance.OnChannelJoins(uid);
	}
	protected void LeaveChannelHandler(RtcStats stats)
	{
	if (this != null) // because the destructor may call it
	{
	string leaveChannelMessage = string.Format("onLeaveChannel callback duration {0}, tx: {1}, rx: {2}, tx kbps: {3}, rx kbps: {4}",
	stats.duration, stats.txBytes, stats.rxBytes, stats.txKBitRate, stats.rxKBitRate);
	Debug_Log(leaveChannelMessage);
	}
	}
	protected void UserJoinedHandler(uint uid, int elapsed)
	{
	string userJoinedMessage = string.Format("onUserJoined callback uid:{0} elapsed:{1}", uid, elapsed);
	Debug_Log(userJoinedMessage);
	AgoraPlayerController.instance.AddAgoraPlayer(uid);
	}
	protected void UserOfflineHandler(uint uid, USER_OFFLINE_REASON reason)
	{
	string userOfflineMessage = string.Format("onUserOffline callback uid {0} {1}", uid, reason);
	Debug_Log(userOfflineMessage);
	}
	protected void VolumeIndicationHandler(AudioVolumeInfo[] speakers, int speakerNumber, int totalVolume)
	{
	if (speakerNumber == 0 || speakers == null)
	{
	Debug_Log(string.Format("onVolumeIndication only local {0}", totalVolume));
	}
	for (int idx = 0; idx < speakerNumber; idx++)
	{
	string volumeIndicationMessage = string.Format("{0} onVolumeIndication {1} {2}",
	speakerNumber, speakers[idx].uid, speakers[idx].volume);
	Debug_Log(volumeIndicationMessage);
	}
	}
	protected void UserMutedHandler(uint uid, bool muted)
	{
	string userMutedMessage = string.Format("onUserMuted callback uid {0} {1}", uid, muted);
	Debug_Log(userMutedMessage);
	}
	protected void SDKWarningHandler(int warn, string msg)
	{
	string description = IRtcEngine.GetErrorDescription(warn);
	string warningMessage = string.Format("onWarning callback {0} {1} {2}", warn, msg, description);
	Debug_Log(warningMessage);
	}
	protected void SDKErrorHandler(int error, string msg)
	{
	string description = IRtcEngine.GetErrorDescription(error);
	string errorMessage = string.Format("onError callback {0} {1} {2}", error, msg, description);
	Debug_Log(errorMessage);
	}
	protected void RtcStatsHandler(RtcStats stats)
	{
	string rtcStatsMessage = string.Format("onRtcStats callback duration {0}, tx: {1}, rx: {2}, tx kbps: {3}, rx kbps: {4}, tx(a) kbps: {5}, rx(a) kbps: {6} users {7}",
	stats.duration, stats.txBytes, stats.rxBytes, stats.txKBitRate, stats.rxKBitRate, stats.txAudioKBitRate, stats.rxAudioKBitRate, stats.users);
	// Debug.Log(rtcStatsMessage);
	int lengthOfMixingFile = mRtcEngine.GetAudioMixingDuration();
	int currentTs = mRtcEngine.GetAudioMixingCurrentPosition();
	string mixingMessage = string.Format("Mixing File Meta {0}, {1}", lengthOfMixingFile, currentTs);
	// Debug.Log(mixingMessage);
	}
	protected void AudioRouteChangedHandler(AUDIO_ROUTE route)
	{
	string routeMessage = string.Format("onAudioRouteChanged {0}", route);
	Debug_Log(routeMessage);
	}
	protected void RequestTokenHandler()
	{
	string requestKeyMessage = string.Format("OnRequestToken");
	Debug_Log(requestKeyMessage);
	}
	protected void ConnectionInterruptedHandler()
	{
	string interruptedMessage = string.Format("OnConnectionInterrupted");
	Debug_Log(interruptedMessage);
	}
	protected void ConnectionLostHandler()
	{
	string lostMessage = string.Format("OnConnectionLost");
	Debug_Log(lostMessage);
	}
	protected void Debug_Log(string text)
	{
	Debug.Log("[Agora] " + text);
	}
	protected void ReJoinChannelSuccessHandler(string channelName, uint uid, int elapsed)
	{
	Debug_Log(string.Format("ReJoinChannelSuccessHandler - channelName:{0} uid:{1} elapsed:{2}",
	channelName, uid, elapsed));
	}
	protected void AudioMixingFinishedHandler()
	{
	Debug_Log("AudioMixingFinishedHandler");
	}
	protected void OnFirstRemoteVideoDecodedHandler(uint uid, int width, int height, int elapsed)
	{
	}
	protected void OnVideoSizeChangedHandler(uint uid, int width, int height, int elapsed)
	{
	}
	protected void OnClientRoleChangedHandler(int oldRole, int newRole)
	{
	}
	protected void OnUserMuteVideoHandler(uint uid, bool muted)
	{
	}
	protected void OnMicrophoneEnabledHandler(bool isEnabled)
	{
	}
	protected void OnFirstRemoteAudioFrameHandler(uint userId, int elapsed)
	{
	}
	protected void OnFirstLocalAudioFrameHandler(int elapsed)
	{
	}
	protected void OnApiExecutedHandler(int err, string api, string result)
	{
	}
	protected void OnLastmileQualityHandler(int quality)
	{
	}
	protected void OnAudioQualityHandler(uint userId, int quality, ushort delay, ushort lost)
	{
	}
	protected void OnStreamInjectedStatusHandler(string url, uint userId, int status)
	{
	}
	protected void OnStreamUnpublishedHandler(string url)
	{
	}
	protected void OnStreamPublishedHandler(string url, int error)
	{
	}
	protected void OnStreamMessageErrorHandler(uint userId, int streamId, int code, int missed, int cached)
	{
	}
	protected void OnStreamMessageHandler(uint userId, int streamId, string data, int length)
	{
	}
	protected void OnConnectionBannedHandler()
	{
	}
	protected void OnNetworkQualityHandler(uint uid, int txQuality, int rxQuality)
	{
	}
	protected void BindEngineCalls()
	{
	mRtcEngine.OnAudioMixingFinished += AudioMixingFinishedHandler;
	mRtcEngine.OnApiExecuted += OnApiExecutedHandler;
	mRtcEngine.OnAudioQuality += OnAudioQualityHandler;
	mRtcEngine.OnAudioRouteChanged += AudioRouteChangedHandler;
	mRtcEngine.OnClientRoleChanged += OnClientRoleChangedHandler;
	mRtcEngine.OnConnectionBanned += OnConnectionBannedHandler;
	mRtcEngine.OnConnectionInterrupted += ConnectionInterruptedHandler;
	mRtcEngine.OnConnectionLost += ConnectionLostHandler;
	mRtcEngine.OnError += SDKErrorHandler;
	mRtcEngine.OnFirstLocalAudioFrame += OnFirstLocalAudioFrameHandler;
	mRtcEngine.OnFirstRemoteAudioFrame += OnFirstRemoteAudioFrameHandler;
	mRtcEngine.OnFirstRemoteVideoDecoded += OnFirstRemoteVideoDecodedHandler;
	mRtcEngine.OnJoinChannelSuccess += JoinChannelSuccessHandler;
	mRtcEngine.OnLastmileQuality += OnLastmileQualityHandler;
	mRtcEngine.OnLeaveChannel += LeaveChannelHandler;
	mRtcEngine.OnMicrophoneEnabled += OnMicrophoneEnabledHandler;
	mRtcEngine.OnNetworkQuality += OnNetworkQualityHandler;
	mRtcEngine.OnReJoinChannelSuccess += ReJoinChannelSuccessHandler;
	mRtcEngine.OnRequestToken += RequestTokenHandler;
	mRtcEngine.OnRtcStats += RtcStatsHandler;
	mRtcEngine.OnStreamInjectedStatus += OnStreamInjectedStatusHandler;
	mRtcEngine.OnStreamMessage += OnStreamMessageHandler;
	mRtcEngine.OnStreamMessageError += OnStreamMessageErrorHandler;
	mRtcEngine.OnStreamPublished += OnStreamPublishedHandler;
	mRtcEngine.OnStreamUnpublished += OnStreamUnpublishedHandler;
	mRtcEngine.OnUserJoined += UserJoinedHandler;
	mRtcEngine.OnUserMuted += OnUserMuteVideoHandler;
	mRtcEngine.OnUserMuteVideo += OnUserMuteVideoHandler;
	mRtcEngine.OnUserOffline += UserOfflineHandler;
	mRtcEngine.OnVideoSizeChanged += OnVideoSizeChangedHandler;
	mRtcEngine.OnVolumeIndication += VolumeIndicationHandler;
	// mRtcEngine.OnWarning += SDKWarningHandler;
	}
	}

view raw AgoraApiHandlersImpl.cs hosted with ❤ by GitHub

AgoraVideoController: this singleton class is the main entry point for the Tanks project to interact with the Agora SDK. It will create the AgoraApiHandlerImpl** **instance and handles interface call to join channel, mute functions, etc. The code also checks for Camera and Microphone access permission for Android devices.

	using UnityEngine;
	using agora_gaming_rtc;
	using Tanks.TankControllers;
	#if (UNITY_ANDROID && UNITY_2018_3_OR_NEWER)
	using UnityEngine.Android;
	#endif
	public class AgoraVideoController
	{
	public static AgoraVideoController instance
	{
	get
	{
	if (_instance == null)
	{
	_instance = new AgoraVideoController();
	}
	return _instance;
	}
	}
	private static AgoraVideoController _instance;
	private IRtcEngine mRtcEngine;
	private AgoraApiHandlersImpl agoraAPI;
	private AgoraVideoController()
	{
	Debug_Log("Agora IRtcEngine Version : " + IRtcEngine.GetSdkVersion());
	QualitySettings.vSyncCount = 0;
	Application.targetFrameRate = 30;
	mRtcEngine = IRtcEngine.GetEngine(Tanks.GameSettings.s_Instance.AgoraAppId);
	Debug.Assert(mRtcEngine != null, "Can not create Agora RTC Engine instance!");
	if (mRtcEngine != null)
	{
	agoraAPI = new AgoraApiHandlersImpl(mRtcEngine);
	}
	Setup();
	}
	private void Setup()
	{
	#if (UNITY_ANDROID && UNITY_2018_3_OR_NEWER)
	if (!Permission.HasUserAuthorizedPermission(Permission.Microphone))
	{
	Permission.RequestUserPermission(Permission.Microphone);
	}
	if (!Permission.HasUserAuthorizedPermission(Permission.Camera))
	{
	Permission.RequestUserPermission(Permission.Camera);
	}
	#endif
	mRtcEngine.SetLogFilter(LOG_FILTER.DEBUG | LOG_FILTER.INFO | LOG_FILTER.WARNING | LOG_FILTER.ERROR | LOG_FILTER.CRITICAL);
	}
	public void JoinChannel(string channelName, uint playerId = 0)
	{
	mRtcEngine.EnableVideo();
	mRtcEngine.EnableVideoObserver();
	mRtcEngine.JoinChannel(channelName, "extra", playerId); // join the channel with given match name
	Debug_Log(playerId.ToString() + " joining channel:" + channelName);
	}
	public void LeaveChannel()
	{
	Debug_Log("Leaving channel now....");
	mRtcEngine.DisableVideo();
	mRtcEngine.DisableVideoObserver();
	mRtcEngine.LeaveChannel();
	}
	public void MuteMic(bool mute)
	{
	mRtcEngine.MuteLocalAudioStream(mute);
	}
	private GameObject localVideoCache = null;
	public void MuteCamera(bool mute)
	{
	mRtcEngine.MuteLocalVideoStream(mute);
	if (mute)
	{
	mRtcEngine.DisableVideo();
	}
	else
	{
	mRtcEngine.EnableVideo();
	}
	GameObject localVideo = GameObject.Find(TankManager.LocalTankVideoName);
	if (localVideo != null)
	{
	localVideo.SetActive(!mute);
	localVideoCache = localVideo;
	}
	else if (localVideoCache != null)
	{
	localVideoCache.SetActive(!mute);
	}
	}
	public void SwitchCamera()
	{
	mRtcEngine.SwitchCamera();
	}
	void Debug_Log(string text)
	{
	Debug.Log("[Agora] " + text);
	}
	}

view raw AgoraVideoController.cs hosted with ❤ by GitHub

AgoraPlayerController: while Unity Unet library maintains the Network player’s profile, the Agora user’s id is created asynchronously. We will maintain a list of network player and a list of Agora user ids. When the game scene actually starts, we will bind the two list together into a dictionary so the Agora id can be looked up by using a Networkplayer’s profile. (We don’t need this binding mechanism if user id is known. In an actual production project, it is recommended to let the game server to provide the user ids to set for JoinChannel() call.)

	using System;
	using System.Collections.Generic;
	using UnityEngine;
	using TankNt = Tanks.Networking;
	public class AgoraPlayerController
	{
	private Dictionary<TankNt.NetworkPlayer, uint> NetworkToAgoraIDMap = new Dictionary<TankNt.NetworkPlayer, uint>();
	private List<TankNt.NetworkPlayer> m_NetworkPlayers = new List<TankNt.NetworkPlayer>();
	private List<uint> m_AgoraUserIds = new List<uint>();
	private static AgoraPlayerController _instance;
	public static AgoraPlayerController instance
	{
	get
	{
	if (_instance == null)
	{
	_instance = new AgoraPlayerController();
	}
	return _instance;
	}
	}
	private AgoraPlayerController()
	{
	// Listen to the network player join and leave events
	TankNt.NetworkManager.s_Instance.playerJoined += AddNetworkPlayer;
	TankNt.NetworkManager.s_Instance.playerLeft += RemoveNetworkPlayer;
	}
	~AgoraPlayerController()
	{
	if (TankNt.NetworkManager.s_Instance != null)
	{
	TankNt.NetworkManager.s_Instance.playerJoined -= AddNetworkPlayer;
	TankNt.NetworkManager.s_Instance.playerLeft -= RemoveNetworkPlayer;
	}
	}
	public void AddNetworkPlayer(TankNt.NetworkPlayer player)
	{
	Debug.LogFormat("Player joined----> {0}", player);
	if (player.isLocalPlayer)
	{
	Debug.Log("List: Player ignore for being LocalPlayer:" + player);
	}
	else
	{
	m_NetworkPlayers.Add(player);
	}
	}
	/// <summary>
	/// A networking user join callback occurs,save it with store networkid
	/// </summary>
	/// <param name="uid"></param>
	public void AddAgoraPlayer(uint uid)
	{
	m_AgoraUserIds.Add(uid);
	Debug.Log("Adding Agora player: " + uid);
	}
	/// <summary>
	/// Clear the ID mappings when starting a new game on new channel join
	/// </summary>
	/// <param name="uid"></param>
	public void OnChannelJoins(uint uid)
	{
	Reset();
	}
	/// <summary>
	/// Create the mapping from network player to agora id
	/// </summary>
	void Bind()
	{
	int total = Math.Min(m_AgoraUserIds.Count, m_NetworkPlayers.Count);
	for(int i=0; i<total; i++)
	{
	NetworkToAgoraIDMap[m_NetworkPlayers[i]] = m_AgoraUserIds[i];
	}
	}
	public void Reset()
	{
	m_NetworkPlayers.Clear();
	m_AgoraUserIds.Clear();
	NetworkToAgoraIDMap.Clear();
	}
	/// <summary>
	/// If a player leaves, update the id mapping
	/// </summary>
	/// <param name="player"></param>
	public void RemoveNetworkPlayer(TankNt.NetworkPlayer player)
	{
	Debug.LogWarningFormat("Player left and removed:{0}", player);
	if (NetworkToAgoraIDMap.ContainsKey(player))
	{
	NetworkToAgoraIDMap.Remove(player);
	}
	}
	/// <summary>
	/// Gets the Agora Id for a network player
	/// </summary>
	/// <param name="player"></param>
	/// <returns></returns>
	public uint GetAgoraID(TankNt.NetworkPlayer player)
	{
	if (NetworkToAgoraIDMap.Count == 0)
	{
	Bind();
	}
	if (NetworkToAgoraIDMap.ContainsKey(player))
	{
	return NetworkToAgoraIDMap[player];
	}
	return 0;
	}
	public void Print()
	{
	Debug.LogFormat("NetWorkPlayers:{0}", String.Join(" : ", m_NetworkPlayers ));
	Debug.LogFormat("AgoraIds:{0}", String.Join(" : ", m_AgoraUserIds ));
	}
	}

view raw AgoraPlayerController.cs hosted with ❤ by GitHub

• AgoraUIManager: position the container game object to top right location of the game screen. It provides three toggling functions:

1. Mic On : mute the audio input.

2. Cam On: mute the local camera streaming and turn off the display of the local player.

3. CamSwitch: switch between the front camera or the back camera on the mobile device.

	using UnityEngine;
	using UnityEngine.UI;
	public class AgoraUIManager : MonoBehaviour
	{
	void Start()
	{
	SetPosition();
	}
	void SetPosition()
	{
	transform.position = new Vector3(Screen.width - 20, Screen.height - 100, 0);
	}
	public void OnMicToggle(Toggle toggle)
	{
	Debug.Log("Toggle mic " + toggle.isOn);
	AgoraVideoController.instance.MuteMic(!toggle.isOn);
	}
	public void OnCamToggle(Toggle toggle)
	{
	Debug.Log("Toggle cam" + toggle.isOn);
	AgoraVideoController.instance.MuteCamera(!toggle.isOn);
	}
	public void OnCamSwitch(Toggle toggle)
	{
	Debug.Log("Switch cam " + toggle.isOn);
	AgoraVideoController.instance.SwitchCamera();
	}
	}

view raw AgoraUIManager.cs hosted with ❤ by GitHub

Tanks Code Modifications

We will interact the above controllers code into the existing project by updating the Tanks code in the following classes:

TankManager

1. Add a field to bring in the VideoSurface instance that we added to the Plane and drag the Plane game object from the children to the field.

1. Add a constant to name the video-surface.
   

   public const string **LocalTankVideoName **= "Video-Local";
   

2. Change code near the end of the initialize() method, where it looked like this before:

old code

The new code:

	// new code
	GameManager.AddTank(this);
	StartCoroutine(SetupVideoSurface(player));
	} // end of Initialize(TanksNetworkPlayer player)
	IEnumerator SetupVideoSurface(TanksNetworkPlayer player)
	{
	Debug.Log("Tank initializing player:" + player);
	yield return new WaitForFixedUpdate();
	if (player.hasAuthority)
	{
	DisableShooting();
	}
	player.CmdSetReady();
	if (player.isLocalPlayer)
	{
	if (videoSurface != null)
	{
	videoSurface.gameObject.name = "video-local";
	}
	}
	else
	{
	uint uid = AgoraPlayerController.instance.GetAgoraID(player);
	if (uid != 0 && videoSurface != null)
	{
	videoSurface.gameObject.name = LocalTankVideoName;
	videoSurface.SetForUser(uid);
	}
	else
	{
	Debug.Assert( uid != 0, "Couldn't find uid for player:" + player.playerId);
	Debug.Assert(videoSurface != null, "videoSurface = null");
	}
	}
	}

view raw TankManager.cs hosted with ❤ by GitHub

Discussion: here is the code to associate the plane display that we created earlier to render the video feed. The VideoSurface script handles this work. The only thing it needs is the Agora Id. If it is the local player, the Agora Id will be default to 0, and the SDK will automatically render the device’s camera video onto the hosting plane. If this is a remote player, then the non-zero Agora Id is required to get the stream to render.

Calls JoinChannel()

The JoinChannel() function calls in AgoraVideoController class establish the local player status and starts a channel server. There are three places to initiate the call.

• *CreateGame.cs: **add a line to the *StartMatchmakingGame() function inside the callback. It will look like this:

    private void StartMatchmakingGame()
    {
       GameSettings settings = GameSettings.s_Instance;
       settings.SetMapIndex(m_MapSelect.currentIndex);
       settings.SetModeIndex(m_ModeSelect.currentIndex);

       m_MenuUi.ShowConnectingModal(false);

       Debug.Log(GetGameName());
       m_NetManager.StartMatchmakingGame(GetGameName(), (success, matchInfo) =>
          {
             if (!success)
             {
                m_MenuUi.ShowInfoPopup("Failed to create game.", null);
             }
             else
             {
                m_MenuUi.HideInfoPopup();
                m_MenuUi.ShowLobbyPanel();
                AgoraVideoController.instance.JoinChannel(m_MatchNameInput.text);
             }
          });
    }

• LevelSelect.cs: add the call in OnStartClick(). And the function will look like this:

    public void OnStartClick()
    {
       SinglePlayerMapDetails details = m_MapList[m_CurrentIndex];
       if (details.medalCountRequired > m_TotalMedalCount)
       {
          return;
       }

       GameSettings settings = GameSettings.s_Instance;
       settings.SetupSinglePlayer(m_CurrentIndex, new ModeDetails(details.name, details.description, details.rulesProcessor));

       m_NetManager.ProgressToGameScene();
      AgoraVideoController.instance.JoinChannel(details.name);
    }

• LobbyServerEntry.cs: add the call in JoinMatch(). Modify the function signature to add string channelName. And the function will look like this:

    private void JoinMatch(NetworkID networkId, string channelName)
    {
       MainMenuUI menuUi = MainMenuUI.s_Instance;

       menuUi.ShowConnectingModal(true);

       m_NetManager.JoinMatchmakingGame(networkId, (success, matchInfo) =>
          {
             //Failure flow
             if (!success)
             {
                menuUi.ShowInfoPopup("Failed to join game.", null);
             }
             //Success flow
             else
             {
                menuUi.HideInfoPopup();
                menuUi.ShowInfoPopup("Entering lobby...");
                m_NetManager.gameModeUpdated += menuUi.ShowLobbyPanelForConnection;

                AgoraVideoController.instance.JoinChannel(channelName);
             }
          });
    }

   public void Populate(MatchInfoSnapshot match, Color c)
    {
       string[] split = match.name.Split(new char[1] { '|' }, StringSplitOptions.RemoveEmptyEntries);
       string channel_name = split[1].Replace(" ", string.Empty);
       m_ServerInfoText.text = channel_name**;

       m_ModeText.text = split[0];

       m_SlotInfo.text = string.Format("{0}/{1}", match.currentSize, match.maxSize);

       NetworkID networkId = match.networkId;

       m_JoinButton.onClick.RemoveAllListeners();
       m_JoinButton.onClick.AddListener(() => JoinMatch(networkId, **channel_name**));

       m_JoinButton.interactable = match.currentSize < match.maxSize;
   }

NetworkManager.cs

Insert code for player leaving the channel in Disconnect():

    public void Disconnect()
    {
       switch (gameType)
       {
          case NetworkGameType.Direct:
             StopDirectMultiplayerGame();
             break;
          case NetworkGameType.Matchmaking:
             StopMatchmakingGame();
             break;
          case NetworkGameType.Singleplayer:
             StopSingleplayerGame();
             break;
       }
       AgoraVideoController.instance.LeaveChannel();
    }

That’s basically all code changes we need to get the video streaming on local and remote player working! But wait, there is a catch we missed. The plane changes rotation with the tank when moving! See one of the tilted position:

Tank Moved with Plane

We will need another script to fix the rotation:

	using UnityEngine;
	namespace Tanks.Utilities {
	public class FixedRotation : MonoBehaviour
	{
	// Plane facing front
	readonly Quaternion m_Rotation = Quaternion.Euler(-45,45,0);
	void LateUpdate()
	{
	transform.rotation = m_Rotation;
	}
	}
	}

view raw FixedRotation.cs hosted with ❤ by GitHub

Build the project deploy the game to iOS or Android devices and start playing to a friend! You may see other person’s face (and yours) on top of the tanks and you can yell to each other now!

So, we are done building a fun project!

Bear vs Duck in a Tank Battle!

The current, complete code is hosted on Github.

Other Resources

• The complete API documentation is available in the Document Center.
• For technical support, submit a ticket using the Agora Dashboard or reach out directly to our Developer Relations team [e-mail]
• Come join the Slack community: https://agoraiodev.slack.com/messages/unity-help-me