How to Integrate Active Speaker in Java for Android Video Chat App?

Introduction

Have you ever been on a crowded video call where you can hear the conversation, but you have no idea who is actually talking? Video conferences with numerous participants can be confusing, making it difficult to follow the conversation and engage effectively.

This is where the Active Speaker Integration comes in! This is an expert guide for Android developers who want to implement active speaker highlighting in their video apps (built with Java) using VideoSDK.

Highlighting the active speaker is especially beneficial in large meetings or webinars where numerous participants are present, which can lead to confusion about who is speaking at which time. By including this feature, you can improve the user experience on large group calls, encouraging more participation and better engagement.

🎯 Goals

By the End of this Article:

1. Create a VideoSDK account and generate your VideoSDK auth token.
2. Integrate the VideoSDK library and dependencies into your project.
3. Implement core functionalities for video calls using VideoSDK
4. Enable active speaker indication

🚀 Getting Started with VideoSDK

To take advantage of the active speaker highlighting functionality, we will need to use the capabilities that the VideoSDK offers. Before we dive into the implementation steps, let's make sure you complete the necessary prerequisites.

🔐 Create a VideoSDK Account

Go to your VideoSDK dashboard and sign up if you don't have an account. This account gives you access to the required Video SDK token, which acts as an authentication key that allows your application to interact with VideoSDK functionality.

🔑 Generate your Auth Token

Visit your VideoSDK dashboard and navigate to the "API Key" section to generate your auth token. This token plays a crucial role in authorizing your application to use VideoSDK features.

For a more visual understanding of the account creation and token generation process, consider referring to the provided tutorial.

🛠️ Prerequisites and Setup

Make sure your development environment meets the following requirements:

• Java Development Kit is supported.
• Android Studio version 3.0 or later.
• Android SDK API level 21 or higher.
• A mobile device with Android 5.0 or later version.

🔌 Integrate VideoSDK

Following the account creation and token generation steps, we'll guide you through the process of adding the VideoSDK library and other dependencies to your project. We'll also ensure your app has the required permissions to access features like audio recording, camera usage, and internet connectivity, all crucial for a seamless video experience.

STEP (a): Add the repositories to the project's settings.gradle file.

dependencyResolutionManagement{
  repositories {
    // ...
    google()
    mavenCentral()
    maven { url 'https://jitpack.io' }
    maven { url "https://maven.aliyun.com/repository/jcenter" }
  }
}

STEP (b): Include the following dependency within your application's build.gradle file:

dependencies {
  implementation 'live.videosdk:rtc-android-sdk:0.1.26'

  // library to perform Network call to generate a meeting id
  implementation 'com.amitshekhar.android:android-networking:1.0.2'

  // Other dependencies specific to your app
}

If your project has set android.useAndroidX=true, then set android.enableJetifier=true in the gradle.properties file to migrate your project to AndroidX and avoid duplicate class conflict.

STEP (c): Add permissions to your project

In /app/Manifests/AndroidManifest.xml, add the following permissions after </application>.

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.CAMERA" />

These permissions are essential for enabling core functionalities like audio recording, internet connectivity for real-time communication, and camera access for video streams within your video application.

Essential Steps for Building the Video Calling Functionality

We'll now delve into the functionalities that make your video application after set up your project with VideoSDK. This section outlines the essential steps for implementing core functionalities within your app.

This section will guide you through four key aspects:

STEP 1: Generate a meetingId

Now, we can create the meetingId from the VideoSDK's rooms API. You can refer to this documentation to generate meetingId.

STEP 2: Initializing the Meeting

After getting meetingId , the next step involves initializing the meeting for that we need to,

1. Initialize VideoSDK.
2. Configure VideoSDK with token.
3. Initialize the meeting with required params such as meetingId, participantName, micEnabled, webcamEnabled and more.
4. Add MeetingEventListener for listening events such as Meeting Join/Left and Participant Join/Left.
5. Join the room with meeting.join() method.

Please copy the .xml file of the MeetingActivity from here.

public class MeetingActivity extends AppCompatActivity {
  // declare the variables we will be using to handle the meeting
  private Meeting meeting;
  private boolean micEnabled = true;
  private boolean webcamEnabled = true;

  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_meeting);

    final String token = ""; // Replace with the token you generated from the VideoSDK Dashboard
    final String meetingId = ""; // Replace with the meetingId you have generated
    final String participantName = "John Doe";

    // 1. Initialize VideoSDK
    VideoSDK.initialize(applicationContext);

    // 2. Configuration VideoSDK with Token
    VideoSDK.config(token);

    // 3. Initialize VideoSDK Meeting
    meeting = VideoSDK.initMeeting(
            MeetingActivity.this, meetingId, participantName,
            micEnabled, webcamEnabled,null, null, false, null, null);

    // 4. Add event listener for listening upcoming events
    meeting.addEventListener(meetingEventListener);

    // 5. Join VideoSDK Meeting
    meeting.join();

    ((TextView)findViewById(R.id.tvMeetingId)).setText(meetingId);
  }

  // creating the MeetingEventListener
  private final MeetingEventListener meetingEventListener = new MeetingEventListener() {
    @Override
    public void onMeetingJoined() {
      Log.d("#meeting", "onMeetingJoined()");
    }

    @Override
    public void onMeetingLeft() {
      Log.d("#meeting", "onMeetingLeft()");
      meeting = null;
      if (!isDestroyed()) finish();
    }

    @Override
    public void onParticipantJoined(Participant participant) {
      Toast.makeText(MeetingActivity.this, participant.getDisplayName() + " joined", Toast.LENGTH_SHORT).show();
    }

    @Override
    public void onParticipantLeft(Participant participant) {
      Toast.makeText(MeetingActivity.this, participant.getDisplayName() + " left", Toast.LENGTH_SHORT).show();
    }
  };
}

STEP 3: Handle Local Participant Media

After successfully entering the meeting, it's time to manage the webcam and microphone for the local participant (you).

To enable or disable the webcam, we'll use the Meeting class methods enableWebcam() and disableWebcam(), respectively. Similarly, to mute or unmute the microphone, we'll utilize the methods muteMic() and unmuteMic()

public class MeetingActivity extends AppCompatActivity {
  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_meeting);
    //...Meeting Setup is Here

    // actions
    setActionListeners();
  }

  private void setActionListeners() {
    // toggle mic
    findViewById(R.id.btnMic).setOnClickListener(view -> {
      if (micEnabled) {
        // this will mute the local participant's mic
        meeting.muteMic();
        Toast.makeText(MeetingActivity.this, "Mic Disabled", Toast.LENGTH_SHORT).show();
      } else {
        // this will unmute the local participant's mic
        meeting.unmuteMic();
        Toast.makeText(MeetingActivity.this, "Mic Enabled", Toast.LENGTH_SHORT).show();
      }
      micEnabled=!micEnabled;
    });

    // toggle webcam
    findViewById(R.id.btnWebcam).setOnClickListener(view -> {
      if (webcamEnabled) {
        // this will disable the local participant webcam
        meeting.disableWebcam();
        Toast.makeText(MeetingActivity.this, "Webcam Disabled", Toast.LENGTH_SHORT).show();
      } else {
        // this will enable the local participant webcam
        meeting.enableWebcam();
        Toast.makeText(MeetingActivity.this, "Webcam Enabled", Toast.LENGTH_SHORT).show();
      }
      webcamEnabled=!webcamEnabled;
    });

    // leave meeting
    findViewById(R.id.btnLeave).setOnClickListener(view -> {
      // this will make the local participant leave the meeting
      meeting.leave();
    });
  }
}

STEP 4: Handling the Participants' View

To display a list of participants in your video UI, we'll utilize a RecyclerView.

(a) This involves creating a new layout for the participant view named item_remote_peer.xml in the res/layout folder. You can copy item_remote_peer.xmlfile from here.

(b) Create a RecyclerView adapter ParticipantAdapter which will be responsible for displaying the participant list. Within this adapter, define a PeerViewHolder class that extends RecyclerView.ViewHolder.

public class ParticipantAdapter extends RecyclerView.Adapter<ParticipantAdapter.PeerViewHolder> {

  @NonNull
  @Override
  public PeerViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
      return new PeerViewHolder(LayoutInflater.from(parent.getContext()).inflate(R.layout.item_remote_peer, parent, false));
  }

  @Override
  public void onBindViewHolder(@NonNull PeerViewHolder holder, int position) {
  }

  @Override
  public int getItemCount() {
      return 0;
  }

  static class PeerViewHolder extends RecyclerView.ViewHolder {
    // 'VideoView' to show Video Stream
    public VideoView participantView;
    public TextView tvName;
    public View itemView;

    PeerViewHolder(@NonNull View view) {
        super(view);
        itemView = view;
        tvName = view.findViewById(R.id.tvName);
        participantView = view.findViewById(R.id.participantView);
    }
  }
}

(c) Now, we will render a list of Participant for the meeting. We will initialize this list in the constructor of the ParticipantAdapter

public class ParticipantAdapter extends RecyclerView.Adapter<ParticipantAdapter.PeerViewHolder> {

  // creating a empty list which will store all participants
  private final List<Participant> participants = new ArrayList<>();

  public ParticipantAdapter(Meeting meeting) {
    // adding the local participant(You) to the list
    participants.add(meeting.getLocalParticipant());

    // adding Meeting Event listener to get the participant join/leave event in the meeting.
    meeting.addEventListener(new MeetingEventListener() {
      @Override
      public void onParticipantJoined(Participant participant) {
        // add participant to the list
        participants.add(participant);
        notifyItemInserted(participants.size() - 1);
      }

      @Override
      public void onParticipantLeft(Participant participant) {
        int pos = -1;
        for (int i = 0; i < participants.size(); i++) {
          if (participants.get(i).getId().equals(participant.getId())) {
            pos = i;
            break;
          }
        }
        // remove participant from the list
        participants.remove(participant);

        if (pos >= 0) {
          notifyItemRemoved(pos);
        }
      }
    });
  }

  // replace getItemCount() method with following.
  // this method returns the size of total number of participants
  @Override
  public int getItemCount() {
    return participants.size();
  }
  //...
}

(d) We have listed our participants. Let's set up the view holder to display a participant video.

public class ParticipantAdapter extends RecyclerView.Adapter<ParticipantAdapter.PeerViewHolder> {

  // replace onBindViewHolder() method with following.
  @Override
  public void onBindViewHolder(@NonNull PeerViewHolder holder, int position) {
    Participant participant = participants.get(position);

    holder.tvName.setText(participant.getDisplayName());

    // adding the initial video stream for the participant into the 'VideoView'
    for (Map.Entry<String, Stream> entry : participant.getStreams().entrySet()) {
      Stream stream = entry.getValue();
      if (stream.getKind().equalsIgnoreCase("video")) {
        holder.participantView.setVisibility(View.VISIBLE);
        VideoTrack videoTrack = (VideoTrack) stream.getTrack();
        holder.participantView.addTrack(videoTrack)
        break;
      }
    }
    // add Listener to the participant which will update start or stop the video stream of that participant
    participant.addEventListener(new ParticipantEventListener() {
      @Override
      public void onStreamEnabled(Stream stream) {
        if (stream.getKind().equalsIgnoreCase("video")) {
          holder.participantView.setVisibility(View.VISIBLE);
          VideoTrack videoTrack = (VideoTrack) stream.getTrack();
          holder.participantView.addTrack(videoTrack)
        }
      }

      @Override
      public void onStreamDisabled(Stream stream) {
        if (stream.getKind().equalsIgnoreCase("video")) {
          holder.participantView.removeTrack();
          holder.participantView.setVisibility(View.GONE);
        }
      }
    });
  }
}

(e) Now, add this adapter to the MeetingActivity

@Override
protected void onCreate(Bundle savedInstanceState) {
  //Meeting Setup...
  //...
  final RecyclerView rvParticipants = findViewById(R.id.rvParticipants);
  rvParticipants.setLayoutManager(new GridLayoutManager(this, 2));
  rvParticipants.setAdapter(new ParticipantAdapter(meeting));
}

🔊 Active speaker Integration

The active speaker highlight feature aims to visually indicate the participant currently speaking in the video call. This functionality is especially valuable in scenarios with a large number of participants, where it can be difficult to identify the source of the incoming audio.

This is how we can integrate this feature with VideoSDK:

Every time a participant actively speaks during the meeting, the onSpeakerChanged event is triggered. This event transmits the participant ID of the person who is currently speaking.

By capturing this event and leveraging the participant ID, you can visually highlight the corresponding participant in your application's user interface. This can be achieved by modifying the user interface element that represents the speaking participant, such as changing the background color or adding a visual indicator.

private final MeetingEventListener meetingEventListener = new MeetingEventListener() {
    @Override
    public void onSpeakerChanged(String participantId) {
        Toast.makeText(MainActivity.this, "Active Speaker participantId" + participantId, Toast.LENGTH_SHORT).show();
        super.onSpeakerChanged(participantId);
    }
};

To implement the dynamic speaker detection UI featured in the above image, feel free to check out our Github repository.

🎉 Conclusion

This expert guide has provided Android developers with complete instructions on how to implement active speaker highlighting in their video apps using VideoSDK. By visually indicating the active speaker, you can reduce confusion and improve engagement among participants.

To unlock the full potential of VideoSDK and create easy-to-use video experiences, developers are encouraged to sign up for VideoSDK and further explore its features. Start implementing Active Speaker Highlighting today to improve your video app functionality and user engagement.

Sign up for VideoSDK now to revolutionize your video app!