DEV Community: DavidRelo

How to build a live streaming app?

DavidRelo for ZEGOCLOUD ・ Jul 29 '22

#android #java #api #programming

Video SDK for easier integration

DavidRelo — Thu, 29 Sep 2022 02:50:34 +0000

A wide variety of Video SDK

More and more companies are joining the video service industry, and various video SDKs are emerging. It has also become increasingly difficult for developers to choose the Video SDK.

How should we choose the Video SDK that suits us?

Simple becomes mainstream

The purpose of developers using the SDK is to obtain stable and high-quality services while saving development time.

Therefore, while ensuring the stability and quality of the SDK, the convenience of access has become the goal pursued by various enterprises.

It is a time saver for developers who build audio and video applications.

For example, the latest UIKits product launched by ZEGOCLOUD has brought the convenience of integrating the video SDK to a new height. Developers can realize the audio and video call function in only 30 minutes.

UI kits SDK provides 20+ UIKits, 50+ Components, supports Video calls, Voice calls, live streaming, and other scenarios to build, supports rich configuration item configuration, convenient and quick custom UI.

Integrate video SDK

Let's take iOS Video SDK as an example to experience the convenience brought by UIKits SDK step by step.

1) Add SDK to project

Introduce ZegoUIKitPrebuiltCall SDK through the pod as follows, add pod 'ZegoUIKitPrebuiltCall' in Podfile file. Then execute the command pod install in Terminal. For detailed operation, please refer to Quick Access Documentation.

target 'ZegoCallDemo' do
  use_frameworks!

  # Pods for ZegoCallDemo
  pod 'ZegoUIKitPrebuiltCall'

end

2) Import iOS video SDK to your project

In the file that needs to be called the SDK interface, import the SDK through import.

import ZegoUIKitSDK
import ZegoUIKitPrebuiltCall
// YourViewController.swift
class ViewController: UIViewController {
    //Other code...
}

3) Show the ZegoUIKitPrebuiltCallVC in your project

Next, you only need to display ZegoUIKitPrebuiltCallVC in the module that needs to start the video call to complete the SDK access.
Before entering the page, there are three steps that need to be done:

Go to ZEGOCLOUD Admin Console, get the appID and app Sign of your project.
Specify the userID and userName for connecting the Call Kit service.
Create a callID that represents the call you want to make.

// YourViewController.swift
class ViewController: UIViewController {
    // Other code...
    var userID: String = <#UserID#>
    var userName: String = <#UserName#>
    var callID: String = <#CallID#>

    @IBAction func makeNewCall(_ sender: Any) {

        let config: ZegoUIkitPrebuiltCallConfig = ZegoUIkitPrebuiltCallConfig()
        let audioVideoConfig: ZegoPrebuiltAudioVideoViewConfig = ZegoPrebuiltAudioVideoViewConfig()
        let menuBarConfig: ZegoBottomMenuBarConfig = ZegoBottomMenuBarConfig()
        config.audioVideoViewConfig = audioVideoConfig
        config.bottomMenuBarConfig = menuBarConfig
        let layout: ZegoLayout = ZegoLayout()
        layout.mode = .pictureInPicture
        let pipConfig: ZegoLayoutPictureInPictureConfig = ZegoLayoutPictureInPictureConfig()
        pipConfig.smallViewPostion = .topRight
        layout.config = pipConfig
        config.layout = layout


        let callVC = ZegoUIKitPrebuiltCallVC.init(yourAppID, 
                                                  appSign: yourAppSign, 
                                                  userID: self.userID, 
                                                  userName: self.userName, 
                                                  callID: self.callID, 
                                                  config: config)


        callVC.modalPresentationStyle = .fullScreen
        self.present(callVC, animated: true, completion: nil)
    }
}

4) Configure your project

Finally, you only need to add the camera and microphone permission to the iOS configuration file Info.plist, and then you can start to experience the audio and video call function.

<key>NSCameraUsageDescription</key>
<string>We require camera access to connect to a call</string>
<key>NSMicrophoneUsageDescription</key>
<string>We require microphone access to connect to a call</string>

Custom prebuilt UI

ZEGOCLOUD Call Kit provides a wealth of custom interfaces that you can modify according to your needs. For example, on the call page, we can achieve the following effects:

1) Display my view when my camera is off

If you want to still show your own video view when the camera is turned off, just set the showMyViewWithVideoOnly parameter in ZegoUIkitPrebuiltCallConfig to true to achieve this.

2) Hide my view when my camera is off

If you want to hide your own video view when closing the camera, just set the showMyViewWithVideoOnly parameter in ZegoUIkitPrebuiltCallConfig to false to hide it.

3) Dragging Small View

If you want to implement the function of dragging Video view, just set the isSmallViewDraggable parameter in ZegoUIkitPrebuiltCallConfig.

4) Switch the content of two video views

If you want to switch the content of two Video views, just set the switchLargeOrSmallViewByClick parameter in ZegoUIkitPrebuiltCallConfig.

Here is the reference code:

class ViewController: UIViewController {

    let selfUserID: String = "userID"
    let selfUserName: String = "userName"
    let yourAppID: UInt32 = YourAppID; // Fill in the appID that you get from ZEGOCLOUD Admin Console.
    let yourAppSign: String = YourAppSign; // Fill in the appSign that you get from ZEGOCLOUD Admin Console.

    @IBOutlet weak var userIDLabel: UILabel! {
        didSet {
            userIDLabel.text = selfUserID
        }
    }
    @IBOutlet weak var userNameLabel: UILabel! {
        didSet {
            selfUserName = String(format: "zego_%@", selfUserID)
            userNameLabel.text = selfUserName
        }
    }
    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view.
    }
    @IBAction func makeNewCall(_ sender: Any) {

        let config: ZegoUIkitPrebuiltCallConfig = ZegoUIkitPrebuiltCallConfig()

        let layout: ZegoLayout = ZegoLayout()
        layout.mode = .pictureInPicture
        let pipConfig: ZegoLayoutPictureInPictureConfig = ZegoLayoutPictureInPictureConfig()
        pipConfig.showMyViewWithVideoOnly = false;
        pipConfig.isSmallViewDraggable = true;
        pipConfig.switchLargeOrSmallViewByClick = true;
        layout.config = pipConfig
        config.layout = layout

        let callVC = ZegoUIKitPrebuiltCallVC.init(yourAppID, appSign: yourAppSign, userID: selfUserID, userName: self.selfUserName ?? "", callID: "100", config: config)
        callVC.modalPresentationStyle = .fullScreen
        self.present(callVC, animated: true, completion: nil)
    }
}

Did you know? 👏

Learn more

This is one of the live technical articles. Welcome to other articles:

How to choose android Voice Chat SDK

DavidRelo for ZEGOCLOUD ・ Sep 26 '22

How To Create Avatar From Photo

DavidRelo for ZEGOCLOUD ・ Sep 6 '22

How to build a live streaming app?

DavidRelo for ZEGOCLOUD ・ Jul 29 '22

How to use video call API to build a live video call app

DavidRelo ・ Jul 25 '22

How to choose android Voice Chat SDK

DavidRelo — Mon, 26 Sep 2022 07:16:33 +0000

The rapidly growing Voice Chat market

The popularity of Clubhouse has given a solid push to the Voice Chat SDK market, soliciting its spread and integration on other platforms. The voice-based social scene dramatically reduces the pressure on users to express themselves. It makes everyone more relaxed and confident talking to strangers on social media applications using only their voices.

How to build a Voice Chat Application

A complete Voice Chat App must provide functions such as:

call invitation
invitation notification
random matching
room management
call status management
network status monitoring
sound waves And more. The complete realization of these functions requires many human and material resources. Therefore, manufacturers encapsulate the full functions of voice chat into SDKs to provide external services. Some examples are LiveAudioRoom, UIKits, and Voice Call.

How to choose Android Voice SDK

Each manufacturer provides its voice chat SDK. Given the wide choice, how do developers pick the SDK that suits their business?
Let's take the android voice chat SDK as an example. In general, there are three SDKs categories on the market:

1) 'Room scene' Voice Chat SDK

The scene SDK focuses on the encapsulation logic of business scenarios, which implements the chat room scene's business logic and UI interface. Users only need to integrate the SDK into the application to realize a complete chat room function.

This type of SDK has the highest level of integration and can complete scene development in one day. Because of the high degree of integration, developers' degree of freedom is relatively low. If the SDK's business requirements and standard functions are pretty different, the cost of secondary development will be high.
For example: LiveAudioRoom SDK

2) Functional Voice Chat SDK

The functional SDK focuses on encapsulating functions into modules, such as the call function, notification function, voice call function, device management function, etc., required by the voice chat scenario.
Developers can choose the required functions according to their business needs and add business logic.

Compared with the scene SDK, the functional SDK has smaller granularity. Developers can focus on business logic development, directly integrating functions. This SDK reduces the developer's workload and ensures the developer's degree of freedom.
For example: UIKits SDK

3) Capability Voice Chat SDK

The capability SDK focuses on the support of underlying capabilities.
Such as the call function. The SDK provides audio transmission capabilities, but the developer needs to control when to start the transmission and which device to output audio.

Therefore, this type of SDK gives developers the most significant degree of freedom and can meet almost all the needs of developers.
Aside from implementing business logic, developers need to be responsible for the calling logic of the underlying capabilities. Users with a complete R&D team and particular business logic can choose the ability-based SDK.

For example Voice Call SDK

There is no best, only the most suitable option

Each type of SDK has various advantages and disadvantages, depending on specific needs.

Therefore, there is no absolute best SDK but it should be chosen according to what you need.

Now, let's take the functional Voice Chat SDK and see together a step-by-step guide to building an android voice chat app.

Build an android Voice Chat app

We take ZEGOCLOUD's functional Voice Chat SDK Call Kit as an example.

1）Integrated SDK

Refers to the documentation to introduce the SDK into the project.
The SDK integration can be completed by displaying the ZegoUIKitPrebuiltCallFragment component to realize the voice chat function.

public class CallActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_call);

        addCallFragment();
    }

    public void addCallFragment() {
        long appID = yourAppID;
        String appSign = yourAppSign;

        String callID = yourCallID;
        String userID = yourUserID;
        String userName = yourUserName;

        ZegoUIKitPrebuiltCallConfig config = new ZegoUIKitPrebuiltCallConfig();


        ZegoUIKitPrebuiltCallFragment fragment = ZegoUIKitPrebuiltCallFragment.newInstance(
            appID, appSign, callID, userID, userName,config);


        getSupportFragmentManager().beginTransaction()
            .replace(R.id.fragment_container, fragment)
            .commitNow();
    }
}

2) Custom UI

ZEGOCLOUD's Call Kit provides a wealth of custom interfaces; we only need to configure the parameters according to business requirements.
The configuration items implemented in the example below are:

camera is not turned on by default when entering a call
the interface displays the audio call page
the bottom button only displays the microphone,
hang up
speaker switch buttons.

    ZegoUIKitPrebuiltCallConfig config = new ZegoUIKitPrebuiltCallConfig();
    config.turnOnCameraWhenJoining = false;
    config.audioVideoViewConfig.showCameraStateOnView = false;
    config.bottomMenuBarConfig.buttons = Arrays.asList(
        ZegoMenuBarButtonName.TOGGLE_MICROPHONE_BUTTON,
        ZegoMenuBarButtonName.HANG_UP_BUTTON,
        ZegoMenuBarButtonName.SWITCH_AUDIO_OUTPUT_BUTTON
    );

If you are thinking of building a voice chat app, pick up a voice chat SDK and take action.

Did you know? 👏

Learn more

This is one of the live technical articles. Welcome to other articles:

How To Create Avatar From Photo

DavidRelo for ZEGOCLOUD ・ Sep 6 '22

How to build a live streaming app?

DavidRelo for ZEGOCLOUD ・ Jul 29 '22

How to use video call API to build a live video call app

DavidRelo ・ Jul 25 '22

Realistic Avatar Creator: the real you in the Virtual World

DavidRelo — Wed, 14 Sep 2022 06:26:10 +0000

Nowadays, we can reshape our world by making realistic Avatar to experiment with digital versions of ourselves, crossing the vast Web horizons. We have learned how personalization and versatility are what characterize avatars. The custom avatars have different features: some are realistic, others very imaginative and cartoonish. But what do users like the most?

Making an AVATAR: what do users want?

1) Realistic Avatar

Realistic avatars allow users to experience their actual life existence on the Internet. One can duplicate itself faithfully with powerful artificial intelligence algorithms and facial recognition technology.
Still, realistic avatars might make users feel constrained to carry their image and reality online, thus losing the freedom that the world wide web permits.

2) Realistic Cartoon Avatar

Realistic cartoon avatars allow users to have their lifelike avatars on the Internet while simultaneously enjoying the fun and creativity that comes with customizing their digital selves. Cartoon characters tend to transmit that sense of freedom and escape from reality that the digital world allows.

3) Pixel Avatar

One of nowadays's top internet users' concerns is privacy. Blockchain technology's diffusion helps overcome this issue, and the use of Pixel Avatar is widespread. Pixel avatars safeguard privacy; however, given their abstract look, users lose the realistic and custom touch experience.
So, each type of avatar has its own advantages and disadvantages. What kind of avatar should we choose?

The main trend now is to become a realistic avatar creator. The imperative is to make our avatars just like social media profiles!

How to be a realistic avatar maker

The Virtual Avatar SDK launched by ZEGOCLOUD allows your users to upload a photo to make a realistic avatar.
This content is only supported in a Lark Docs

Make an avatar from a photo

Virtual Avatar SDK can quickly analyze avatar features in photos.

// Extract facial features based on the introduced image.
ZegoFaceFeature faceFeature = ZegoAvatarService.getInteractEngine().detectFaceFeature(bitmap);

// Create a ZegoCharacterHelper class to simply the implementation process for API call.
// The absolute path of basic resources.
mCharacterHelper = new ZegoCharacterHelper(getFilesDir().getAbsolutePath() + "/assets/base.bundle"); 

// Set the avatar creation coefficient.
mCharacterHelper.applyFaceFeature(faceFeature);

// Get display view
mZegoAvatarView = findViewById(R.id.zego_avatar_view);

// Display the avatar on the screen and call the API on a UI thread.
mCharacterHelper.setCharacterView(mZegoAvatarView);

Beautify and custom your avatar

After obtaining the initial avatar model, we can set various feature
values through the interface to beautify the avatar.
We can adjust Skin tone, Eyebrows, Eyes, Nose, Mouth, Lip, Mouth corner, Chin, Cheek, and more. For detailed parameters, please refer to Development Documentation.

// Set the avatar creation coefficient. For the value of faceshapeID, see the following table. Constants defined in ZegoCharacterHelper can be used directly.
mCharacterHelper.setFaceShape(ZegoCharacterHelper.FACESHAPE_EYE_SIZE, 0.5f)

Outfits and style

Virtual Avatar SDK provides various makeup and accessories such as eyebrows, tattoos, beards, cosmetic contact lenses, glass, earphones, earrings, hair, clothes, pants, shoes, and many others.
Users can render and replace these materials on their avatars in real time, building exclusive images in line with their preferences.
Please refer to Development Documentation.

// Ensure that the external directory of Packages is set before the API is called.
mCharacterHelper.setExtendPackagePath(getFilesDir().getAbsolutePath() + "/assets/Packages"); // Set the directory where the makeup, hair, glass, and other resource packages are stored.

// Set the glass. Ensure that the resource already exists in the path specified by setExtendPackagePath.
String packageID = "earphone7"; // earphone7 is the directory name of an earphone resource. Use the directory name under Packages provided by the ZegoAvatar SDK.
mCharacterHelper.setPackage(packageID);

Speech simulation

We see eye and lip movements when we interact with other people in the real world. The same applies to the metaverse.
The Virtual Avatar SDK of ZEGOCLOUD supports sound-driven mouth movements based on the sound waves of voice. This feature drives a virtual avatar to change its mouth shape in real-time.

// Start voice detection.
ZegoAvatarService.getInteractEngine().startDetectExpression(ZegoExpressionDetectMode.Audio,expression -> {
    // Drive mouth shape changes of the virtual avatar.
    mCharacterHelper.setExpression(expression);
});

Facial expression mirroring

Non-verbal communication represents a significant part of human communication. According to body language researcher Analysis by Albert Mehrabian, over 55% of communication is non-verbal. Digital avatars should be able to mimic facial expressions in real time and make natural eye contact.

By displaying people's expressions in real-time, we allow users to interact more naturally in a virtual environment.

The Virtual Avatar SDK provides the facial expression mirroring feature. This technology captures facial expressions on users' faces based on accurate recognition of face key points and 52 basic facial expression dimensions, including the face, tongue, and eyeball. It restores and renders them on virtual avatars in real-time.

// Start facial expression detection.
ZegoAvatarService.getInteractEngine().startDetectExpression(ZegoExpressionDetectMode.Camera, expression -> {
     // Ensure that mCharacterHelper is created, AvatarView is set, and a default avatar is set by running setDefaultAvatar or setAvatarJson.
    // Drive facial expression changes of the virtual avatar.
     mCharacterHelper.setExpression(expression);
});

Did you know? 👏

Learn more

This is one of the live technical articles. Welcome to other articles:

How To Create Avatar From Photo

DavidRelo for ZEGOCLOUD ・ Sep 6 '22

How to build a live streaming app?

DavidRelo for ZEGOCLOUD ・ Jul 29 '22

How to use video call API to build a live video call app

DavidRelo ・ Jul 25 '22

#mobile #android #programming #ios

Do you understand the Video API?

DavidRelo for ZEGOCLOUD ・ Jul 22 '22

How To Create Avatar From Photo

DavidRelo — Tue, 06 Sep 2022 04:13:40 +0000

What is an AVATAR?

Avatar is an electronic image representing a computer user in the digital world. Major games and social platforms have introduced the virtual character function, in which one can create an avatar from a photo.

The idea behind the users virtually represented on-screen is driven by the intention of creating an interactive experience. This way, people enjoy the content immersively and tend to be more conscious of the game's ethical questions. The same principle now applies to social media.

Avatar Market Outlook

Technological advancement improves mobile phone performance, network bandwidth, screen rendering, and other capabilities. Users have, therefore, higher expectations in their pursuit of immersive experiences. Virtual avatars and metaverse-related applications receive more attention.

According to Allied Market Research, the global metaverse market was worth $41.9 billion in 2020 and will reach $1,237 billion by 2030.
The compound annual growth rate from 2021 to 2030 is 40.4%.

How do we join the metaverse industry and seize the market in such a rapidly developing emerging market?

Make avatar from photo

Facebook, Reddit, Youtube, and other social media have launched the 3D Avatar function. Let users become avatar photo creators.

Let's explore how to quickly integrate the function of creating avatars from a photo in your application so that your customers can have their avatars to enjoy their interactive experience fully.

The ZEGOAvatar SDK launched by ZEGOCLOUD allows you to create your metaverse application quickly.

Create a virtual avatar in three steps

1) Get avatar data from photo

// Extract facial features based on the introduced image.
ZegoFaceFeature faceFeature = ZegoAvatarService.getInteractEngine().detectFaceFeature(bitmap);

2) Set a virtual avatar data

// Create a ZegoCharacterHelper class to simply the implementation process for API call.
// The absolute path of basic resources.
mCharacterHelper = new ZegoCharacterHelper(getFilesDir().getAbsolutePath() + "/assets/base.bundle"); 

// Set the avatar creation coefficient.
mCharacterHelper.applyFaceFeature(faceFeature);

3) Render the virtual avatar

// Get display view
mZegoAvatarView = findViewById(R.id.zego_avatar_view);

// Display the avatar on the screen and call the API on a UI thread.
mCharacterHelper.setCharacterView(mZegoAvatarView);

Make the app experience more realistic

The metaverse doesn't apply only to virtual avatars. It extends to outfits, actions, images, expressions, etc. of real people. Together with the virtual scenes, it makes the metaverse world.

ZEGOAvatar SDK realizes the connection between virtual and real through AI, being able to smoothly replicate the natural expressions and emotions of characters.

1) Facial expression mirroring

Capture and mirror the user's facial expressions to their avatar in real-time. The avatar mimics the same expressions as the user in real-time.

2) Speech simulation

Capture the user's speech to drive the avatar's facial and mouth movements in real-time.

3) Gesture recognition

Recognize the hand gestures of the user and control the avatar to perform the movements and facial expressions.

4) Body pose recognition

Recognize the body poses of the user and control the avatar to perform the movements and facial expressions.

Join us now and create your Avatar from photos with ZEGOAvatar SDK!

Did you know? 👏

Learn more

This is one of the live technical articles. Welcome to other articles:

How to build a live streaming app?

DavidRelo for ZEGOCLOUD ・ Jul 29 '22

How to use video call API to build a live video call app

DavidRelo ・ Jul 25 '22

#mobile #android #programming #ios

Do you understand the Video API?

DavidRelo for ZEGOCLOUD ・ Jul 22 '22

How to implement a video call app using iOS CallKit and ZEGOCLOUD

DavidRelo for ZEGOCLOUD ・ Jul 8 '22

Embed Video Call on Website with Prebuilt SDK

DavidRelo — Thu, 11 Aug 2022 10:53:00 +0000

This article will introduce how to embed Video Call into your website using the Prebuilt SDK with two lines of code.

Video call function: how to implement it?

Information transmission has evolved from text to pictures and now to video.

Video calls are becoming more and more a necessary feature for applications.

From IM, dating, and meetings to live streaming and education, applications in all walks of life will integrate the video call function.

For this function to be implemented, If you don't use the prebuilt SDK, there are some complex tasks to accomplish. For instance:

Interactive interface development - Although the video call has only one page, there are various interactive buttons; each button has multiple states, that can be switched in different ways, rendering the interactive interface quite hard to handle.
Audio and video management - Audio and video management require a lot of testing and verification to ensure the stability of calls, such as different platforms, devices, system versions, and network environments which all need to be targeted. It's a time-consuming effort.
User permissions, call status, and other business-related - includes camera and microphone switch permissions, call permissions, call duration, hang up, waiting, and call status. These states seem simple, but since each of them has multiple modification paths, the logic underpinning a correct state is complex. For example, if the call is 'hung up', there can be 'active hang-up', 'passive hang-up', network 'error hang-up', 'insufficient quota', etc.

Not every team can invest a lot of workforce in solving this heavy R&D work, yet Video Call is an essential product function. The solution is to use the Prebuilt SDK to implement the Video Call function quickly.

What is the Prebuilt SDK?

Video Call Prebuilt SDK sorts out the personalized interface and interaction in Video Calls by:

abstracting the standard functions and logic in Video Call
encapsulating these standardized functions and logic in the Prebuilt SDK.
integrating personalized interfaces and interactions into configuration items. Users only need two lines of code to implement the Video Call function; personalized interface and interaction can be achieved through item configuration.

ZEGOCLOUD Prebuilt web SDK

ZEGOCLOUD Prebuilt SDK is a set of standard solutions sorted out from the integration process of thousands of customers in Video Call customers.

Based on ensuring the high encapsulation of the Video Call function, this solution provides a concise configuration file, which is convenient for users to build and configure flexibly quickly.

Using the ZEGOCLOUD Prebuilt SDK enables:

An out-of-the-box video chat interface
Built-in bandwidth management
Cross-browser compatibility
Text chat
Participant list
Network quality monitoring
Recording
Call Invitation
Customizable branding LOGO

Step-by-step guide: embed ZEGOCLOUD Prebuilt

Register an account

Create an account in ZEGOCLOUD Official.

Create a project in ZEGOCLOUD Admin Console.

Two lines of code complete the access

ZEGOCLOUD's prebuilt SDK only needs two lines of code to complete SDK access.

Call the init interface to complete the SDK initialization.
Call the joinRoom interface to join the audio and video call.

<html>
  <style>
      #root {
         width: 100vw;
         height: 100vh;
      }
  </style> 
  <body>
  </body>
  <script crossorigin src="https://zegocloud.github.io/zegocloud_prebuilt_webrtc/ZegoPrebuilt/index.umd.js"></script>
    <script>
      const TOKEN = ''; // need to generate TOKEN first
      const prebuilt = ZegoPrebuilt.init(TOKEN);
      prebuilt.joinRoom({
         container: document.querySelector("#root"),
      });
    </script>
</html>

Custom prebuilt UI

ZEGOCLOUD's prebuilt SDK provides a concise configuration file, which is convenient for you to build and configure flexibly quickly.

 prebuilt.joinRoom({
        container: document.querySelector("#root"), // Mount the container, which fills the entire screen by default
        joinScreen: {
          visible: true; // Whether to display the device detection page, default display
          title:"Join Room"; // Device detection page title, default Join Room
          inviteURL:"https://zegocloud.github.io/zegocloud_prebuilt_webrtc/build/index.html?RoomID=test"; // Invite link, not displayed by default
        };
        micEnabled: true; // Whether to enable your own microphone, it is enabled by default
        cameraEnabled: true; // Whether to open your own camera, open by default
        userCanToggleSelfCamera: true; // Whether you can control your own microphone, enabled by default
        userCanToggleSelfMic: true; // Whether you can control the body's own camera, enabled by default
        deviceSettings: true; // Whether to display audio settings, default display
        chatEnabled: true; // Whether to enable chat, enabled by default
        userListEnabled: true; //Whether to display the member list, the default display
        notification: { // message notification in the lower left corner of the video
          userOnlineOfflineTips: true; //Whether to display member in and out, default display
          unreadMessageTips: true; // Whether to display unread messages, default display
        };
        leaveRoomCallback: () => void; // leave the room callback
        branding: {
          logoURL: "https://www.zegocloud.com/_nuxt/img/zegocloud_logo_white.ddbab9f.png", // Brand logo in the upper left corner, not displayed by default
        };
        leftScreen: true; // leave the room page, default is
  });

Run the demo

Try a live demo: See a live demo

Learn more about ZEGOCLOUD's Prebuilt SDK.

Did you know? 👏

Learn more

This is one of the live technical articles. Welcome to other articles:

How to build a live streaming app?

DavidRelo for ZEGOCLOUD ・ Jul 29 '22

How to use video call API to build a live video call app

DavidRelo ・ Jul 25 '22

#mobile #android #programming #ios

Do you understand the Video API?

DavidRelo for ZEGOCLOUD ・ Jul 22 '22

How to implement a video call app using iOS CallKit and ZEGOCLOUD

DavidRelo for ZEGOCLOUD ・ Jul 8 '22

How to build a live streaming app?

DavidRelo — Fri, 29 Jul 2022 08:18:00 +0000

What is a Live Streaming App?

Live streaming technology allows you to broadcast, watch, create and share videos in real-time. All you need to be able to live stream is an internet-enabled device, like a smartphone or tablet, and a platform (such as a website or app) to live stream.

Live streaming apps are rapidly developing, given this trend's tremendous opportunities in many fields. Depending on the domain, the basic requirements for building a live streaming app differ as follows:

Game live streaming app - The picture must be clear and smooth to construct a game live streaming app.
Entertainment live streaming app - Building an entertainment app requires various interactive functions such as gifts, barrage, and mini-games.
Shopping live streaming app - It must provide functions such as shopping cart and live broadcast playback.
Sports live streaming app - It is necessary to ensure a clear and smooth picture and provide a variety of resolutions for users to choose from.
Educational live streaming app - Interactive whiteboard, file sharing, screen sharing, and other functions are essential to delivering the best teaching experience to students.

Must-Have Features Of a Live Streaming app

Despite the differences, there are some standard functions that all live streaming apps should have, such as:

Client join/sign-in - A straightforward enrollment structure for the clients to make a record and sign in to the application with the certification.
Search - A search box allowing users to search content by topic, popularity, trending, channel, location, or interests.
Client profile - Show the client's data and profile picture to companions and subscribers.
Live streaming - The app's core permits the client to record and communicate a live stream to individuals who subscribe to their channel or the public.
Chats - Adding a chat function helps the host communicate with the audience in real-time.

Building a live streaming app in different fields will require integrating some unique features. Relevant examples are:

Screen Sharing - For a game live streaming app, you must share the game screen with the audience through the screen sharing function.
Host PK - Required for the entertainment live streaming apps. Through the PK between anchors, one can enrich the content of the live streaming app, enhancing the audience's enthusiasm by sending gifts.
Shopping Cart - The live shopping streaming app provides a shopping portal for viewers to place orders and purchase products directly in the live streaming room.
Share whiteboard or file - The education live streaming app must share various teaching courseware, so providing functions such as shared whiteboard and transferred files to enrich the teaching form is necessary.

What is a Live Streaming SDK?

The Live Streaming App is complex, especially if each function is self-developed. It requires a vast technical team and consumes many human and financial resources.

Many companies encapsulate the standard functions of Live Streaming apps into SDK, thus, developing a Live Streaming app only needs to focus on business interaction development.

Among these functions:

Express SDK - Audio and video transmission service SDK encapsulates audio and video capture, preprocessing, encoding, transmission, decoding, rendering, and so on.
ZIM SDK - Message communication SDK, which encapsulates the transmission, and broadcasting of text messages, picture messages, video messages, and other messages, ability to notify.
SuperBoard SDK - Encapsulates interactive whiteboard, file management, file rendering, and other capabilities.
RoomKit SDK - Encapsulates a complete educational scene, including audio and video calls, barrage messages, shared whiteboards, shared files, equipment management, student management and other teaching scene logic.

Steps to Build a Live Streaming App

Prerequisites

Before you begin, make sure you complete the following steps:

Create a project in ZEGOCLOUD Console, and get the AppID of your project.
The ZEGO Express SDK has been integrated into the project. For details, see Integration.

Implementation process

The following diagram shows the basic process of User A playing a stream published by User B:

The following sections explain each step of this process in more detail.

Create a ZegoExpressEngine instance

To create a singleton instance of the ZegoExpressEngine class, call the createEngine method with the AppID of your project.

Login to a room

To log in to a room, call the loginRoom method.

Then, to listen for and handle various events that may happen after logging in to a room, you can implement the corresponding event callback methods of the event handler as needed. The following are some common event callbacks related to room users and streams:

Start the local video preview

To start the local video preview, call the startPreview method with the view for rendering the local video passed to the canvas parameter.

You can use a SurfaceView, TextureView, or SurfaceTexture to render the video.

Publish streams

To start publishing a local audio or video stream to remote users, call the startPublishingStream method with the corresponding Stream ID passed to the streamID parameter.

Then, to listen for and handle various events that may happen after stream publishing starts, you can implement the corresponding event callback methods of the event handler as needed. The following is a common event callback related to stream publishing:

Play streams

To start playing a remote audio or video stream, call the startPlayingStream method with the corresponding Stream ID passed to the streamID parameter and the view for rendering the video passed to the canvas parameter.

You can obtain the stream IDs of the streams published by other users in the room from the callback onRoomStreamUpdate.

You can use a SurfaceView, TextureView, or SurfaceTexture to render the video.

Stop publishing and playing streams

To stop publishing a local audio or video stream to remote users, call the stopPublishingStream method.

If local video preview is started, call the stopPreview method to stop it as needed.

To stop playing a remote audio or video stream, call the stopPlayingStream method with the corresponding stream ID passed to the streamID parameter.

Log out of a room

To log out of a room, call the logoutRoom method with the corresponding room ID passed to the roomID parameter.

Destroy the `ZegoExpressEngine` instance

To destroy the ZegoExpressEngine instance and release the resources it occupies, call the destroyEngine method.

API call sequence diagram

The following diagram shows the API call sequence of the stream publishing and playing process:

Did you know? 👏

Learn more

This is one of the live technical articles. Welcome to other articles:

How to Build a Clubhouse Clone App with Android and ZEGOCLOUD - A Social Audio App Development

DavidRelo for ZEGOCLOUD ・ Jun 29 '22 ・ 3 min read

Built an App to Watch Movies Together Online in a Few Hours (source code inside)

DavidRelo ・ Mar 11 '22 ・ 7 min read

Improve live streaming experience with stream mixing

DavidRelo ・ Jun 21 '22 ・ 4 min read

How to implement a video call app using iOS CallKit and ZEGOCLOUD

DavidRelo for ZEGOCLOUD ・ Jul 8 '22 ・ 5 min read

How to use video call API to build a live video call app

DavidRelo — Mon, 25 Jul 2022 12:07:33 +0000

Introduction

With the continuous improvement of device performance and network quality, more and more audio and video call scenarios have been applied, and video call API has also been applied more and more.

What capabilities can the video call API provide?

With the increasing demand for video call API, there are more and more companies providing video call API, but the functions provided are similar.

The capabilities provided by the video call API mainly include the following:

Call invitation
Voice and Video call
Offline push function
Call UI and interaction

Below is a live video call app built using ZEGOCLOUD's video call API.

What is ZEGOCLOUD's video call API?

ZEGOCLOUD's video call API uses a layered architecture. Details are as shown in the figure below:

ZEGOCLOUD's video call API provides two different SDKs for you: the CallUIKit and the Call SDK.

CallUIKit: Provide a complete live video call UI, you only need to call the call interface and hang up the interface where you need to complete the call process.

Call SDK: Provides the underlying logic of live video calls, such as audio and video management, call management, network status management, etc. You only need to focus on the implementation of the upper-layer custom UI and call the corresponding interactive interface to complete the call process.

How to use ZEGOCLOUD's Video call API

Step 1. Create a ZEGOCLOUD account

Create an account in ZEGOCLOUD Official.

Step 2. Create a new project

Create a project in ZEGOCLOUD Admin Console.

Step 3. Create a Firebase project

Create a Firebase project in the Firebase console. For more details, see Firebase Documentation.

Step 4. Deploy the Firebase Cloud Functions

ZEGO Call uses the Firebase Cloud Functions as the business server by default, we recommend you activate and deploy it before integrating the ZEGOCall SDK.

Create a new Realtime Database in Firebase.
Edit the rules of the Realtime Database by adding the following:

{
  "rules": {
        ".read": "auth.uid != null",
        ".write": "auth.uid != null",
  }
}

Install the CLI via npm.
```
npm install -g firebase-tools
```
Run the firebase login to log in via the browser and authenticate the firebase tool.
Run firebase init functions. The tool gives you an option to install dependencies with npm. It is safe to decline if you want to manage dependencies in another way, though if you do decline you'll need to run npm install before emulating or deploying your functions.

Download the Cloud function sample code.
Copy the firebase.json, functions\index.js files and functions\token04 folder in the sample code to your cloud function project, overwrite files with the same name.

Modify the index.js file, fill in the AppID and ServerSecret you get from ZEGOCLOUD Admin Console correctly.

In Firebase CLI, run the firebase deploy --only functions command to deploy the cloud functions.

Step 5. Copy file into the project

To integrate the ZEGOCallUIKit automatically with CocoaPods, do the following:

Download the Sample codes, and copy the ZEGOCallUIKit and ZEGOCall folders to your project directory (if you have no project, create a new one).

Open Terminal, navigate to your project directory, run the pod init command to create a Podfile, and then add the following to the file.

pod 'ZegoExpressEngine'

pod 'FirebaseAuth'
pod 'GoogleSignIn'
pod 'Firebase/Database'
pod 'Firebase/Analytics'
pod 'Firebase/Crashlytics'
pod 'Firebase/Messaging'
pod 'Firebase/Functions'

In Terminal, navigate to the directory where the Podfile locates, and run the pod install command.
```
pod install
```
Restart the Xcode, and open the newly generated .workspace file.

Step 6. Add permissions

Permissions can be set as needed.

Open Xcode, select the target object, and then click Info > Custom iOS Target Properties.

Click the Add button (+) to add camera and microphone permissions.

Privacy-Camera Usage Description
Privacy-Microphone Usage Description

Select the target project, select Sign&Capabilities > Capability, search for Background Modes and make a double click.
Check the following options in Background Modes.
Enable the Push Notification feature.
Download the config file GoogleService-Info.plist of firebase, and add it to your project. To get the config file, refer to How to get the config file?.
Set up the URL Types: Fill in the URL Schemes with the REVERSED_CLIENT_ID field in the GoogleService-Info.plist file.

Initialize the ZEGOCallUIKit

To initialize the ZEGOCallUIKit, get the CallManager instance, pass the AppID of your project.

To receive callbacks, set the corresponding delegate to self.

User login

ZEGO Call does not provide user management capabilities yet. You will need to have users log in to Firebase and then call the setLocalUser method to set user information to CallUIKit based on the login result.

Firebase provides multiple login authentication modes. The following uses Google login as an example.
For more modes, refer to the Firebase official.

Get a Token

ZEGO Call implements the voice and video call feature using the RTC SDK. The caller and called user join the same room and the stream publishing and stream playing start upon the call gets connected. Therefore, to make an outbound call, you will need to provide a token for the RTC SDK for validation. For details, see Use Tokens for authentication.

Before making or accepting an outbound call, you will need to get a Token from the Firebase Cloud Functions.

To set a Token to the token property of the CallManager, implement the agent-owned callback getRTCToken of the TokenProvider.

Make outbound calls

To make an outbound voice call, call the callUser method and set the type parameter to CallType.voice.

To make an outbound video call, call the callUser method and set the type parameter to CallType.video.

After making a call, the CallUIKit shows the UI of the current state automatically for both the callee and caller (integrated the CallUIKit is required), and, you can operate on the UI for further operations.

The following displays when the caller calls the method to make a call (make a voice call for illustration)
The following black pop-up prompts when the callee receives an incoming call (make a voice call for illustration)

Upload logs

When you encounter problems when using your app, call the uploadLog method to upload logs for us to locate and help you faster.

Deinitialize the ZEGOCallUIKit

After finishing using the ZEGO Call, call the unInit method to deinitialize the SDK.

Do you understand the Video API?

DavidRelo — Fri, 22 Jul 2022 02:10:00 +0000

What is video API?

Video API is an interface dedicated to providing audio and video transmission services, mainly divided into two types: static video API and live video API.

Static Video API
Static Video API is an API that provides video file play services. Service providers offer video file cloud storage and CDN distribution services and provide video services through protocol interfaces such as HTTP and RTMP.

For example, YouTube and Instagram use Static Video API.

Live Video API
For example, Live.me, Yalla, and Uplive use the Live Video API.

The static video API is easy to understand and pulls video files from the cloud through the streaming protocol.

The live video API is more complicated. How does it ensure the video can be transmitted to the other end quickly, smoothly, and clearly?

In this article, we will introduce the logic behind the live video API in detail.

What Can The Video API Do?

The live video API application is becoming more extensive with the continuous improvement of network bandwidth and device performance. It makes many scenarios possible, such as:

Live
Online education
Video conferencing
Telemedicine
Video calls
Multiplayer

What Happens After The Video API?

For the live video API, it is necessary to ensure that the video data can complete the end-to-end transmission within 500ms and, simultaneously, guarantee the clarity of the video picture and the flow of the call.

Therefore, the live video API needs to ensure that the audio and video data can realize end-to-end data transmission in a fast, large and stable manner. A complex system is required to ensure the availability of the live video API.

As shown in the figure, the live video API mainly layered the functions of 6 modules:

1. Audio and video capture

Audio and video capture is the source of audio and video data collected via cameras, microphones, screens, video files, recording files, and other channels.

It involves using color spaces, such as RGB and YUV, and extracting audio features, such as sampling rate, number of channels, bit rate, audio frame, etc.

2. Audio and video preprocessing

The audio and video preprocessing are mainly for business scenarios, and the collected data is processed once and again to optimize the customer experience.

For example:

Video data processing for beauty, filters, special effects, etc.
Audio data processing includes voice change, reverberation, echo cancellation, noise suppression, volume gain, etc.

3. Audio and video encoding

Audio and video coding ensures that audio and video data can be transmitted quickly and safely on the network.

Commonly used encoding formats are Video encoding format: H264, H265 Audio coding format: OPUS, AAC.

4. Audio and video transmission

Audio and video transmission is the most complex module in the video API. To ensure that the audio and video data can be transmitted to the opposite end quickly, stably, and with high quality in a complex network environment, streaming protocols such as RTP, HTTP-FLV, HLS, and RTMP can be used.

Various anomalies such as data redundancy, loss, out-of-order, flow control, adaptive frame rate, resolution, jitter buffering, etc., must be resolved using multiple algorithms.

So whether video API is worth choosing, we need to focus on whether the manufacturer has outstanding audio and video transmission advantages.

5. Audio and video decoding

Audio and video decoding means that after the audio and video data is transmitted to the receiving end, the receiving end needs to decode the data according to the received data encoding format.

There are two video decoding methods, hardware decoding, and software decoding. Software decoding generally uses the open source library FFMpeg. Audio decoding only supports software decoding; use FDK_AAC or OPUS decoding according to the audio encoding format.

6. Audio and video rendering

Audio and video rendering is the last step in the video API processing process. This step seems to be very simple. You only need to call the system interface to render the data to the screen.

However, one must process much logic to align the video picture with the audio. Still, now there is a standard processing method to ensure the synchronization of the audio and video.

How to use ZEGOCLOUD video API

Building a complete set of real-time audio and video systems is complex work. Nevertheless, many Video APIs help us solve the underlying complex operations. We only need to focus on the upper-level business logic.

The following will introduce how to use the ZEGOCLOUD Video API to implement the video calling function.

1. Implementation process

The following diagram shows the basic process of User A playing a stream published by User B:

The following sections explain each step of this process in more detail.

2. Optional: Create the UI

Before creating a ZegoExpressEngine instance, we recommend you add the following UI elements to implement basic real-time audio and video features:

A view for local preview
A view for remote video
An End button

3. Create a ZegoExpressEngine instance

To create a singleton instance of the ZegoExpressEngine class, call the createEngine method with the AppID of your project.

4. Log in to a room

To log in to a room, call the loginRoom method.

Then, to listen for and handle various events that may happen after logging in to a room, you can implement the corresponding event callback methods of the event handler as needed.

5. Start the local video preview

To start the local video preview, call the startPreview method with the view for rendering the local video passed to the canvas parameter.

You can use a SurfaceView, TextureView, or SurfaceTexture to render the video.

6. Publish streams

To start publishing a local audio or video stream to remote users, call the startPublishingStream method with the corresponding Stream ID passed to the streamID parameter.

Then, to listen for and handle various events that may happen after stream publishing starts, you can implement the corresponding event callback methods of the event handler as needed.

7. Play streams

To start playing remote audio or video stream, call the startPlayingStream method with the corresponding Stream ID passed to the streamID parameter and the view for rendering the video passed to the canvas parameter.

You can use a SurfaceView, TextureView, or SurfaceTexture to render the video.

8. Stop publishing and playing streams

To stop publishing a local audio or video stream to remote users, call the stopPublishingStream method.

If local video preview is started, call the stopPreview method to stop it as needed.

To stop playing a remote audio or video stream, call the stopPlayingStream method with the corresponding stream ID passed to the streamID parameter.

9. Log out of a room

To log out of a room, call the logoutRoom method with the corresponding room ID passed to the roomID parameter.

Did you know? 👏

Learn more

This is one of the live technical articles. Welcome to other articles:

How to Build a Clubhouse Clone App with Android and ZEGOCLOUD - A Social Audio App Development

DavidRelo for ZEGOCLOUD ・ Jun 29 '22

Built an App to Watch Movies Together Online in a Few Hours (source code inside)

DavidRelo ・ Mar 11 '22

Improve live streaming experience with stream mixing

DavidRelo ・ Jun 21 '22

How to implement a video call app using iOS CallKit and ZEGOCLOUD

DavidRelo for ZEGOCLOUD ・ Jul 8 '22

How to implement a video call app using iOS CallKit and ZEGOCLOUD

DavidRelo — Fri, 08 Jul 2022 14:21:17 +0000

Introduction

iOS CallKit is a new framework introduced by Apple on iOS10 in 2016 to improve the VoIP experience.

This article will explain how to use iOS CallKit and ZEGOCLOUD to build a Video Call App with a native calling experience.

Demo download

Source code download

What is ZEGOCLOUD?

ZEGOCLOUD is a global cloud communication service provider that was founded in June 2015.

It provides high-quality audio/video communication and IM services for more than 4,000 clients around the world.

What is CallKit？

CallKit is a framework that aims to improve the VoIP experience by allowing apps to integrate with the native phone UI. By adopting CallKit, your app will be able to:

Use the native incoming call screen in both the locked and unlocked states.
Start calls from the native phone app’s Contacts, Favorites, and Recents screens.
Interplay with other calls in the system.

CXProvider

Your app will use CXProvider to report any out-of-band notifications to the system. These are usually external events, such as an incoming call.

When such an event occurs, CXProvider creates a CXCallUpdate to notify the system.

CXCallUpdate

CXCallUpdate is used to encapsulate new or changing call-related information. The properties included in this class are:

Caller's name
whether a call is video or audio-only.

CXAction

CXAction is an abstract class that represents the phone's action. For different action, CallKit will provide different CXAction implementations.

For example, outgoing calls are represented by CXStartCallAction, and CXAnswerCallAction is used to answer incoming calls. Action is identified by a unique UUID.

When the system wants to notify the app that it has received an event, it will inform it in the form of CXAction.

The app communicates with CXProvider through CXProviderDelegate, this protocol defines the life cycle event methods of CXProvider, and calls CXAction.

CXCallController

Apps use CXCallController to let the system know about user-initiated requests, such as "call" actions. The biggest difference between CXProvider and CXCallController is that the job of CXProvider is to notify the system, while CXCallController makes requests to the user on behalf of the system.

CXCallController uses transactions when making requests. Transactions are represented by CXTransaction, which contains one or more CXAction instances. CXCallCotroller sends the transaction to the system, and if everything is OK, the system will respond with the corresponding CXAction to the CXProvider.

Incoming Calls

The following will explain how to use CallKit to implement the Incoming Call function.

When an incoming call comes in, the App will create a CXCallUpdate and send it to the system through CXProvider.
The system will issue an incoming call to its service.
When the user answers the call, the system will send a CXAnswerCallAction to the CXProvider.
App can respond to this animation by implementing the corresponding CXProviderDelegate protocol method.

ProviderDelegate

Create a ProviderDelegate.swift file in your project and add the following code:

ProviderDelegate needs to deal with CXProvider and CXCallController, so keep two references to both.
Initialize CXProvider with a CXProviderConfiguration, which will be defined as a static property later. CXProviderConfiguration is used to define the behavior and capabilities of the call.
To be able to respond to events from CXProvider, you need to set its delegate.
In this app, CXProviderConfiguration supports video call, and phone number handling, and limits the number of call groups to 1. For more customization, please refer to the CallKit documentation.

Just below the configuration, add the following helper method:

This utility method allows the App to report an incoming call through the CXProvider API. The code is explained as follows:

Prepare to report a call update event to the system, which contains all call-related metadata.
Call the reportIcomingCall(with:update:completion:) method of CXProvider to notify the system of an incoming call.

CXProviderDelegate

The next step is to ensure protocol conformance. Still in ProviderDelegate.swift, declare a new extension to conform to CXProviderDelegate:

CXProviderDelegate specifies only one required method, providerDidReset(_:). The provider invokes this method when reset, giving your app the opportunity to clean up any ongoing calls and revert to a clean state. In this implementation, you’ll terminate the ongoing audio session and dispose of any active calls.

Open AppDelegate.swift and start by adding a new property to the class:

The provider delegate is ready for use! Add the following method to AppDelegate.swift:

This method lets other classes access the provider delegate’s helper method.

The final piece of the puzzle is hooking up this call to the user interface. Open CallsViewController.swift, which is the controller for the main screen of the app. Find the empty implementation of unwindForNewCall(_:) and replace it with the following code:

Extracts the properties of the call from NewCallViewController, which is the source of this unwind segue.
The user can suspend the app before the action completes, so it should use a background task.

Ending the Call

Next, I will explain how to end a call. This App will support two ways to end a call:

The native interface ends the call.
End the call in the app.

Notice the difference between steps A and B. When the user ends the call from the in-call screen (A), the system automatically sends a CXEndCallAction to the provider.

However, if you want to end a call using Hotline (B), it’s your job to wrap the action into a transaction and request it from the system. Once the system processes the request, it will send the CXEndCallAction back to the provider.

CXProviderDelegate

However it supports ending calls, your app has to implement the necessary CXProviderDelegate method for it to work. Open ProviderDelegate.swift and add the following implementation to the class extension:

Now add the following methods to the class:

Create a CXEndCallAction. Pass in the call’s UUID to the initializer so it can be identified later.
Wrap the action into a transaction so you can send it to the system.
Invoke request(_:completion:) from the call controller. The system will request that the provider perform this transaction, which will in turn invoke the delegate method you just implemented.

Did you know? 👏

Learn more

This is one of the live technical articles. Welcome to other articles:

How to Build a Clubhouse Clone App with Android and ZEGOCLOUD - A Social Audio App Development

DavidRelo for ZEGOCLOUD ・ Jun 29 '22 ・ 3 min read

Built an App to Watch Movies Together Online in a Few Hours (source code inside)

DavidRelo ・ Mar 11 '22 ・ 7 min read

Improve live streaming experience with stream mixing

DavidRelo ・ Jun 21 '22 ・ 4 min read

Making and Receiving VoIP Calls with CallKit | Apple Developer Documentation

Use the CallKit framework to integrate native VoIP calling.

developer.apple.com

Enhancing VoIP Apps with CallKit - PDF Free Download

App Frameworks #WWDC16 Enhancing VoIP Apps with CallKit Session 230 Sirisha Pillalamarri CallKit Engineer Stuart Montgomery CallKit Engineer Nick Fraioli CallKit Engineer 2016 Apple Inc. All rights reserved.

docplayer.net

How to Implement Super Live Chat: 3 Methods You Should Know

DavidRelo — Mon, 04 Jul 2022 11:39:26 +0000

Introduction

Super live chat is a service that allows users to communicate with each other in real-time over the Internet.

Super live chat is an important function of popular apps on the market, such as WhatsApp and Facebook Messenger. Developing a super live chat architecture is much harder than it looks — numerous technologies have to be employed to meet high real-time demands, including massive concurrent requests, super-low latency, and 100% delivery of messages.

In recent years, driven by the rapid development of mobile technology and the growing trend for social features in the app market, Super live chat is widely used for communication on social media platforms, and in scenarios as diverse as e-commerce, live streaming, and customer service, etc.

Research shows that more than 60% of apps on the market have the super live chat feature, through which users can chat with each other in real-time, thus improving app engagement and user experience. For app developers, Super live chat implementation has become a must-have skill.

With more than 10 years of experience in developing super live chat apps, the author shares three options for super live chat implementation and hopes this article could be of help to super live chat developers.

How to implement Super live chat

Developers may build a super live chat app via
1) open-source code
2) in-house development
3) commercial SDK integration based on the project needs.

Client-side development, server-side development, and O&M are the three key parts for turning an idea into an IM product and keeping it up and running.

Client-side development involves mobile apps, desktop software, webpages, and mini-programs.
Server-side development is about the implementation of all super live chat functions, such as user access, relationship maintenance, sending/receiving messages, file and photo storage, and security audit.

Continuous O&M is required to ensure a stable and reliable environment on the server side for the IM app, such as scaling up the server in case of a surge in workload.

It's not easy to realize any of the above from scratch. In particular, in-house development requires a highly capable project team and abundant resources. Generally, large companies would develop software entirely on their own while others use open source codes or integrate a commercial SDK. Differences between the three methods are summarized as follows:

Table: 3 methods of Super live chat implementation

	Open source code	In-house development	Commercial SDK
Difficulty	Low	High	Medium
Functional scalability	Difficult; dependent on the open source project plan	Easy; independent of external conditions	Medium; dependent on SDK providers
O&M cost	High; in-house O&M	High; in-house O&M	In-house O&M not required
Product launch cycle	Medium	Long	Short
Target audience	Individuals learning app development	Large companies with strong R&D capabilities	Small and medium-sized enterprises and personal studios

Method 1: Use open-source projects

Using an open-source project is the fastest way to implement super live chat as you can embark on a fast-track product development by drawing on the wisdom of the whole community.

What should we focus on when choosing an open-source project for super live chat? The maturity of functions and the number of active users are two important aspects to consider when choosing an open-source project. Based on my experience, I would like to share with you two good open-source projects.

MongooseIM

Project URL: https://github.com/esl/MongooseIM

MongooseIM is Erlang Solutions' robust, scalable and efficient XMPP server, aimed at large installations. Specifically designed for enterprise purposes, it is fault-tolerant and can utilize the resources of multiple clustered machines.

OpenIM

Project URL: https://github.com/OpenIMSDK/Open-IM-Server

OpenIM was founded and developed by senior IM architects and IM/WebRTC experts, aiming at creating service value and developing lightweight, high-availability super live chat architecture through open source technology. With OpenIM, developers can easily build multiple IM and real-time interactive audio/video scenarios in an app.

An open-source project is a great option for developers who have to meet a tight deadline and without too many needs for customization to realize fast IM implementation. However, the project provides only simple and basic functions, and adding new functions relies heavily on its development progress. If you want to create a bespoke IM app, or if you expect a growing business, I advise you against using this method.

Teams with R&D capabilities usually prefer in-house development if they want a custom IM product. The next part will guide you through the technical difficulties and pitfalls in the process of in-house development.

Method 2: In-house development

Developing a super live chat product entirely by in-house teams requires a high cost and extensive, essential software development expertise within an organization. In addition, the development cycle may be lengthy. You need to make long-term planning to avoid missing business opportunities.

When developing a super live chat app in-house, we have met some technical difficulties in the following areas:

Reliability and order of messages.
Real-time messaging and pulling of messages in case of massive concurrent requests.
Message storage options: "pull" or "push" model.
Accurate calculation of unread messages.
Group chat acknowledgment messages.
Multi-data center deployment.

You may bundle some features of full-fledged products on the market into your app, such as file storage, security audit, and offline push notification, to accelerate the development. If you decide to develop such features by yourself in the future, you can easily replace them to maximize efficiency.

Deep-pocketed and technical-savvy enterprises that expect an enormous user base are advised to choose in-house development to fulfill their needs for functional scalability, fast iteration, and stable O&M.

Given the considerable financial and human resources required during this process, I recommend that in-house developers make a sound development plan to avoid unnecessary loss.

Here is a summary of the downsides of the two super live chat implementation methods we have discussed: If you choose an open-source project, it would be a bit difficult to add new functions and the O&M is complex, making it hard to support the product's long-term development. In-house development takes time and costs a lot of money.

Is there a compromise solution that needs a short development timeline, requires less investment, and supports customization? Yes. Integrating a commercial SDK is the easiest and a popular way to develop software applications. Many companies would choose SDKs from full-blown providers.

Method 3: Integrate a commercial SDK

This method is appealing because it:

enables quick implementation of an IM product to gain competitive edges.
offers stability to reduce O&M work.
supports functional scalability: If a new function is required, just get your request to the SDK provider.
requires much less cost than in-house development.

Process of super live chat implementation
When you choose to integrate a commercial SDK, follow the process below to implement super live chat:

Apply for a user account and get the password from the SDK provider.
Develop a service backend used to calculate the authentication information.
Integrate the SDK and develop your app.

Let's take sending/receiving messages as an example. Modules interact with each other, as shown below:

To sum up, if you choose this method, all you need to do is develop a simple service backend, integrate the SDK, and develop your app. With all being done, you can launch your app quickly.

Recommended SDK providers

There are many mature IM SDK providers on the market. I would like to recommend some of them.

ZEGOCLOUD

ZEGOCLOUD, a global cloud communication service provider that founded in June 2015.

It provides high-quality audio/video communication and IM services for more than 4,000 clients around the world.

ZEGOCLOUD's IM SDK is specially developed for audio/video communications. It can be used with voice and video SDKs to enable, for example, real-time engagement during live-streamed content. If your app's audio/video and the IM capabilities are built on products from different suppliers, users may experience a huge stream delay. ZEGOCLOUD's IM SDK can address this problem.

You can choose ZEGOCLOUD if you need to build IM into your audio/video-related platforms, such as live streaming, conference, education, and games.

CometChat

Founded in 2009, CometChat is committed to IM technology development and supports many platforms and frameworks, such as iOS, Android, React, Angular, and Vue.

It also provides developers with various solutions, such as Chat Widgets, Chat UI Kits, and Chat SDKs.

If you are developing a dating app with IM as the core feature, CometChat would be a good choice for you.

Learn more

This is one of the live technical articles. Welcome to other articles:

How to Build a Clubhouse Clone App with Android and ZEGOCLOUD - A Social Audio App Development

DavidRelo for ZEGOCLOUD ・ Jun 29 '22

Built an App to Watch Movies Together Online in a Few Hours (source code inside)

DavidRelo ・ Mar 11 '22

Improve live streaming experience with stream mixing

DavidRelo ・ Jun 21 '22