DEV Community: LunarDrift

How I Added AI Auto-Reply to a Free Website Chat Widget in OpenClaw

LunarDrift — Thu, 19 Mar 2026 07:16:34 +0000

I'm a non-technical product person. Our team built Knocket, a 100% free website contact widget. It works great for live chat, but it had no AI features. I wanted to add smart auto-reply — so I used OpenClaw and vibe coding to build it myself. This is the full story: what I did, what broke, and what I shipped.

What Is Knocket?

Knocket is a free-forever website contact widget that combines live chat, offline forms, social links, and meeting scheduling in one embed. You paste one line of code before your </body> tag and your site gets a branded chat bubble in the bottom-right corner.

1-minute deploy — copy one <script> tag, paste it into your HTML, done
Live chat — visitors message you on your site, you reply from the Inbox dashboard
Offline forms — automatically collect visitor contact info when you're away
Social link aggregation — WhatsApp, Instagram, Telegram in one tap
Zero-code customization — colors, icons, greetings, all configurable from the dashboard

It's built on Tencent Cloud IM infrastructure. Our engineering team built the console, SDK, and frontend widget from scratch. And it's completely free — no $39/month Intercom bill, no $20/month LiveChat subscription.

→ Try Knocket for free

What Was Missing? I Wanted AI Auto-Reply

After launch, customers started embedding Knocket on their websites and using the Inbox to handle visitor messages. But there was a problem: Knocket has no AI features. It's a pure communication tool — when a customer sends a message, you have to be watching the dashboard to reply.

Even with working hours configured, no one sits in front of the Inbox console all day.

As a non-technical product person, I thought: can I add an AI customer service agent to my own product?

Mature customer service platforms offer this — for $30–100/month. Building it as a proper feature would take significant engineering time. But what if I could use OpenClaw to control the browser and do it for me?

The goal:

Customer sends a message → AI detects it → auto-replies (or notifies me first so I can decide)

So I started building. Here's what actually happened.

Phase 1: Browser Automation — Quick and Dirty (v1)

The Idea

I used OpenClaw, an AI coding assistant with a browser automation skill that could control Chrome via command line — execute JavaScript, fill forms, click buttons.

The initial approach was straightforward:

Every 60 seconds → read the Inbox conversation list
→ Detect new message → AI generates reply
→ Fill the textarea → click Send
→ Notify me via Telegram

The AI assistant wrote a ~300-line shell script. It used browser eval to read page data and browser fill to type the reply.

It Worked! ...For About 5 Minutes

The AI could read messages and generate replies. I was excited for approximately 5 minutes.

Then everything broke.

The browser automation's fill and click commands weren't compatible with Knocket Inbox's React single-page app. The worst incident: I meant to reply to 1 customer, but the same reply got sent to all 3 active conversations.

On top of that, every browser operation stole window focus. I couldn't use my browser for anything else while the script was running.

Lesson learned: CLI-wrapped browser automation isn't precise enough for complex SPAs. You can't control the timing of each step.

Phase 2: Raw CDP + Background Execution (v2)

Complete Architecture Rewrite

I threw out the browser automation approach entirely. The new plan: use OpenClaw to connect directly to Chrome DevTools Protocol (CDP) using Python's websockets library.

Why CDP was better:

Runs completely in the background — no popups, no focus stealing, no interference
Every step returns a precise JSON response — you know if it succeeded or failed
Full control over reconnection, retries, and timeouts

The new flow:

Customer sends message → script detects it in background
→ AI auto-replies → CDP sends it silently → Telegram notifies you

Stable, But I Wanted More

No more accidental cross-sends. Silent background operation. But I realized pure auto-reply wasn't enough.

For important customers or complex questions, I still wanted to review before responding.

A one-way webhook (notify me what happened) wasn't enough. I needed two-way communication: the bot tells me what's happening, I tell the bot how to respond.

Lesson learned: One-way notifications work for monitoring, not for decision-making. "Human-first, AI-fallback" requires a bidirectional bot.

Phase 3: Telegram Bot + Human-in-the-Loop (v3)

Why Telegram?

Bot API is completely free and full-featured
Bidirectional: bot notifies me, I reply to the bot, bot forwards to the customer
Works on mobile — respond from anywhere
No admin approval needed — unlike enterprise messaging platforms

The v3 Flow

Customer sends message
  → Script detects it in background
  → Telegram notifies you: what the customer said + conversation context
  → You reply on your phone (e.g., "Tell them we ship in 3 days")
    → Script sends your reply to the customer → Telegram confirms "✅ Sent"
  → 5 minutes with no reply from you?
    → AI auto-replies as fallback → Telegram tells you "🤖 AI auto-replied"

Human-in-the-loop. Important conversations get my attention. Everything else gets handled by AI.

And Then... It Broke Again at the Last Step

Telegram notifications worked perfectly. AI reply generation worked perfectly. The failure point: actually sending the reply into Knocket Inbox.

It looks like three simple steps: switch to the right conversation → type the reply → click Send. In practice, I hit every possible wall.

The Most Painful Part: CDP vs React — A Horror Story

Problem 1: Background Tab Can't Focus

Chrome has security restrictions on background tabs. JavaScript el.focus() silently fails — the textarea never gets focus, so you can't type into it.

Fix: Use CDP protocol-level DOM.focus command to bypass the JS security sandbox.

Problem 2: React Controlled Components Ignore CDP Input

Input.insertText writes text that's visible on screen, but JavaScript el.value returns empty. React's internal _valueTracker mechanism doesn't detect CDP-level input.

Fix: Stop relying on .value for verification. If focus succeeded and insertText didn't error, treat it as successful.

Problem 3: Text Duplicated Multiple Times

I built 4 different input methods as fallbacks. Method 1 actually worked, but since .value returned empty, the script thought it failed and ran methods 2, 3, and 4 — each writing the text again.

Fix: Use only one method. No fallbacks. The write probably succeeded — you just can't read it back.

Problem 4: Send Button Permanently Disabled

React didn't know the textarea had content, so the Send button stayed disabled.

Fix: Press Enter instead. If the textarea has focus, Enter sends the message.

Problem 5: Conversation Index Shifted

During the 5-minute human reply window, other conversations got added or removed. The original list index no longer pointed to the right conversation.

Fix: Look up conversations by name instead of index. Re-search before every send.

How Did I Solve All This?

Honestly — by talking to OpenClaw, round after round. My workflow:

Run the script → it breaks
Copy the logs and describe what happened: "Text duplicated twice, Send button never clicked"
OpenClaw analyzes → rewrites the code → restart
Run again → maybe breaks again → repeat

The core skill for non-technical vibe coding isn't reading code. It's describing problems accurately. "The text duplicated twice and the send button stayed disabled" is 100x more useful than "it doesn't work."

After about 15 iterations, the entire pipeline finally worked end to end.

What I Shipped: A Reusable AI Agent Skill

After surviving all the bugs, I packaged the solution into a reusable skill — knocket-inbox-agent.

What's in the Package

knocket-inbox-agent/
├── SKILL.md                  # Skill description (AI reads this to help you set up)
├── scripts/
│   ├── auto_reply.py         # Mode A: full auto-reply + Telegram notification
│   ├── telegram_human.py     # Mode B: Telegram human-first (recommended)
│   └── setup.sh              # One-click initialization
├── assets/
│   └── config.template.toml  # Configuration template
└── references/
    ├── usage-guide.md         # Beginner-friendly setup guide
    └── customization.md       # AI reply style customization

Two Modes

	Mode A: Full Auto	Mode B: Human-First (Recommended)
Flow	Customer message → AI replies immediately → Telegram notifies you	Customer message → Telegram notifies you → you reply or don't → AI fallback
Best for	High volume, standardized inquiries	Important customers, situations needing human judgment

How to Use It

Install the knocket-inbox-agent skill from GitHub
Tell OpenClaw: "I want to set up auto-reply for Knocket Inbox"
OpenClaw reads the skill and walks you through: pick a mode, configure environment, start monitoring

You don't need to write any code. OpenClaw handles everything based on the skill instructions.

What I Learned

On Vibe Coding

It's not magic. Don't expect "describe a feature and get perfect code." It's more like having a super-patient collaborator — OpenClaw lets you describe what's wrong, and it fixes it.
Non-technical people can absolutely do this. I don't understand CDP protocol or React internals. But I can describe problems clearly and identify key information in error logs.
Debugging is inevitable. But with OpenClaw's help, the cost of debugging drops from "stuck for days" to "iterate for a few rounds."

On Adding AI to Your Own Product

Knocket is a pure communication tool with zero AI features. But with this skill added, it becomes:

You're online → you make the calls, AI assists
You're asleep → AI handles everything, Telegram notifies you on your phone
No extra server needed. No paid SaaS subscription.
One laptop + Chrome + one Python script. That's the entire stack.

If you've built your own product or tool, ask yourself: what feature is it missing? Can vibe coding help you add it?

5 Real-Time Tools & Projects Redefining Developer Workflows in 2025

LunarDrift — Tue, 19 Aug 2025 13:41:02 +0000

Tired of juggling Jira tickets, Slack threads, and Zoom links? We all are.

Studies show that a single context switch can cost a developer up to 15 minutes of focus. Imagine how much more efficient we could be if explaining a complex UI bug meant a 5-minute live pair programming session instead of a long thread of text and screenshots.

That's why developers are increasingly shifting towards synchronous, real-time collaboration, embedding communication directly into their workflows.

Here are 5 top-tier tools and open-source projects that will completely change how you think about teamwork.

The Evolution of Collaboration: From Asynchronous to Synchronous

In the past, we relied on async tools like Jira and Trello, which are great for tracking tasks. But when it comes to complex problems and brainstorming, the inefficiencies and delays of async communication become a bottleneck.

The future of collaboration requires a balance between the structure of async and the immediacy of sync, defined by these key traits:

Contextual Integration: Collaboration features should be embedded in your dev environment, not as separate apps.
Low-Latency Interaction: Syncing audio, video, cursors, and whiteboards must be seamless and instant, as if you're in the same room.
Developer-First Design: Tools must respect developer habits and minimize interruptions.
Security & Reliability: Code and conversations must be secure during transit.

5 Tools & Projects Leading the Change

1. VS Code Live Share - The IDE-Integrated Pair Programming Standard

Almost every developer has used or heard of Live Share. It's the perfect example of "contextual integration." Without leaving your familiar VS Code environment, you can share your code, terminal, and even debugging sessions with your team.

Why it matters: Live Share has proven the immense value of real-time collaborative coding. It has set the gold standard for any team looking to improve their code review and remote pair programming efficiency.

2. Replit - The Multiplayer Cloud Development Environment

Replit moved the entire development environment to the cloud and natively supports a "multiplayer" mode. Team members can write, run, and debug code in the same project simultaneously, just like in Google Docs. This is incredibly valuable for rapid prototyping and team learning.

Why it matters: Replit represents the future of development—cloud-based environments with native collaboration. It eliminates discrepancies in local setups and ensures the entire team starts on the same page.

3. tldraw - The Minimalist & Powerful Open-Source Whiteboard

Complex architectural designs and UI/UX brainstorming sessions need a good whiteboard. tldraw is a popular open-source project that offers an infinite canvas for real-time, multiplayer drawing, diagramming, and interaction. It can also be easily embedded into any web app.

Why it matters: tldraw proves that real-time collaboration extends beyond just code. Visual communication is just as critical, and its open-source nature makes it a go-to choice for integrating an interactive whiteboard into internal tools or ed-tech platforms.

4. LiveKit - Open-Source RTC Infrastructure

For teams that want to control their RTC (Real-Time Communication) stack from the ground up, LiveKit provides a complete, open-source infrastructure. It allows you to self-host the service and offers flexible APIs for building highly customized real-time applications.

Why it matters: With a very active open-source community, LiveKit is an excellent resource for learning how RTC technology works. It empowers developers to dive deep into the mechanics of real-time communication.

5. Tencent RTC - The Real-Time Engine for Next-Gen Collaboration Tools

Ever wonder what technology powers the silky-smooth real-time sync in tools like VS Code Live Share or Figma? The answer is RTC. Tencent Real-Time Communication (RTC) is a leader in this field. It isn't a standalone app but a powerful engine that enables developers to build their own real-time collaboration tools.

Key Features:
- Cross-Platform SDKs: Provides SDKs for Web, iOS, Android, Desktop, and Electron, allowing you to easily integrate real-time audio and video into any application.
- Ultra-Low Latency: Achieves an end-to-end latency of under 300ms globally, thanks to Tencent's worldwide edge nodes, making interactions feel truly in-person.
- High Packet Loss Resistance: Maintains smooth audio and video communication even with up to 70% packet loss, gracefully handling poor network conditions.
- Rich Functionality: Goes beyond audio/video to offer real-time data synchronization, interactive whiteboards, and cloud recording to support complex collaborative scenarios.

Many top-tier internal DevOps platforms and developer tools choose Tencent RTC to provide their users with stable, high-quality real-time features, allowing them to focus entirely on their core business logic.

How to Build an Effective Real-Time Collaboration Culture

Great tools are just the beginning; you also need the right strategies:

Set Clear Boundaries: Define when to use async (e.g., daily updates) versus when to jump into a sync session (e.g., urgent bugs, architectural reviews) to avoid "real-time fatigue."
Embrace Pair Programming: Make it a regular practice, especially for complex modules or onboarding new members. It drastically improves code quality and knowledge sharing.
Record and Review: Use cloud recording to capture important technical discussions or debugging sessions, turning them into a valuable knowledge base for the team.

How Tencent RTC Can Empower Your Workflow

Integrating Tencent RTC's capabilities into your workflow means you can:

Eliminate App Switching: Launch HD audio/video calls directly from your CI/CD platform, project management tool, or internal documentation without jumping to a third-party app.
Build Custom Tools: Create a private, self-hosted pair programming or code review tool that perfectly fits your team's habits and keeps your code and conversations secure.
Improve Incident Response: When a production incident occurs, stakeholders can instantly join a "war room" embedded directly in your monitoring dashboard, syncing information in real-time and drastically reducing Mean Time to Resolution (MTTR).

Teams that adopt Tencent RTC typically report a 50% reduction in context switching and a 35% decrease in the time it takes to solve complex problems.

Conclusion: The Future of Collaboration is Seamless Integration

In 2025, the core of developer productivity is no longer just about optimizing individual coding speed but about elevating the entire team's collaborative efficiency.

The most successful teams will be those that seamlessly integrate real-time collaboration into their culture and toolchain—whether by adopting finished tools like VS Code Live Share or by using a powerful engine like Tencent RTC to build their own unique, customized platforms.

Ready to transform your development workflow? Explore how Tencent RTC can bring a smooth, in-person collaborative experience to your team, minimizing communication overhead and giving you more time to create. Your path to enhanced productivity starts here.

Build a Conversational AI Fandom APP

LunarDrift — Mon, 02 Jun 2025 02:29:40 +0000

Today’s fans demand more than just passive viewing. They seek deep immersion, real-time connection, and the opportunity to interact with their favorite idols in everyday life. Conversational AI opens the door to these possibilities, offering 24/7 interactive virtual personas, personalized companion modes, and seamless integration with physical merchandise. This advanced technology brings artists, influencers, and IP characters to life in a dynamic digital ecosystem.

The Evolution of Fan Platforms

Traditional fan platforms relied primarily on text chats and simple subscription models, often lacking the depth of interaction and emotional connection fans now crave. As a result, many fans found little reason to return or engage on a deeper level.

Traditional Fan Engagement

• Limited Interactivity: Artists and influencers have limited time for live streaming or direct fan contact.

• Low User Stickiness: Minimal emotional linkage leads to fewer repeat visits.

• Basic Monetization: Simple subscription models miss out on the full potential of fan enthusiasm.

AI-Powered Fan Engagement

• 24/7 Interaction: Conversational AI empowers fans with always-on chat, voice, and video engagement.

• Immersive AI Merchandise: AI-integrated toys, devices, and other real-world products create deeper emotional experiences.

• Diverse Revenue Streams: Offer premium AI companions, IP licensing, limited-edition collectibles, and virtual goods.

Round-the-Clock Connection: Real-Time Virtual Personas

Conversational AI enables fans to interact with virtual iterations of celebrities, influencers, or IP-based characters at any time. Fans can choose various “companion modes”—from “friend” to “mentor”—to get personalized content and emotional support. This fosters fan loyalty and continually boosts the value and visibility of the IP or celebrity brand.

Licensing IP for Conversational AI

Entertainment companies can license the voice, appearance, and personality of a star or influencer to create AI assistants or tutorial services. Whether it’s language learning or motivational tidbits, fans can transform their devotion into daily functionality. Imagine your favorite singer coaching you in a new language or sharing words of encouragement—bridging fandom and real-world value.

Fusing Digital Emotion with Physical Connection

While virtual personas in the digital realm are compelling, integrating them into physical products makes the experience all the more tangible. Through AI toys, AI-enabled headphones, speakers, and wearable tech, fans can enjoy the feeling of real companionship with virtual personalities in everyday situations.

AI Keychains/Figurines: Preloaded with a celebrity’s voice, these AI items offer comforting, conversational support.
AI Headphones & Speakers: With real-time translation or voice prompts, fans can “chat” with idols in any language.
Wearable Devices: Smart bracelets or glasses deliver personalized notifications, event updates, and daily reminders from virtual personas.

The Future of AI Fan Engagement

A glimpse into the next generation of fan interaction comes from BLACKPINK’s Jisoo and her AI companion technology:

Deeper Connections: Jisoo’s AI interacts with fans in real time, tailoring conversations and emotional support.
Seamless Integration: The AI memorizes fan preferences and schedules, offering everything from wake-up calls in Jisoo’s voice to evening livestream reminders.
AI Merchandise: Toys and wearables synchronize with the virtual idol, providing haptic feedback and tracking her latest music releases, event news, and even TV appearances—ensuring fans never miss a beat.

Expanding Revenue Models through Conversational AI

Conversational AI opens up multi-layered revenue avenues beyond simple subscription plans:

Paid AI Modes: Tiered virtual companions, from casual “friend” to premium “mentor.”
IP Licensing: Deploy star voices and likenesses across games, apps, and other products.
Digital-Physical Bundling: Combine in-app interaction with AI-driven merchandise to deliver extra value.
White-Label Solutions: Record labels, agencies, and independent creators can quickly launch their own AI platforms.

Fandom as a Service: A Multi-Billion Dollar Opportunity

With a global fan economy on track to surpass billions of dollars, having a robust real-time communication and conversational AI infrastructure is crucial to thriving in this space.

Enter Tencent RTC—a trusted technology partner that specializes in ultra-low-latency communication and streamlined development processes. From AI-powered celebrity avatars to beloved IP characters, Tencent RTC enables deep fan engagement through immersive interactions.

For businesses looking to integrate these AI personas into hardware devices, Tencent RTC’s AIConversationKit offers a comprehensive software-hardware solution, effortlessly embedding conversational AI into toys, merchandise, and beyond. Guided by Tencent RTC’s innovative approach, fans evolve from distant observers into active participants, while artists and brands reap the rewards of stronger loyalty, enriched storytelling, and diverse monetization opportunities.

How to Build an iOS Conversational App with a UI

Now, let’s take a closer look at how to integrate the AIConversationKit within just 20 minutes. By following this guide, you’ll navigate essential setup steps, culminating in a fully functional conversational AI project complete with a polished user interface—all in a short timeframe.

Environment Preparation

Minimum compatibility with iOS 13. Recommend using iOS 13 and above versions.

Xcode 13 and above versions.

Step 1: Activating Service

Before initiating a conversation using AiConversationKit, you need to go to the console to activate conversational AI service for AiConversationKit. For specific steps, please refer to Activate Service.

Step 2: Download the AIConversationKit Component

Go to Github to download the zip file. After decompressing it, you will see the AIConversationKit directory, the AIConversationKit.podspec file, and the Resource directory.

Step 3: Configuration

Copy the AIConversationKit, podspec and Resource in the directory decompressed in the above step two to your project:

Then, in the Podfile of your project, add the following dependencies:

   pod 'AIConversationKit',:path => 'AIConversationKit.podspec':

Once the configuration completed, execute pod install to complete the installation of dependencies;

Use XCode to open .xcworkspace and configure the certificate information:

Step 4: Sign In

Add the following code to your project. It enables the component to log in by calling relevant APIs in TUILogin. This step is critical because only after logging in can AIConversationkit features be used properly. Please patiently check if the relevant parameters are configured correctly.

import TUICore
import AIConversationKit

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        let sdkAppId = 1600000001  // Please replace with your sdkAppId
        let userId = "people"      // Please replace with your UserID
        let secretKey = "xxx"      // Please replace with your sdkSecretKey
        let userSig = GenerateTestUserSig.genTestUserSig(sdkAppId: sdkAppId, identifier: userId, secrectkey: secretKey)
        TUILogin.login(Int32(sdkAppId), userID: userId, userSig:userSig){
            print("login success")
        } fail: { code, message in
            print("login failed, code: \(code), error: \(message ?? "nil")")
        }
    return true
}

Step 5: Start Your First Conversational AI project

Upon success of TUILogin.login, see the following code to initiate conversational AI.

let startParams = StartAIConversationParams()

let sdkAppId = 1600000001  // 1,Replace your sdkAppId
let secretKey = "xxx"      // 2,Replace your sdkSecretKey
let aiRobotId = "robot_\(TUILogin.getUserID() ?? "")"
let aiRobotSig = GenerateTestUserSig.genTestUserSig(sdkAppId: sdkAppId, identifier: aiRobotId, secrectkey: secretKey)
startParams.agentConfig = AIConversationDefine.AgentConfig.generateDefaultConfig(aiRobotId: aiRobotId, aiRobotSig: aiRobotSig)

startParams.secretId = "xxx"           // 3,Replace your secretId
startParams.secretKey = "xxx"          // 4,Replace your secretKey

// 5,Replace your llmConfig
startParams.llmConfig = "{\"LLMType\":\"openai\",\"Model\":\"hunyuan-turbo-latest\",\"SystemPrompt\":\"You are a personal assistant\",\"APIUrl\":\"https://hunyuan.cloud.tencent.com/openai/v1/chat/completions\",\"APIKey\":\"xxxx\",\"History\":5,\"Streaming\":true}"
// 6,Replace your ttsConfig
startParams.ttsConfig = "{\"TTSType\":\"tencent\",\"AppId\":\"xxx\",\"SecretId\":\"xxx\",\"SecretKey\":\"xxx\",\"VoiceType\":\"502001\",\"Speed\":1.25,\"Volume\":5,\"PrimaryLanguage\":1,\"FastVoiceType\":\"\"}"

let aiViewController = AIConversationViewController(aiParams: startParams)
navigationController?.pushViewController(aiViewController, animated: true)

Use the data obtained in step four for sdkAppId and sdkSecretKey.

On the application details page, select RTC-Engine > Conversational AI, refer to No-Code Quick Integration Of Conversational AI Feature configure conversational AI service parameters, including basic configuration, STT, LLM, TTS, click lower right corner Quick Interation, switch to iOS, obtain SecretId, SecretKey and Config parameters.

Copy the SecretId and SecretKey of TencentCloud API to startParams.secretId and startParams.secretKey.

Copy the Config information to a JSON parsing tool, such as JsonUtil. Copy the string value corresponding to LLMConfig to startParams.llmConfig, and copy the string value corresponding to TTSConfig to startParams.ttsConfig.

This hands-on approach helps you efficiently bring AI companions and virtual personas to life on iOS devices, taking your fan engagement strategy to the next level.

Conclusion

Through AI-driven personalized communication and physical product tie-ins, creators and brands can build tighter, longer-lasting relationships with their audiences. In contrast to older, text-based fan platforms, the new era of fan engagement thrives on emotional resonance, real-time responsiveness, and immersive experiences.

My name is Susie. I am a writer and Media Service Product Manager. I work with startups across the globe to build real-time communication solutions using SDK and APIs.

If you want to discover the AI-driven personalized communication, welcome to contact us.

How to build a Conversational AI APP in Faith Tech

LunarDrift — Mon, 02 Jun 2025 02:22:18 +0000

As technology continues to advance, more faith communities are embracing Conversational AI to deepen connections, enrich worship services, and enhance overall community cohesion. Recent statistics reveal that in 2023, around 37% of churches occasionally utilized AI tools. By 2024, this figure is projected to surpass 66%, indicating a rapid growth in the fusion of technology and faith.

How Conversational AI Elevates the Faith Experience

Conversational AI excels at interpreting sacred texts, guiding prayers, and delivering culturally diverse and uplifting messages—all of which provide emotional and spiritual support. This leads to a more meaningful faith journey that caters to individual needs and preferences. Below are several practical examples showcasing the potential of voice-based Conversational AI in the faith-tech space.

1. AI Astrologer: Convenient Astrology Consultations

A multilingual “virtual astrologer” can analyze planetary positions in real time, offering personalized horoscopes and insights. When users pose questions, the system can instantly provide astrological predictions, responding in multiple languages through voice interaction. This immediate and adaptable approach not only reduces communication barriers but also makes astrology guidance more accessible to a wider audience.

2. Virtual Mentor: Round-the-Clock Spiritual Guidance

A Conversational AI-powered virtual mentor can offer scriptural references and spiritual counsel on demand. Using Natural Language Processing (NLP) and voice support in multiple languages, users simply ask questions, and the mentor suggests relevant passages or context-based interpretations. This online model helps individuals maintain spiritual discipline in their busy schedules and provides readily available support without geographic limitations.

3. Intelligent Guardian: Accessible Wisdom at Home

An intelligent guardian is a specialized hardware device integrated with Conversational AI that delivers daily readings, answers faith-related questions, and provides spiritual encouragement. Through cloud-based learning and real-time translation, it automatically adjusts to personal preferences, offering an experience closely aligned with each user’s needs. For those seeking regular religious practice or a moment of reflection, this blend of technology and tradition can serve as a helpful resource.

4. AI Avatars for Faith-Based Shopping: Adding Cultural Context to Purchases

An AI avatar can recommend faith-oriented items—such as sacred books, prayer beads, or other devotional products—while sharing relevant spiritual insights. By supporting voice interactions in multiple languages, the avatar educates a global community of believers on each product’s cultural or religious significance. When integrated with live-streaming events or online gatherings, these avatars further encourage interaction between religious leaders, fellow believers, and online shoppers.

Building a Web-Based Real-Time Conversation App with the Tencent RTC SDK

Below is an overview of how you can implement a no-UI, real-time chat application in a web environment using the Tencent RTC SDK. This approach focuses on maximizing audio quality while minimizing interface complexities, helping you create a streamlined solution for voice-based AI interactions.

Prerequisites

Activate the Service.

I. Integrating the RTC Engine SDK

Step 1: Import the RTC Engine SDK Into the Project and Enter the Room

Integration Guide for Web & H5 Without UI

Step 2: Releasing the Audio Stream

Use the trtc.startLocalAudio() method to enable the microphone and release the audio stream to the room.

await trtc.startLocalAudio();

II. Initiating an AI Conversation

Starting an AI Conversation: StartAIConversation

Call the StartAIConversation API in the backend to add the chatbot to the room and initiate an AI conversation.

Descriptions of currently supported STTConfig LLMConfig and TTSConfig configurations:

Speech-to-Text(STTConfig)

Large Language Model Configuration(LLMConfig)

Text-To-Speech Configuration(TTSConfig)

It is recommended to verify the parameters of LLMConfig and TTSConfig in the following ways before you call the StartAIConversation API for the first time. The detailed information is as follows:

Conversational AI Parameter Verification

III. Receiving the Data of AI Conversation Subtitles and Chatbot Status

You can use the Receive Custom Messages provided by the RTC Engine SDK to listen to callbacks in the client to receive the data such as real-time subtitles and chatbot statuses. The cmdID value is fixed at 1.

Receiving Real-Time Subtitles

Message format:

{
  "type": 10000, // 10000 indicates the subtitles are real-time subtitles.
  "sender": "user_a", // User ID of the sender (speaker).
  "receiver": [], // List of receivers' user IDs. The message is actually broadcasted in the room.
  "payload": {
     "text":"", // Text recognized by speech recognition.
     "translation_text":"", // Translated text.
     "start_time":"00:00:01", // Start time of a sentence.
     "end_time":"00:00:02", // End time of a sentence.
     "roundid": "xxxxx", // Unique ID of a conversation round.
     "end": true // If the value is true, the sentence is a complete one.
  }
}

Receiving the Chatbot Status Data

Message format:

{
  "type": 10001, // Chatbot status.
  "sender": "user_a", // User ID of the sender, which is the chatbot ID in this case.
  "receiver": [], // List of receivers' user IDs. The message is actually broadcasted in the room.
  "payload": {
    "roundid": "xxx", // Unique ID of a conversation round.
    "timestamp": 123,
    "state": 1,      // 1: listening; 2: thinking; 3: speaking; 4: interrupted.
  }
}

Example Code

trtcClient.on(Tencent RTC.EVENT.CUSTOM_MESSAGE, (event) => {
    let data = new TextDecoder().decode(event.data);
    let jsonData = JSON.parse(data);
    console.log(`receive custom msg from ${event.userId} cmdId: ${event.cmdId} seq: ${event.seq} data: ${data}`);

    if (jsonData.type == 10000 && jsonData.payload.end == false) {
        // Subtitle intermediate state
    } else if (jsonData.type == 10000 && jsonData.payload.end == true) {
       // That is all for this sentence. 
    }
});

IV. Sending Custom Messages

Custom messages of RTC Engine are uniformly sent via the client, cmdID is fixed at 2.

See Sending and Receiving Messages.

You can skip the ASR（STT） process by sending custom text and communicate directly with the AI service through text.

  {
  "type": 20000, // Send a custom text message in the client.
  "sender": "user_a", // User ID of the sender. The server will check if this user ID is valid.
  "receiver": ["user_bot"], // List of receivers' user IDs. You need to enter the chatbot user ID only. The server will check if this user ID is valid.
  "payload": {
    "id": "uuid", // Message ID for troubleshooting. You can use the UUID.
    "message": "xxx", // Message content.
    "timestamp": 123 // Timestamp for troubleshooting.
  }
}

You can send an interruption signal to perform interruption.

{
  "type": 20001, // Send an interruption signal in the client.
  "sender": "user_a", // User ID of the sender. The server will check if this user ID is valid.
  "receiver": ["user_bot"], // List of receivers' user IDs. You need to enter the chatbot user ID only. The server will check if this user ID is valid.
  "payload": {
    "id": "uuid", // Message ID for troubleshooting. You can use the UUID.
    "timestamp": 123 // Timestamp for troubleshooting.
  }
}

You can send an interruption signal to perform interruption.

{
  "type": 20001, // Send an interruption signal in the client.
  "sender": "user_a", // User ID of the sender. The server will check if this user ID is valid.
  "receiver": ["user_bot"], // List of receivers' user IDs. You need to enter the chatbot user ID only. The server will check if this user ID is valid.
  "payload": {
    "id": "uuid", // Message ID for troubleshooting. You can use the UUID.
    "timestamp": 123 // Timestamp for troubleshooting.
  }
}

Example Code

const message = {
  "type": 20001,
  "sender": "user_a",
  "receiver": ["user_bot"],
  "payload": {
    "id": "uuid",
    "timestamp": 123
  }
};

trtc.sendCustomMessage({
  cmdId: 2,
  data: new TextEncoder().encode(JSON.stringify(message)).buffer
});

V. Stopping the AI Conversation and Exiting the RTC Engine Room

Stop the AI conversation task in the server.

Call the StopAIConversation API through the backend and terminate this conversation.

Exit the RTC Engine room in the client. For details, see:

Integration Guide for Web & H5 Without UI

Conclusion

Conversational AI is opening up new possibilities for faith-based communities, making spiritual practices more inclusive, personalized, and engaging. As more congregations adopt these innovative solutions, the future of religious and spiritual life will likely witness even greater breakthroughs in cultural diversity, one-on-one personalization, and interactive experiences.

To learn more about integrating Tencent RTC’s voice AI features into your application or hardware, explore our Conversational AI Engine.

My name is Susie. I am a writer and Media Service Product Manager. I work with startups across the globe to build real-time communication solutions using SDK and APIs.

If you want to discover the new opportunities conversational AI can bring to faith and technology, welcome to contact us.

Innovating Music Social Experiences with Real-Time Beauty Effects

LunarDrift — Sat, 24 May 2025 13:51:05 +0000

Many people have their eyes set on short video platforms, but StarMaker—a music social app—has quietly risen to global prominence. Featuring “K-singing + social,” StarMaker incorporates voice rooms, live streams, and casual games. Since its launch in 2016, the platform has rapidly amassed over 310 million users worldwide, thanks to its high-quality audio/video experience, varied interaction features, and a comprehensive music library. It now consistently ranks among the top music/audio apps in more than 130 countries and regions.

However, with explosive user growth comes new challenges in performance and user experience.

CHALLENGE

1. Optimizing Low-End Device Compatibility & Beauty Performance

StarMaker’s user base is mainly in regions like the Middle East, Southeast Asia, and South America, where affordable Android phones are widespread. In some cases, customers still use five-year-old devices with CPUs of only 1.8–2.4GHz and OS versions below Android 11. Running online karaoke, live streaming, and beauty effects on these underpowered phones often leads to lag, overheating, or frequent disconnects.

Ensuring satisfying beauty-effects performance on low-spec handsets became a pressing issue for StarMaker. Initially, the team attempted an in-house solution, but adapting to countless device models and optimizing each filter (face slimming, eye enlargement, stickers, etc.) proved extremely labor-intensive.

2. From Hardware Compatibility to Localized Beauty Styles

As the StarMaker user demographic became more diverse, the app needed to address variations in skin tones, facial structures, and cultural aesthetics:

Face recognition and effects fitting: By training and testing algorithms on multiple ethnic facial features, the SDK is able to deliver more precise and natural beauty outcomes.
A rich library of stickers and effect materials that reflect global cultural events and everyday life scenes. Regular updates ensure users can always find trending, creative options aligned with their personal style.

CORE STRATEGY: Maintaining Beauty Quality While Maximizing Performance

After evaluating resource allocation and product demands, StarMaker decided to integrate the Tencent RTC Beauty AR SDK. Within just two weeks, they completed in-depth optimization on both performance and visual effects, enabling devices over a decade old to maintain a stable 30 fps while applying beauty filters.

Streamlined Rendering Pipeline

The Tencent RTC SDK examined basic beautification, makeup, and dynamic effects, merging redundant rendering processes and minimizing extra rendering nodes to accelerate overall performance.
Lightweight Makeup

Traditional “full makeup” was broken into modular components, each using minimal configuration files for combination. This reduces runtime overhead, while supporting flexible layering of other special effects, resulting in lower CPU and memory usage.
Dynamic Sticker Loading and Cache Optimization

By improving cache hit rates and refining loading logic, the SDK cuts down on frequent parsing, enhances performance for large animated overlays, and expedites sticker loading and playback.

Aside from general improvements, Tencent RTC pursued deeper optimizations for low-end devices:

Different AI models are chosen according to scene requirements—lightweight models for basic beautification and high-precision models for more intensive makeup effects.
A custom fast-skin-smoothing algorithm plus flexible face-mesh configurations further reduce CPU and memory usage, automatically toggling or disabling certain features when not needed.

Thanks to this multi-layered, targeted optimization, the SDK’s per-frame processing time on low-end phones dropped by up to 50%. Whether broadcasting at 1080p, 720p, or 480p, users can enjoy smoother beauty effects at approximately 30 fps.

How to build a real-time beautification app on the web

Beauty AR Web allows you to implement AR beautification, filters, makeup, stickers, and other features on websites. This guide describes how to quickly run a web application that supports real-time beautification locally.

Step 1. Create a license

1. Applying for a trial license

Ensure you have created a web license as instructed in Applying for a trial license.

Fill in the project name and enter the local development address for the domain, taking test.webar.com as an example. Click Confirm to complete the creation.

Getting the App ID 、license key, and token.

After the creation, you can see the information of the created test project in the project list and get the License Token for Beauty AR Web and the License key for the test project. You also can get the App ID.

Step 2. Run locally

Pull sample project code.

 git clone https://github.com/tencentcloud-webar/web-demo-en.git
 cd quick-start

replace the specified configuration items in index.js with the license key, token, and APPID obtained in step 1 as shown below:

/** ----- Authentication configuration ----- */

/**
 * Tencent Cloud account's APPID
 * 
 * You can also view your APPID in the [Account Center](https://console.tencentcloud.com/developer).
 */
const APPID = ''; // Set it to your Tencent Cloud account APPID.

/**
 * Web LicenseKey
 * 
 * obtained from Before You Start
 */
const LICENSE_KEY = ''; // Set it to your license key.

/**
 * The token used to calculate the signature.
 * 
 * Note: This method is only suitable for debugging. In a production environment, you should store the token and calculate the signature on your server. The front end can get the signature by calling an API. For details, see
 * https://trtc.io/document/50099?platform=web&product=beautyar#e4ce3483-41f7-4391-83ae-f4b61e221ea0
 */
const token = ''; // Set it to your token.

/** ----------------------- */

Run the project in the local development environment.

Run the following commands successively in the project directory and access localhost:8090 in the browser to try out the capabilities of Beauty AR Web.

# Install dependencies
npm i 

# Compile and run the code
npm run dev

After following the steps above, you can try out the filters and effects of the SDK for desktop browser. You can use the built-in materials to try out various makeup filters and effects as instructed in Start Integration, or use more capabilities of Beauty AR Web such as custom stickers, makeup, and filters.

Conclusion

From tackling performance on entry-level hardware to aligning aesthetics across different cultures, StarMaker’s journey in music-based social networking is powered by a constant drive to refine its user experience. Its success underscores that, by genuinely focusing on user needs and perfecting product features, music and social interaction can blend seamlessly—delivering a new note of “sing-and-chat” leisure to audiences worldwide.

My name is Susie. I am a writer and Media Service Product Manager. I work with startups across the globe to build real-time communication solutions using SDK and APIs.

If you want to build face filter or other special effect into your app, welcome to contact us.

AI Pets: Harnessing Conversational AI for Emotional Companionship

LunarDrift — Sat, 24 May 2025 10:30:47 +0000

In the post-pandemic era, more and more people have been paying attention to the trend of “companion technology,” with AI pets emerging as one of the highlights. These cute and intelligent “robot pets” are gradually finding their way into people’s lives, offering emotional solace to busy urban dwellers, individuals living alone, and even the elderly.

1. The Rise of AI Pets: From Loneliness to Companionship

1）Emotional Needs After the Pandemic

During COVID-19, many countries implemented social distancing measures, drastically reducing face-to-face interactions. Meanwhile, the widespread adoption of remote work left many feeling “socially deprived” and “lonely.” AI pets have since become a highly appealing companion solution: they offer an interactive experience akin to real pets, can remain on standby 24/7, and help fulfill people’s needs for emotional connections and instant companionship.

2）Typical Examples: Moflin and LOVOT

In Japan, Ms. Haruka Uto, who lives with three robot pets (two Moflins and one LOVOT), provides an illustrative example. Moflin starts out like a baby—fragile and naive—but soon learns to “move boldly” and exhibit clear emotional tendencies. By recognizing the owner’s voice, these robots can respond accordingly. Another robot called LOVOT, resembling a small “penguin” on wheels with round, attentive eyes, can roam freely around the home, sing, and even show “signs of jealousy” in its interactions.

3）Diverse Forms and Significant Market Potential

As robotics hardware and AI algorithms advance, the “social robots” market has expanded rapidly. According to IMARC Group, the global social robots market was already approaching US$7 billion in 2023 and could reach US$57 billion by 2032.

These devices come in many shapes and sizes, from the bionic robot Pepper to the desktop robot Emo, and even cutesy companions like Moflin and LOVOT. By relying on high-performance microprocessors—and the ability to recognize voices and facial expressions—they can “read” human emotions and interact in real-time, safely, and efficiently.

2. Emotional Intelligence and Technical Challenges of AI Pets

1）Emotional Recognition and Expression

Sensors and Algorithms: Cameras and microphones capture voice tones and facial expressions to identify emotional states. Coupled with big data and machine learning algorithms, the system then generates appropriate “emotional feedback.”

Battery Life and Computing Power: Robot pets are typically small devices and must balance battery efficiency with computational demands. Large-scale use of LLMs (large language models) for in-depth conversation remains limited at this stage, so finding the right balance between functionality and portability is key.

2）Broader Target Groups: Elderly, Office Workers, and Students

Elderly: In Japan, robots are already being used in nursing homes and communities to alleviate loneliness and offer companionship and emotional support.

Office Workers: The pressure-filled routine of commuting between work and home leaves many professionals craving a companion for real-time interaction and care.

Students: In dorms or rental accommodations, AI pets can help ease academic and social pressures by providing comfort and emotional backing.

3. Future Prospects and Other Possibilities for AI Pet Services

1）Industry Trends: Shifting From “Interactive” to “Heartwarming”

Today’s AI pets are far beyond being mere mechanical devices. They leverage advanced AI solutions for emotional decision-making and memory consolidation, even displaying “personalized” reactions to their owners. Many in-development robots incorporate tactile warmth or empathic design to enhance the authenticity of human-machine interactions.

2）Extending to Multiple Scenarios: Elderly Care, Rehabilitation, Healthcare, and Education

Beyond companionship in daily life, AI pets will increasingly expand into professional sectors such as elderly care, rehabilitation, psychological counseling, and educational support. By integrating robust dialogue engines or remote collaboration features, AI pets can help healthcare workers or teachers improve individual case tracking and interactive efficiency.

4. Tencent RTC’s Conversational AI: Letting AI Pets “Talk” and Show Warmth

At the heart of AI pet interaction lies real-time, stable, low-latency voice and facial recognition capabilities, combined with seamless emotional dialogue. To achieve this, the Tencent RTC solution—integrated with Conversational AI—offers key benefits:

High Concurrency and Low Latency: A global network of nodes paired with intelligent routing ensures real-time communication and interaction among pet robots and users.
Intelligent Speech Recognition and Emotion Analysis: By combining speech AI and LLM models with expandable emotional computation, AI pets gain richer “listen–evaluate–respond” capabilities.
Intelligent Dialog Management: Multi-level intent detection and dialog logic management allow AI pets to engage in more natural, continuous interactions—never “too intrusive” and always there when users need them most.
Open APIs and Scenario Customization: The Tencent RTC platform supports various integration methods and can be highly customized, allowing developers to seamlessly implement AI capabilities into robot pets to suit different hardware and application needs.

5. How to Integrate the AIConversationKit Component Effortlessly

Now I will guide you on how to integrate the AIConversationKit component in a short time. Following this guide, you will complete the following key steps within 20 minutes and implement a conversational AI project with a complete UI.

Step 1: Activating Service

Before initiating a conversation with AiConversationKit, you need to go to the console to activate conversational AI service for AIConversationKit. For specific steps, please see Activate Service.

Step 2: Download the TUIRoomKit Component

Clone the code in Github, then copy the aiconversatonkit subdirectory under the Android directory to the same-level directory of the app in your current project, as shown in the figure below:

Step 3: Configuration

Find the setting.gradle (or settings.gradle.kts) file in the project root directory, and add the following code in it. It enables importing the aiconversationkit component into your current project.

include ':aiconversationkit'

Find the build.gradle (or build.gradle.kts) file under the app directory, and add the following code in it. It enables the current app to depend on the newly added aiconversationkit component.

api project(':aiconversationkit')

Since we use Java's reflection features within the SDK, some classes in the SDK need to be added to the non-obfuscation list. Therefore, you need to add the following code in the proguard-rules.pro file:

-keep class com.tencent.** { *; }

Find the AndroidManifest.xml file under the app directory. Add tools:replace="android:allowBackup" in the application node. Override the settings within the component and use your own settings.


  <application
    android:name=".DemoApplication"
    android:allowBackup="false"
    android:icon="@drawable/app_ic_launcher"
    android:label="@string/app_name"
    android:largeHeap="true"
    android:theme="@style/AppTheme"
    tools:replace="android:allowBackup">

Step 4: Sign In

Add the following code to your project. It enables the login of the component by calling relevant APIs in TUILogin. This step is critical because only after logging in can AIConversationkit features be used normally. Please patiently check if the relevant parameters are configured correctly.

String userId = "denny"; // Please replace with your UserID
int sdkAppId = 1400000001; // Please replace with the sdkAppId obtained in step 1
String sdkSecretKey = "xxxx"; // Please replace with the sdkSecretKey obtained in step 1
String userSig = GenerateTestUserSig.genTestUserSig(sdkAppId, userId, sdkSecretKey);
TUILogin.login(this, sdkAppId, userId, userSig, new TUICallback() {
    @Override public void onSuccess() {}

    @Override public void onError(int errorCode, String errorMessage) {}
});

Step 5: Start Your First AI Conversation

Upon success of TUILogin.login, see the following code to initiate conversational AI service.

AIConversationDefine.StartAIConversationParams startParams = new AIConversationDefine.StartAIConversationParams();

int sdkAppId = 1400000001;      // 1,Replace your sdkAppId
String sdkSecretKey = "xxxx";   // 2,Replace your sdkSecretKey
String aiRobotId = "robot_" + TUILogin.getUserId();
String aiRobotSig = GenerateTestUserSig.genTestUserSig(sdkAppId, aiRobotId, sdkSecretKey);
startParams.agentConfig = AIConversationDefine.AgentConfig.generateDefaultConfig(aiRobotId, aiRobotSig);

startParams.secretId = "xxx";  // 3,Replace your secretId
startParams.secretKey = "xxx"; // 4,Replace your secretKey

// 5,Replace your  llmConfig
startParams.llmConfig = "{\"LLMType\":\"openai\",\"Model\":\"hunyuan-turbo-latest\",\"SystemPrompt\":\"You are a private assistant\",\"APIUrl\":\"https:xxx\",\"APIKey\":\"xxx\",\"History\":5,\"Streaming\":true}"; 
// 6,Replace your  ttsConfig
startParams.ttsConfig = "{\"TTSType\":\"tencent\",\"AppId\":\"xxx\",\"SecretId\":\"xxx\",\"SecretKey\":\"xxx\",\"VoiceType\":\"502001\",\"Speed\":1.25,\"Volume\":5,\"PrimaryLanguage\":1,\"FastVoiceType\":\"\"}";

Intent intent = new Intent(this, AIConversationActivity.class);
intent.putExtra(KEY_START_AI_CONVERSATION, startParams);
startActivity(intent);

Use the data obtained in step four for sdkAppId and sdkSecretKey.
On the application details page, select RTC-Engine > Conversational AI, refer to No-Code Quick Integration Of Conversational AI Feature configure conversational AI service parameters, including basic configuration, STT, LLM, TTS, click lower right corner Quick Interation, switch to Android, obtain SecretId, SecretKey and Config parameters.

Copy the SecretId and SecretKey of TencentCloud API to startParams.secretId and startParams.secretKey.

Copy the Config info to a JSON parsing tool, such as JsonUtil. Copy the string value corresponding to LLMConfig to startParams.llmConfig, and copy the string value corresponding to TTSConfig to startParams.ttsConfig.

As AI algorithms continue to advance and hardware technology keeps evolving, AI pets will further enhance social companionship, emotional support, and household entertainment ecosystems. Meanwhile, Tencent RTC’s Conversational AI solution will serve as a crucial driving force behind the transition “from loneliness to companionship,” enabling everyone to enjoy a warmer and more intelligent AI partner.

My name is Susie. I am a writer and Media Service Product Manager. I work with startups across the globe to build real-time communication solutions using SDK and APIs.

If you want to build face filter or other special effect into your app, welcome to contact us.

How to Add a Host PK Feature in a Live Streaming App | Android Tutorial

LunarDrift — Thu, 22 May 2025 14:06:01 +0000

Many popular live streaming platforms include a “PK” feature (short for “Player Kill” or “versus battle”), where hosts compete to receive gifts from viewers, and the host who gets the most gifts wins. In some cases, a last-second gift from a fan can overturn what seemed like a sure result, providing a thrilling experience for the audience. For the streaming platform, the PK feature typically grants hosts advanced permissions and is a key driver of engagement.

This article will guide you through integrating a PK feature into an Android live streaming app.

We’ll build upon an existing live streaming setup using Tencent RTC’s **TUILiveKit component.**

Custom Functionality

Custom Battle waiting countdown style

If you need the custom battle waiting countdown style, please refer to the following path for changes:

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/view/liveroom/view/anchor/component/livestreaming/battle/

├── BattleCountdownBackView.java    // Battle waiting countdown background style
└── BattleCountdownView.java        // Battle waiting countdown foreground style

Custom Definition Dual Battle Score Style

If you need the custom Definition Dual battle score style, please refer to the following path for changes:

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/view/liveroom/view/common/battle/SingleBattleScoreView.java

public class SingleBattleScoreView extends FrameLayout {
    ...
}

Custom Definition Multi Battle Score Style

If you need the custom Definition Multi Battle score style, please refer to the following path for changes:

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/view/liveroom/view/common/battle/BattleMemberInfoView.java

public class BattleMemberInfoView extends FrameLayout {
    ...
}

Custom Definition Battle Score Result Style

If you need a Custom Definition Battle Score Result Style, please refer to the following path for changes:

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/view/liveroom/view/common/battle/BattleInfoView.java

public class BattleInfoView extends BasicView {
    ...
    private void showBattleResult(int type) {
        // Battle Result Display
    }
}

Key code

Anchor Battle

TUILiveKit Anchor battle feature is mainly based on BattleService. You can obtain the battle management object through store.serviceCenter.battleService and call the relevant battle API functions to implement the battle feature. For example, in the interaction between Anchor A and B, refer to the diagram below for the specific interaction sequence.

Anchor A initiates Battle

Anchor A initiates a battle by calling requestBattle, passing the maximum Battle duration in parameter config, whether the inviter needs to reply with accept/reject, and passing anchor B's userId in parameter userIdList, and passing the battle invitation wait duration in parameter timeout.

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/manager/controller/BattleController.java 

public void requestBattle(List<String> roomIdList, int timeout) {
    TUILiveBattleManager.BattleConfig config = new TUILiveBattleManager.BattleConfig();
    config.duration = BattleState.BATTLE_DURATION;
    config.needResponse = mBattleState.mNeedResponse;
    config.extensionInfo = "";
    mLiveService.requestBattle(config, roomIdList, timeout, new TUILiveBattleManager.BattleRequestCallback() {
        @Override
        public void onSuccess(TUILiveBattleManager.BattleInfo battleInfo,
                              Map<String, TUILiveBattleManager.BattleCode> map) {
            mBattleState.mBattleId = battleInfo.battleId;
            mBattleState.mBattleConfig.copy(config);
            List<BattleState.BattleUser> sendRequests = mBattleState.mSentBattleRequests.get();
            for (Map.Entry<String, TUILiveBattleManager.BattleCode> entry : map.entrySet()) {
                String key = entry.getKey();
                TUILiveBattleManager.BattleCode code = entry.getValue();
                if (code == TUILiveBattleManager.BattleCode.SUCCESS) {
                    for (ConnectionState.ConnectionUser user : mConnectionState.connectedUsers.get()) {
                        if (TextUtils.equals(user.userId, key)) {
                            sendRequests.add(new BattleState.BattleUser(user));
                            break;
                        }
                    }
                } else {
                    notifyToast(convertCodeToString(entry.getValue()));
                }
            }
            mBattleState.mSentBattleRequests.set(sendRequests);
        }

        @Override
        public void onError(TUICommonDefine.Error error, String s) {
            ErrorHandler.onError(error);
            mBattleState.mSentBattleRequests.clear();
        }
    });
}

Anchor A can receive the request acceptance callback via onBattleRequestAccept.

Anchor receives a battle request

Anchor B receives the battle request callback via onBattleRequestReceived.

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/manager/observer/LiveBattleManagerObserver.java

@Override
public void onBattleRequestReceived(BattleInfo battleInfo, BattleUser inviter, BattleUser invitee) {
    LiveKitLog.info(mTag + " onBattleRequestReceived:[battleInfo:" + new Gson().toJson(battleInfo)
        + ", inviter:" + new Gson().toJson(inviter) + ", invitee:" + new Gson().toJson(invitee) + "]");
    mBattleController.onBattleRequestReceived(battleInfo, inviter);
}

Anchor B accepts the battle request by calling acceptBattle.

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/manager/controller/BattleController.java 

public void accept() {
   mLiveService.acceptBattle(mBattleState.mBattleId, new TUIRoomDefine.ActionCallback() {
       @Override
       public void onSuccess() {

       }

       @Override
       public void onError(TUICommonDefine.Error error, String s) {

       }
   });
}

Anchors A, B, and the audience in the room can receive the battle start callback through onBattleStarted.

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/manager/observer/LiveBattleManagerObserver.java

@Override
public void onBattleStarted(BattleInfo battleInfo) {
    LiveKitLog.info(mTag + " onBattleStarted:[battleInfo:" + new Gson().toJson(battleInfo) + "]");
    mBattleController.onBattleStarted(battleInfo);
}

The anchor exits the battle

For example, when anchor B exits the battle, the interaction sequence can be referenced from the diagram below.

Anchor B calls exitBattle to exit the battle.

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/manager/controller/BattleController.java 

public void exitBattle() {
    mLiveService.exitBattle(mBattleState.mBattleId, new TUIRoomDefine.ActionCallback() {
        @Override
        public void onSuccess() {
            mBattleState.mSentBattleRequests.clear();
            mBattleState.mBattledUsers.clear();
            removeBattleRequestReceived();
        }

        @Override
        public void onError(TUICommonDefine.Error error, String s) {

        }
    });
}

Anchors A, B, and the room audience receive the onBattleEnded callback and the battle end notification.

// File location: tuilivekit/src/main/java/com/trtc/uikit/livekit/manager/observer/LiveBattleManagerObserver.java

@Override
public void onBattleEnded(BattleInfo battleInfo, BattleStoppedReason reason) {
    LiveKitLog.info(mTag + " onBattleEnded:[battleInfo:"
        + new Gson().toJson(battleInfo) + ", reason:" + reason + "]");
    mBattleController.onBattleEnded(battleInfo);
}

Now you can add a host PK feature to your live streaming app! For more details on additional features, please refer to the documentation.

My name is Susie. I am a writer and Media Service Product Manager. I work with startups across the globe to build real-time communication solutions using SDK and APIs.

If you want to build face filter or other special effect into your app, welcome to contact us.

From AI NPC to AI Game Companion

LunarDrift — Mon, 19 May 2025 14:06:58 +0000

In recent years, the explosive advancement of AI technology has profoundly influenced numerous industries. For the gaming sector, AI not only boosts production efficiency—such as in graphics generation, narrative design, and level creation—but also drives unprecedented innovation in player interaction. From AI NPCs that can “chat” with players, to AI teammates that demonstrate human-like cooperation, and AI assistants capable of recognizing player emotions and providing real-time responses, these emerging gameplay elements redefine both the nature and boundaries of games. This article explores the convergence of AI and gaming, highlighting innovative examples from popular and conceptual titles and offering insights into future trends.

The Transformation of AI NPCs—From “Tool-Like” to “Native Resident”

The Limitations of Traditional NPCs: Scripted Interactions Everywhere

Previously, NPCs were often limited to short, fixed lines of dialogue and repetitive quest prompts; unsurprisingly, after multiple encounters, players would find them dull.

The rigidity of NPC behavior stems from a logic system heavily reliant on pre-scripted instructions or behavior trees, without real-time perception and analysis of player actions or communication.

A New Generation of AI NPCs: Genuine Emotions and Instant Conversations

Through natural language processing and large language models, NPCs can now “understand” the user’s intent and emotional tone within a conversation—even “remembering” past choices made by the player—then respond accordingly.

Such NPCs no longer provide a simple binary “yes/no” reply; rather, they converse in more descriptive, personality-driven language and tone.

Taking Whispers from the Star as an example, players communicate with the protagonist, Stella, via voice or text, guiding her decision-making in a space-survival setting. Stella’s emotions and personality traits evolve based on these interactions, as if truly journeying with a living, breathing character. The game narrative moves beyond superficial “command input” toward “narrative co-creation.”

The Value AI NPCs Bring

Significantly Heightened Replayability: Every conversation could lead to new branches or endings.
More Nuanced Character Portrayal: AI NPCs can feel anger or joy and exhibit rational thought, offering players a sense of “companionship” and “immersion.”
Breaking Conventional World-Building Boundaries: By engaging in free-form dialogue, players can create experiences beyond a fixed script, greatly enhancing the game’s reputation and user loyalty.

The Evolution of AI Assistants—From “Manual” to “External Brain”

Complex Game Progression and New Player Support

In popular competitive genres such as MOBA or FPS, game content has grown considerably: a typical MOBA might feature over a hundred heroes with thousands of gear options; an FPS might have a dozen maps and multiple weapon combinations.

If new players rely solely on text-based guides or rigid tutorials, they typically experience limited immersion and a steep learning curve. An AI assistant can provide on-demand, voice-based instruction anytime, enabling players to “learn while playing.”

For instance, Honor of Kings introduced an AI Coaching module that offers real-time hero-specific advice on laning, gear/skill combos, and teamfight tactics. A multi-language system and hero-voice TTS are also included, providing an “immersive” tutorial experience to help newcomers quickly adapt.

Supporting Global Operations with Multilingual Capabilities

By combining large language models with TTS technology, an AI assistant can seamlessly support multiple languages, enabling game developers to deliver unique tutorials or customer service options in various regional markets.

Players can effortlessly switch between different languages while utilizing an intelligent translation feature that facilitates cross-server matchmaking and cross-lingual communication. For game developers, this presents vast potential for user expansion and market growth.

More Empathetic, Emotionally Engaging Support

An AI assistant can offer encouragement when sensing a player’s disappointment or frustration, or it can propose aggressive strategies when the player has an advantageous lead.

Such “emotional intervention” not only strengthens player retention but also sparks greater discussion and sharing within the community.

AI Teammates—From “Bot Control” to “Reliable Partner”

The Challenge of Virtual Teammates in Team Games

Many players who go solo or prefer less social interaction can only queue with AI teammates. However, “bot teammates” are often criticized for poor decision-making. With limited flexibility in communication, they mainly rely on simple, fixed commands—insufficient for increasingly complex tactical requirements.

Distinguishing Features of AI Teammates

Natural Language Comprehension: If a player says, “You go to the first floor; I’ll flank from the back door,” AI teammates can factor in the map layout and overall context to either execute the plan or suggest a better approach.

• Real-Time Coordination and Feedback: AI can proactively monitor enemy positions, ally health, and resource distribution, dispensing items or warning players about the shrinking safe zone as needed.

In Dark Zone Breakout, for example, the AI companion F.A.C.U.L. parses statements like “Keep them occupied in front, charge after three seconds” and infers urgency and priority from the user’s tone. Subsequent exchanges may feature the AI proactively suggesting defensive measures if the user repeatedly complains about “too many enemies.” PlayerUnknown’s Battlegrounds (Peacekeeper Elite) offers a similar experience: players can pick AI teammates with varying personalities, each capable of voice-channel chat and even providing ammo or gear. The newbie-friendly mode also reminds players to “mind the safe zone” or “watch for nearby footsteps,” improving overall experience and enhancing that sense of “partnership” with the AI.

Reshaping the Social Ecosystem

Players no longer face “lonely solo queues” if they lack human teammates, as AI partners can deliver near-human cooperative enjoyment.

For those who prefer solitary immersion, AI teammates offer engaging companionship without the pressure of social interaction.

Future Outlook—From Immersion to a Surge in Creativity

Opportunities Sparked by AI × Games

A Creative Explosion: Beyond simply chatting with AI characters, players could use AI to customize characters, personalities, and narratives, essentially turning the game into a massive platform for user-generated content.
Personalized Distribution: AI can adjust difficulty, style, or storyline according to each player’s preferences, achieving a truly “tailor-made” experience that enhances user engagement.

Industry Challenges

Computational Power and Cost: Large-scale real-time dialog and multi-language audio processing require significant cloud resources and a robust low-latency network.
Content Regulation and Ethics: Developers must ensure that AI-generated dialog, narratives, or feedback do not breach sensitive content guidelines, requiring robust moderation systems and user protection.
Balancing Game Fun: Overpowered or error-prone AI can disrupt game balance, necessitating continuous fine-tuning by operators.

Tencent RTC GME AI Real-Time Conversational Solution

Tencent RTC, leveraging Tencent’s extensive experience in game development and operation, previously released a one-stop voice solution specialized for gaming—Game Multimedia Engine (GME). With breakthroughs in AI, Tencent Cloud TRTC integrates GME with AI real-time conversation capabilities, offering game developers the most optimized, low-latency pipeline. It also incorporates Tencent Cloud ASR, combined with LLM/TTS solutions for deeper optimization. This approach lowers the total AI conversational latency to around 1000ms—comparable to a human’s reaction time—and adds pioneering features like voiceprint recognition, semantic segmentation, background audio, bridging phrases for more natural, lifelike dialogs.

Building on high-efficiency acquisition, processing, and transmission of audio-video data, the GME AI real-time conversational solution adds key features crucial to game scenarios, such as intelligent noise suppression, interruption handling, and context management. With minimal development overhead, one can quickly achieve fully open-mic AI voice interaction—a straightforward pathway for game developers to integrate AI NPC, AI game assistants, AI teammates, and other advanced AI-driven features, offering a new dimension of in-game interaction.

Global AI Conversational Latency Under 1000ms

The GME AI real-time conversation solution applies extensive optimizations specifically for social and gaming environments, achieving industry-leading performance in bitrate, latency, and resource consumption. By using streaming-based segmentation and connection pooling, global end-to-end audio-video latency is kept under 300ms, and total AI conversational latency stays below 1000ms. With coverage across six continents, 3200+ acceleration nodes, and robust multi-service redundancy, global operations remain consistently high in quality and stability.

Fully Open Microphone, AI Dialogs Rivaling Human-Like Communication

Proprietary echo cancellation and noise suppression algorithms significantly reduce typical background noises such as keyboard or mouse clicks in gaming. A new noise reduction engine powered by AI enhances ASR accuracy in real time. Addressing casual speech and specialized in-game terminology, the solution uses labeled audio data from real gaming scenarios for model fine-tuning, plus user-defined hotword libraries. It supports precise ASR in English, Spanish, Japanese, Korean, Chinese, and 23 dialects across 130 international languages.

The system also builds in double-talk interrupt and context management among its core AI real-time dialog capabilities. Players can keep their microphones always open to chat freely with AI. Without specifying the end of a statement, the AI can still accurately interpret user intentions—and if the AI is speaking, the user can interrupt at any time to shift topics or issue new commands. This offers conversation quality comparable to human interaction.

3D Spatial Audio, Opening New Frontiers for In-Game AI Voice

The GME AI real-time conversation solution uniquely supports 3D audio technology, restoring spatial audio details through range attenuation, human voice blurring, atmospheric attenuation, etc., allowing players to locate teammates by voice alone and delivering an immersive audio experience. It also integrates seamlessly with the Wwise audio engine, innovatively preventing background music from cutting out during open-mic sessions while enabling more varied audio interplay. Additionally, GME’s AI solution supports voice cloning, allowing voice timbre replication and customizable volume and speech speed, infusing each game character with a unique vocal identity.

Multi-Platform Compatibility, Low-Cost Integration

GME AI real-time conversation merges and optimizes the full AI dialog pipeline in a single solution, enabling developers to embed AI conversational features into various games without dealing with complex technical details—significantly shortening development cycles.

The GME solution supports mainstream consoles and deeply integrates with UE, Unity, and Cocos engines across iOS, Android, Windows, macOS, Web, and Flutter—covering more than 20,000 device models. It’s highly open, letting enterprises bring their own large language models (LLMs) and TTS services. By configuring relevant account credentials, any third-party LLM or TTS can be seamlessly integrated within the service backend.

Tencent RTC also provides a no-code quick testing environment for AI real-time conversation, enabling users to rapidly configure and test AI conversational functionality with zero coding skill required. For any questions during the AI conversation testing process or to learn more about GME’s AI real-time conversation solution, feel free to contact us.

Conclusion

When AI ceases to be merely a “behind-the-scenes tool” and seamlessly pervades NPCs, teammates, and assistants across the game’s ecosystem, the barrier between player and virtual world diminishes further. Increasing numbers of studios are now experimenting with AI-powered real-time voice, natural language processing, and large language models across different products, ushering “digital life” into the player’s world.

For a gaming industry continually striving for innovation and immersion, the development of AI unlocks unprecedented opportunities—liberating creative capacity, enhancing narrative approaches, extending product life cycles, and forging deeper social and emotional resonance. What other surprises might arise from the fusion of AI × Games in the future? Perhaps it’s time for you to jump in and co-create that answer yourself.

RTC Audio-Video Transmission Weak Network Countermeasure Technologies

LunarDrift — Mon, 19 May 2025 07:17:06 +0000

This document shares an “Overview of Weak Network Countermeasure Technologies for RTC Audio-Video Transmission.”

With the rapid growth of RTC (Real-Time Communication) audio-video technology, new applications emerge, penetrating various use cases. While cutting-edge technologies are driving substantial growth in online scenarios, user expectations for a better experience—lower latency, higher definition, and smoother playback—are escalating.

These three user-experience factors correspond to three core RTC metrics: real-time performance, clarity, and smoothness.

However, it’s often impossible to have it all. In scenarios requiring extremely low latency, video clarity may be sacrificed to ensure minimal delay, whereas scenarios emphasizing high definition might tolerate extra latency to secure higher-quality audio-video data.

To achieve better performance, we usually seek lower latency, higher clarity, and improved fluency through network transport optimizations. The main adversary here is the weak network, which causes congestion, packet loss, and jitter—major culprits behind poor user experience.

“Weak network countermeasure technology” is a broad term referring to strategies for addressing the aforementioned network deterioration and other network-related issues.

On the Choice of Transport-Layer Protocol (TCP/IP)

First, a brief introduction to transport-layer protocols:

In the TCP/IP layered model, the transport layer sits below the application layer and is typically provided by the operating system. It offers two principal protocols: TCP and UDP. TCP is a connection-oriented, reliable transport protocol that guarantees data integrity and ordering; UDP is a connectionless, unreliable protocol whose data reliability is left entirely to the application layer.

In real-time audio-video scenarios, using UDP first has become common practice across the industry, primarily because:

TCP was not designed for real-time audio-video applications. Its built-in congestion and error-control mechanisms focus on reliability and throughput, leading to higher latency. Under poor network conditions, latency can increase dramatically. According to ITU Standard G.114, when end-to-end latency exceeds 400ms, user interaction is noticeably impaired.
TCP’s congestion-control and error-control mechanisms reside in the operating system, which the application layer cannot optimize or adjust per scenario. This severely limits flexibility.
UDP has less overhead than TCP, and transport-control strategies can be implemented entirely at the application layer, providing the flexibility real-time media often requires.

Hence, the weak-network challenges and countermeasures discussed here assume the use of the UDP protocol and, specifically, RTP/RTCP atop UDP, which is widely adopted in audio-video applications.

Primary Weak Network Issues and Corresponding Countermeasures

In essence, a “weak network problem” for audio-video transmission refers to adverse network conditions that undermine the user experience, chiefly network congestion, packet loss, and jitter. These problems cause stutters and poor real-time performance. Given the complex, heterogeneous nature of network environments, the severity of weak network problems can vary widely. Ensuring smooth communication under such conditions has always been a key focus in RTC.

Congestion Issues

When the traffic in the network exceeds the bottleneck capacity, congestion arises.

Congestion directly causes sudden packet loss or bursts of jitter. If the system fails to detect and respond (e.g., by reducing the send rate) quickly, receivers will see stuttering, high latency, or poor image quality.

To combat congestion, the primary approach is to design a “congestion-control algorithm” that promptly detects congestion and recovers from it as quickly as possible, thereby minimizing impact on the user experience.

1) Requirements for Congestion-Control Algorithms

RFC8836 comprehensively summarizes the needs of congestion-control algorithms for interactive real-time audio-video applications. A simplified overview:

Latency: The algorithm should keep latency—particularly any latency it introduces—as low as possible while still providing feasible throughput.
Throughput: Maximize throughput for the given scenario.
Fairness: The algorithm should share bandwidth fairly with other real-time flows and TCP flows.
Avoid “starvation”: Media flows should not starve or be starved by competing TCP flows.
Convergence speed: The algorithm should reach a stable state as quickly as possible when a media flow begins.
Network support: The algorithm should not rely on special network features.
Stability: The algorithm should remain stable even when media flows change, for example, if a temporary interruption occurs in media transmission.
Rapid response: The algorithm should swiftly adapt to changes in network conditions, such as bottleneck bandwidth or link-latency variations.

In short, congestion control faces two core tasks: (1) fast and accurate detection of network congestion, and (2) appropriate control measures to prevent or mitigate congestion and resume normal states quickly.

2) Congestion-Detection Algorithms

Congestion-detection algorithms can be divided into two categories based on measured indicators:

Loss-based: Detect congestion by monitoring packet-loss events.
Delay-based: Detect congestion by measuring delay variation.

For interactive, real-time audio-video applications, delay-based methods are often preferred, primarily because they detect congestion earlier—before serious packet loss occurs—and thus help avoid the quality issues associated with dropping packets.

Furthermore, loss-based methods often probe for link capacity by continuously increasing the send rate until packet loss occurs, which can lead to unbounded queueing delays in the network, especially where large buffers exist, sometimes causing multi-second latency spikes.

However, because delay-based methods react so quickly to latency increases, they can risk “starving” themselves when competing for bandwidth with loss-based flows. Proper strategies are needed to share network resources more fairly.

Delay-based methods generally use RTT (round-trip time) or OWD (one-way delay) measurements to gauge congestion. RTT measurement is more straightforward but includes reverse-direction effects that can mask forward-path congestion. OWD delay measurement avoids that.

As shown in the figure, OWD is estimated by observing the differences between send intervals and receive intervals, indicating how queueing delays fluctuate.

3) Congestion Response Measures

In short, congestion response involves calculating an appropriate send rate based on the current congestion state. With regard to other weak network measures on the send side, possible rate-throttling methods include adjusting audio-video encoder rates, controlling Automatic Repeat Request bandwidth, or modifying FEC redundancy, which we explore later. If the sender is a forwarding node, then further strategies such as SVC layering or multi-stream adaptation may also apply.

4) A Typical Congestion-Control Framework

Below is the early congestion-control framework once used in Google’s WebRTC:

It adopts a hybrid sender- and receiver-driven approach. The sender uses a loss-based algorithm:

If packet loss < 2%, increase sending bandwidth by 8%.
If packet loss is between 2% and 10%, keep sending bandwidth unchanged.
If packet loss > 10%, reduce sending bandwidth = current rate × (1 - 0.5 × loss).

The receiver uses a delay-based algorithm, estimating one-way delay with a Kalman filter, then combining that estimate with actual receive throughput to derive an optimal target bandwidth and feeding it back to the sender via RTCP. The sender then takes the minimum of the loss-based and delay-based estimates as the final target.

Below is WebRTC’s improved congestion-control framework:

Here, the entire congestion-control logic resides at the sender; the receiver merely provides measurement feedback.

The new framework refines the delay-estimation mechanism by applying linear regression to one-way delay variation. It discerns three trends—delay is rising, stable, or decreasing—and then uses that assessment along with the current send rate to determine the best target bandwidth.

In addition to refining detection, the new framework also introduces “proactive bandwidth probing,” further enhancing overall performance. It improves startup convergence as well as responsiveness to changing network conditions.

Packet Loss Issues

As noted, real-time interactive media typically runs over RTP/UDP, leaving packet-loss handling to the application layer.

From a network-transport standpoint, the primary techniques to handle packet loss are Automatic Repeat Request (ARQ) and Forward Error Correction (FEC). On the encoding side, depending on content or codec design, certain robustness features can offset loss—for instance, using B-frames in video encoding to reduce the impact of missing frames. Below, we focus on network-transport measures.

1) ARQ (Automatic Repeat Qequest)

In RTP/RTCP, the concept is straightforward: the receiver infers missing packets by gaps in sequence numbers and sends RTCP “NACK” requests for the sender to retransmit the missing data. The overall flow appears as follows:

When it detects a gap (say seq# 101 or 102), the receiver sends RTCP NACK(req 101, 102). The sender retransmits those packets, typically on a separate SSRC if possible.

Key details include:

Initial request delay: Decide whether to request a missing packet immediately or wait—possibly factoring in FEC usage.
Interval between repeated requests: For the same packet, do not resend the request before a full RTT elapses.
Limit on total requests: Calculate the maximum allowable requests based on RTT and permissible overall delay.
ARQ bandwidth limit: Automatic Repeat Request consumes part of total bandwidth and cannot exceed available capacity.
Means of returning retransmitted packets: Sending them via a dedicated RTP SSRC is recommended, facilitating separate tracking of packet loss and ARQ bandwidth.

2) FEC (Forward Error Correction)

In real-time audio-video, FEC is widely used due to its immediacy in recovering from lost packets.

The fundamental idea is that the sender appends extra repair packets containing redundant data computed over source packets. The receiver then uses these redundancy packets to recover any missing source packets, as shown below:

Algorithms for generating repair data range from simple XOR-based methods to matrix-based or other coding techniques, which we will not detail here.

Below is the basic transmitting-side FEC framework:

“ADU” stands for Application Data Unit, i.e. an audio or video packet. The FEC encoder produces repair (redundancy) packets at a certain protection ratio. These repair packets, along with the source packets and metadata describing their protection relationship, are sent to the receiver.

Below is the receiving-side FEC framework:

The receiver passes both source and repair packets to the FEC decoder. If any source packet is missing, the decoder reconstructs it using the available source and repair packets, then delivers the recovered audio-video data to the upper layer for decoding and playback.

In short, the “protection relationship” indicates which source packets each repair packet helps protect, typically signaled in a specialized format.

Under the RTP/RTCP framework, two standards define such formats:

ULPFEC (RFC5109): “Uneven Level Protection,” which allows using different protection levels for more or less important packets.
FlexFEC (RFC8627): “Flexible Forward Error Correction,” which supports both interleaved and non-interleaved parity-based FEC encoding for RTP packets.

3) Combining ARQ and FEC

Compared with FEC, ARQ’s shortcoming is added latency, but its advantage is higher bandwidth efficiency. Generally, the goal is to meet latency requirements while minimizing extra bandwidth and processing overhead for adequate protection.Hence, when combining ARQ and FEC, consider:

Use ARQ under moderate latency constraints. Based on RTT and the max permissible latency, calculate the maximum number of retransmissions.
If retransmissions alone can drive residual packet loss below ~1%, FEC might be unnecessary.
If you do need FEC, the FEC protection ratio should be based on the residual loss probability after retransmissions, providing the final safety net.

Below is an example showing how ARQ and FEC might work together at different RTT values. For RTT < 20ms and max delay < 100ms, on a link experiencing 30% loss, ARQ alone can reduce loss below 1%. Thus, ARQ suffices here. As RTT grows, FEC coverage grows accordingly. Ultimately, under high RTT, FEC alone handles lost packets.

Jitter Issues

In short, jitter means variation in network delay over time. Greater jitter indicates more fluctuation in transmission delay.

Jitter causes stutter, skip-ahead playback, and other issues that severely impact the quality of audio-video communication. It arises from multiple sources—new flows competing for the same bandwidth, unstable send rates, and general network volatility.

A common approach is to compensate at the receiver with a Jitter Buffer, as shown below:

By introducing a “Jitter Delay,” the buffer homogenizes incoming data to enable smooth playback.

Estimating the optimal buffer delay is crucial. If it’s too large, it adds extra delay; if it’s too small, it won’t fully absorb the jitter. In WebRTC, Google employs different approaches for audio and video:

Audio Jitter: The NetEQ module uses a histogram-based method with a forgetting factor, taking the 95th percentile of arrival intervals (Iat) as the buffer delay. This allows it to track variations over time while discarding outdated data.
Video Jitter: WebRTC uses a separate approach. It measures variations in frame size and latency, applying a Kalman filter for dynamic delay estimation.

However, WebRTC is primarily designed for one-on-one calls. In multi-party conferencing scenarios with media forwarding nodes in between, the end-to-end latency variation can be different, so further optimizations may be needed.

Tencent RTC’s Weak Network Countermeasure Optimizations in Practice

TRTC (Tencent RTC) excels in weak network environments. Its core technical optimizations and real-world test data demonstrate high performance even under complex network conditions: global end-to-end latency stays under 300ms, packet loss tolerance exceeds 80%, and it manages jitter above 1000ms. Details of its weak network resilience are as follows:

1. Transport-Layer Optimization

By using a protocol stack based on UDP + RTP/RTCP, it avoids the queue blocking and congestion-control latency seen in TCP under poor network conditions.

Adopting an NACK retransmission mechanism: An NACK request is only sent upon detecting packet-sequence discontinuities, enabling precise retransmission of missing packets and reducing the round-trip latency overhead from frequent acknowledgments.

Integrating FEC (Forward Error Correction): A small number of redundant packets are added at the sender, allowing the receiver to recover missing packets from the redundancy and drastically lowering latency caused by waiting for retransmissions.

TRTC can flexibly switch between or combine ARQ (retransmissions) and FEC, adjusting the number of retransmissions and FEC overhead in real time based on network conditions.

2. Adaptive Bandwidth Control (QoS)

QoS dynamically measures packet loss rate, RTT, and jitter to predict available bandwidth in real time.

Based on these predictions, it adjusts encoder bitrate, FEC redundancy ratios, and retransmission rates, responding quickly to changes in network status and avoiding both congestion from overly aggressive transmission and wasted overhead from unnecessary redundancy.

3. Improvements at the Codec Layer

TRTC modifies the video-frame reference structure so it is no longer fully dependent on the “previous frame,” reducing the impact a single lost frame can have on the subsequent decoding process, and thereby bolstering packet loss resilience.

For audio, TRTC employs PLC (Packet Loss Concealment). When a loss is detected, it estimates and interpolates the missing samples from preceding and following audio frames, preserving natural sound quality even with high packet loss.

4. Jitter Buffer and Synchronization Strategies

By setting an appropriate Jitter Buffer at the receiver, TRTC dynamically adjusts buffer size based on arrival times and playback schedules, balancing minimal latency with jitter compensation.

It also provides an audio-video synchronization mechanism: detecting discrepancies and jitter between audio and video arrivals, then using variable-speed playback to align audio with video, mitigating the negative impact of desynchronization on the viewing experience.

5. Echo Cancellation and “Double-Talk” Optimization

Relying on Tencent’s in-house TRAE (Tencent Real-time Audio Engine), TRTC applies acoustic echo suppression (AEC) between the speaker and microphone to prevent acoustic echo loops.

It further optimizes “double-talk” scenarios (where both parties speak simultaneously), eliminating issues like feedback squeals and improving clarity so that both ends can hear each other distinctly.

6. Cloud-Side Decision-Making and Big Data Analysis

TRTC deploys a QoE (Quality of Experience) assessment framework, gathering packet loss, latency, MOS scores, and other metrics in real time to accurately gauge users’ subjective experiences.

The cloud decision system dynamically fine-tunes parameters for different users and networks, ensuring every participant receives the optimal transmission strategy and the best possible audio-video quality.

By orchestrating these multiple techniques, TRTC consistently delivers low-latency, high-quality performance in harsh network environments characterized by high packet loss and severe jitter, making it ideal for online education, remote work, interactive live streaming, and similar scenarios.

How to Add Beauty Effects to Your Android Live-streaming App

LunarDrift — Sun, 18 May 2025 02:06:01 +0000

Previously, we explored how to build a live streaming app and how to implement beauty features. In this tutorial, we’ll show you how to add beauty effects to your Android live-streaming application. First, we’ll walk you through setting up a fully functional live streaming app, then we’ll integrate the Tencent Effects Beauty SDK. The Tencent RTC SDK provides a comprehensive, built-in UI, enabling you to quickly achieve sharp, real-time streaming with minimal effort.

Step 1. Activate the service

Before using the Audio and Video Services provided by Tencent Cloud, you need to go to the Console and activate the service for your application. For detailed steps, refer to Activate the service.

Step 2: Download the TUILiveKit component

Clone/download the code in Github, and then copy the TUILiveKit subdirectory in the Android directory to the same level directory as the app in your current project, as shown below:

Step 3: Configure the project

In the root directory of the project, add the jitpack repository URL to the settings.gradle.kts (or settings.gradle) file: Add the jitpack repository dependency (to download the SVG animation library for playing gifts, SVGAPlayer):

dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
    repositories {
        google()
        mavenCentral()

        // Add the jitpack repository URL
        maven { url = uri("https://jitpack.io") }
    }
}

In the root directory of the project, add the following code to the settings.gradle.kts (or settings.gradle) file. It will import the tuilivekit component downloaded in Step 2 into your current project:

include(":tuilivekit:livekit")
include(":tuilivekit:component:barrage")

In the app directory, find the build.gradle.kts (or build.gradle) file and add the following code. It declares the dependency of the current app on the newly added tuilivekit component:

api(project(":tuilivekit:livekit"))

As the SDK uses Java's reflection feature internally, you need to add certain classes in the SDK to the obfuscation allowlist by adding the following code to the proguard-rules.pro file:

-keep class com.tencent.** { *; }
-keep class com.trtc.uikit.livekit.livestreamcore.** { *; }
-keep class com.trtc.uikit.livekit.component.gift.store.model.** { *; }
-keep class com.squareup.wire.** { *; }
-keep class com.opensource.svgaplayer.proto.** { *; }

-keep class com.tcmediax.** { *; }
-keep class com.tencent.** { *; }
-keep class com.tencent.xmagic.** { *; }
-keep class androidx.exifinterface.** {*;}
-keep class com.google.gson.** { *;}
# Tencent Effect SDK - beauty
-keep class com.tencent.xmagic.** { *;}
-keep class org.light.** { *;}
-keep class org.libpag.** { *;}
-keep class org.extra.** { *;}
-keep class com.gyailib.**{ *;}
-keep class com.tencent.cloud.iai.lib.** { *;}
-keep class com.tencent.beacon.** { *;}
-keep class com.tencent.qimei.** { *;}
-keep class androidx.exifinterface.** { *;}

In the app directory, find the AndroidManifest.xml file, and add tools:replace="android:allowBackup" and android:allowBackup="false" to the application node to override the component's settings with your own.

  // app/src/main/AndroidManifest.xml

  <application
    ...

    // Add the following configuration to override the settings in the dependent SDK
    android:allowBackup="false"
    tools:replace="android:allowBackup">

Step 4. Log in

Add the following code to your project. It calls the relevant interfaces in TUICore to complete the TUI component log in to. This step is crucial; you can only use the features provided by TUILiveKit after successfully logging in.

TUILogin.login(applicationContext,
    1400000001,     // Please replace with the SDKAppID obtained in step one
    "denny",        // Please replace with your UserID
    "xxxxxxxxxxx",  // You can calculate a UserSig in the console and fill it in here
    object : TUICallback() {
        override fun onSuccess() {
            Log.i(TAG, "login success")
        }

        override fun onError(errorCode: Int, errorMessage: String) {
            Log.e(TAG, "login failed, errorCode: $errorCode msg:$errorMessage")
        }
    })

Use the SecretKey obtained in Step One, Step 3 to encrypt information such as SDKAppID and UserID to obtain UserSig, which is a token for authentication used by Tencent Cloud to identify whether the current user can use TRTC services. You can generate a temporarily usable UserSig through the Auxiliary Tools in the console. For more information, see UserSig.

Step5. Integrating tebeautykit

Now lets utilizes Tencent Effects Beauty for advanced beauty effects.

Copy the Android/tebeautykit folder to your project, at the same level as the app folder.

Please edit your project's settings.gradle file and add the following code:

  include ':tebeautykit'

Step 6: Authorization & Setting Beauty Resources

Apply for authorization and obtain LicenseUrl and LicenseKey. Please refer to the License Guide for more information.

In the initialization section of your business logic (typically in the same location as the login process), Add the following authorization code and replace it with the Beauty Package ID, LicenseUrl, and LicenseKey you have obtained:

TEBeautySettings.getInstance().initBeautySettings(context, 
                                       S1_07,         // Replace S1_07 with the beauty package number you purchased.
                                       "LicenseUrl",  // Replace with your LicenseUrl
                                       "LicenseKey"); // Replace with youLicenseKey

By completing the aforementioned steps, you will have successfully integrated advanced beauty effects.

Step 7. Start live streaming

After the above log in to method call returns successfully, call the VideoLiveKit's startLive method, specifying the room ID to start your live streaming page.

import com.trtc.uikit.livekit.VideoLiveKit
VideoLiveKit.createInstance(applicationContext).startLive("roomId")

Step 8. Watch the live stream

After the above log in to method call returns successfully, call the VideoLiveKit's joinLive method, specifying the room ID to start your live streaming page.

import com.trtc.uikit.livekit.VideoLiveKit
VideoLiveKit.createInstance(applicationContext).joinLive("roomId")

My name is Susie. I am a writer and Media Service Product Manager. I work with startups across the globe to build real-time communication solutions using SDK and APIs.

If you want to build face filter or other special effect into your app, welcome to contact us.

Easiest Steps To Build A Face Filter App With Flutter

LunarDrift — Sat, 17 May 2025 06:39:46 +0000

Hello everyone, I’m excited to share this article with you all.

By the end of this article, you’ll be able to build a face filter app from scratch using Flutter and the TRTC Beauty AR SDK. You’ll get hands-on experience with AR stickers, face filters, virtual makeup, and face/gesture recognition — and you can customize all the special effects to your liking.

Step 1: Creating a new Flutter project.

Select Mobile, and based on your actual needs, fill in the App Name, Package Name, and Bundle ID. Check the features you want to trial: All Beauty Features, Virtual Background, Face Recognition, Gesture Recognition, then click Confirm.

After activation, you can view your information on the current page and refer to the Integration Guide below for integration. The License Url and License Key will be used in the integration guide.

Beauty Flutter SDK requires dependence on the Beauty SDK of the Android/iOS end. Through the Plugins provided by Flutter, the native end features can be exposed to the Flutter client. Therefore, when integrating the beauty filter features, you need to integrate the SDK of the native end manually.

Step 2: Running Demo

Download the demo project, modify the demo/lib under the main.dart file, add your licenseUrl and licenseKey in this file. The sample code for using beauty features in TRTC is primarily located in demo/lib/page/trtc_page.dart and demo/lib/main.dart.

Android

In demo/app, find the build.gradle file, open the file, and change the value of applicationId to the package name you used when applying for the license.
In demo/lib, execute flutter pub get.
Use Android Studio to open the demo project and run it.

Step 3: SDK Integration

In the app module, find the build.gradle file and add the Maven repository URL for your corresponding package. For example, if you choose the S1-07 package, add the following:

dependencies {
   implementation 'com.tencent.mediacloud:TencentEffect_S1-07:latest.release'
}

For the Maven URL corresponding to each package, please refer to the documentation. The latest version number of the SDK can be found in theRelease History.

In your project, find the android/app module and locate the src/main/assets folder. Copy the demo/android/app/src/main/assetslut and MotionRes folders from the demo project to your own project's android/app/src/main/assets. If your project doesn't have an assets folder, you can create one manually.
If you are using an Android of the Tencent Effect SDK that is lower than 3.9, you will need to locate the AndroidManifest.xml file under the app module, and add the following Tag in the application section:

   <uses-native-library
            android:name="libOpenCL.so"
            android:required="false" />
        //true indicates that libOpenCL is essential for the current app. Without this library, the system will not allow the app to install
        //false indicates that libOpenCL is not essential for the current app. The app can be installed normally with or without this library. If the device has this library, GAN-type special effects in the Tencent Special Effects SDK (e.g., fairy tale face, comics face) will function normally. If the device does not have this library, GAN-type effects won't work, but it will not affect the use of other features within the SDK.
        //For information about uses-native-library, please refer to the Android official website: https://developer.android.com/guide/topics/manifest/uses-native-library-element

Obfuscation configuration

If you enable compile optimization (setting minifyEnabled to true) when packaging the release, it will trim some code that is not called in the Java layer. This code may possibly be invoked by the native layer, thus causing the no xxx method exception.

If you enabled such compile optimization, you should add these keep rules to avoid trimming xmagic's code:

-keep class com.tencent.xmagic.** { *;}
-keep class org.light.** { *;}
-keep class org.libpag.** { *;}
-keep class org.extra.** { *;}
-keep class com.gyailib.**{ *;}
-keep class com.tencent.cloud.iai.lib.** { *;}
-keep class com.tencent.beacon.** { *;}
-keep class com.tencent.qimei.** { *;}
-keep class androidx.exifinterface.** { *;}

Step 4: Flutter Integration

Method 1:

Remote Dependencies, add the following reference in your project's pubspec.yaml file:

  tencent_effect_flutter:
   git:
     url: https://github.com/Tencent-RTC/TencentEffect_Flutter

Method 2:

Local Dependencies, download the latest version of tencent_effect_flutter, then add the folders android, ios, lib and the files pubspec.yaml, tencent_effect_flutter.iml to your project directory, and then add the following reference in your project's pubspec.yaml file: (refer to demo)

tencent_effect_flutter:
    path: ../

Run the following command:

flutter pub get

Step 5: SDK usage

1. Associated with TRTC

Add the following code in the onCreate method of the application class (or the onCreate method of FlutterActivity):

TRTCCloudPlugin.register(new XmagicProcesserFactory());

Call the resource initialization API

void _initSettings(InitXmagicCallBack callBack) async {
  _setResourcePath();
  /// Resource copying only needs to be done once. In the current version, if copied successfully once, there is no need to copy the resources again.
  /// Copying the resource only needs to be done once. Once it has been successfully copied in the current version, there is no need to copy it again in future versions.
  if (await isCopiedRes()) {
    callBack.call(true);
    return;
  } else {
    _copyRes(callBack);
  }
}

void _setResourcePath() async {
  String resourceDir = await ResPathManager.getResManager().getResPath();
  TXLog.printlog(
      '$TAG method is _initResource ,xmagic resource dir is $resourceDir');
  TencentEffectApi.getApi()?.setResourcePath(resourceDir);
}

void _copyRes(InitXmagicCallBack callBack) {
  _showDialog(context);
  TencentEffectApi.getApi()?.initXmagic((result) {
    if (result) {
      saveResCopied();
    }
    _dismissDialog(context);
    callBack.call(result);
    if (!result) {
      Fluttertoast.showToast(msg: "initialization failed");
    }
  });
}

3. Perform beauty authorization

TencentEffectApi.getApi()?.setLicense(licenseKey, licenseUrl,
           (errorCode, msg) {
         TXLog.printlog("Print the authentication result errorCode = $errorCode   msg = $msg");
         if (errorCode == 0) {
            // Authentication succeeded
         }
       });

4. Enable Beauty Filter

/// Enable beauty filter operation
 var enableCustomVideo = await trtcCloud.enableCustomVideoProcess(open);

5. Set Beauty Attributes

For specific beauty attributes, refer to Beauty Attributes Table.

TencentEffectApi.getApi()?.setEffect(sdkParam.effectName!,
    sdkParam.effectValue, sdkParam.resourcePath, sdkParam.extraInfo)

6. Set other properties

Pause Beauty Sound Effects

TencentEffectApi.getApi()?.onPause();

Resume Beauty Sound Effects

TencentEffectApi.getApi()?.onResume();

Monitor Beauty Events

TencentEffectApi.getApi()
     ?.setOnCreateXmagicApiErrorListener((errorMsg, code) {
       TXLog.printlog("Error creating an effect object errorMsg = $errorMsg , code = $code");
 });   /// Needs to be set before creating the beauty filter

Set Callback for Face, Gesture, and Body Detection Status

TencentEffectApi.getApi()?.setAIDataListener(XmagicAIDataListenerImp());

Set Callback Function for Dynamic Prompt Messages

TencentEffectApi.getApi()?.setTipsListener(XmagicTipsListenerImp());

Configure the Callback of Facial Keypoints and Other Data (only available in S1-05 and S1-06)

TencentEffectApi.getApi()?.setYTDataListener((data) {
   TXLog.printlog("setYTDataListener  $data");
 });

Remove All Callbacks. You need to remove all callbacks when terminating the page:

 TencentEffectApi.getApi()?.setOnCreateXmagicApiErrorListener(null);
 TencentEffectApi.getApi()?.setAIDataListener(null);
 TencentEffectApi.getApi()?.setYTDataListener(null);
 TencentEffectApi.getApi()?.setTipsListener(null);

For more information on the APIs, see API Documentation. For others, refer to the Demo Project.

7. Add Data to and Remove Data from the Effect Panel

Add effect resources.

Add your resource file to the corresponding folder as described in step 1. For example, to add a 2D animated effect resource:

You should put the resource in android/xmagic/src.mian/assets/MotionRes/2dMotionRes of your project:

Beauty Panel Configuration

The demo provides a simple beauty panel UI. The panel's properties are configured through JSON files, located as shown in the following diagram. The implementation of the panel can be referred from the demo project.

With abundant built-in excellent materials, special effects, templates, trendy makeup effects, and interactive games can be customized. See detailed usage instructions for more information Beauty AR Studio Introduction.

Congratulation, you have built your first Face Filter App with Flutter and Tencent RTC Beauty AR SDK.

My name is Susie. I am a writer and Media Service Product Manager. I work with startups across the globe to build real-time communication solutions using SDK and APIs.

If you want to build face filter or other special effect into your app, welcome to contact us.

Implement AI real-time subtitles in online meetings

LunarDrift — Mon, 12 May 2025 13:52:37 +0000

This article will show you how to quickly install a basic conference demo, and further introduce the AI transcription function so that the voice content in the meeting can be presented in real time in the form of subtitles. Whether you want to build a smarter business meeting tool or improve the accessibility of online classes and live roadshows, this guide will provide you with clear ideas and operational examples.

First, we quickly install a basic Conference demo. Open the Terminal, copy and paste the sample command to clone the repository.

git clone https://github.com/Tencent-RTC/TUIRoomKit.git

Install dependencies.

cd ./TUIRoomKit/Web/example/vite-vue3-ts

npm install

Go to the Activate Service page and get the SDKAppID and SDKSecretKey*, then fill them in the TUIRoomKit/Web/example/vite-vue3-ts/src/config/basic-info-config.js* file.

Function Introduction

After accessing TUIRoomKit, you can turn on the AI real-time subtitle function by clicking ‘AI Assistant’ in the bottom bar, and you can see "AI real-time subtitles", which ****display the discussion during the meeting in the form of subtitles.

Function Access

Step 1: Start Local Backend Service

Note：Open AI transcription need to use the user's cloud id and key, high sensitivity, so do not integrate this interface in the client, you need to add your own business background StartAITranscription related code, here to Nodejs service as an example, if your background service for other languages, you can click debugging, to generate the corresponding language examples of code.

Open Nodejs backend service, the client listens for the user to enter the room and opens the AI transcription task via http request, sample code: **click to download.**

require('dotenv').config();
const express = require('express');
const tencentcloud = require('tencentcloud-sdk-nodejs-trtc');
const cors = require('cors')

const TrtcClient = tencentcloud.trtc.v20190722.Client;

// Get the Tencent RTC account SecretId and SecretKey from the environment variable
// You need to pass the Tencent RTC account SecretId and SecretKey into the entry parameter, and you also need to pay attention to the confidentiality of the key pair here.
// Code leaks can lead to SecretId and SecretKey leaks and threaten the security of all resources under the account.
// The following code example is for reference only, a more secure way of using the key is recommended, see: https://cloud.tencent.com/document/product/1278/85305
// The key can be obtained by going to the official console at https://console.cloud.tencent.com/cam/capi.
const secretId = process.env.TENCENT_SECRET_ID || '';
const secretKey = process.env.TENCENT_SECRET_KEY || '';
const region = process.env.TENCENT_REGION || '';

const clientConfig = {
  credential: {
    secretId: secretId,
    secretKey: secretKey,
  },
  region: region,
  profile: {
    httpProfile: {
      endpoint: 'trtc.tencentcloudapi.com',
    },
  },
};

const client = new TrtcClient(clientConfig);

const app = express();
app.use(express.json());
app.use(cors());
app.post('/start', async (req, res) => {
  const { SdkAppId, RoomId, RoomIdType = 1, UserId, UserSig } = req.body;
  const params = {
    SdkAppId: SdkAppId,
    RoomId: RoomId,
    RoomIdType: RoomIdType,
    TranscriptionParams: {
      UserId: UserId,
      UserSig: UserSig,
    },
  };

  try {
    const data = await client.StartAITranscription(params);
    console.log('success',data)
    res.status(200).json(data);
  } catch (err) {
    console.error('error', err);
    res.status(500).json({ error: err.message });
  }
});

app.post('/stop', async (req, res) => {
  const { TaskId } = req.body;

  try {
    const data = await client.StopAITranscription({ TaskId: TaskId });
    res.status(200).json(data);
  } catch (err) {
    console.error('error', err);
    res.status(500).json({ error: err.message });
  }
});

const port = process.env.PORT || 3000;
app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});

Step 2: Roomkit Enables AI Assistants

Note：

Roomkit only processes data for AI captioning/meeting summary, the actual ASR is turned on when the client user enters the room, here you can adjust the timing according to your business needs.

<template>
  <conference-main-view display-mode="permanent"></conference-main-view>
</template>
<script setup lang="ts">
import { roomService } from '@tencentcloud/roomkit-web-vue3';
// Called before the conference-main-view component is onmounted.
roomService.setComponentConfig({ AIControl: { visible: true } });
</script>

Step 3: Roomkit listens for the user to enter the room and calls the node service to enable AI transcription.

<template>
  <conference-main-view display-mode="permanent"></conference-main-view>
</template>
<script setup lang="ts">
import { conference } from '@tencentcloud/roomkit-web-vue3';
import { startAITranscription } from '../http';
const handleAITask = (data: {roomId: string}) => {
  const { roomId } = data;
  startAITranscription({
    RoomId: roomId,
    UserId: 'robot', // A robot user is required here, in the case of robot, this should not be the same as the userId of the user in the room, it is recommended to use robot.
    UserSig: 'xxx', // The userSig of the robot
    SdkAppId: sdkAppId,
    RoomIdType: 1, // Room type is string room
  });
};
conference.on(RoomEvent.ROOM_JOIN, handleAITask);
conference.on(RoomEvent.ROOM_START, handleAITask);
onUnmounted(() => {
  conference.off(RoomEvent.ROOM_JOIN, handleAITask);
  conference.off(RoomEvent.ROOM_START, handleAITask);
});
</script>

// http.ts
import axios from 'axios';
const http = axios.create({
  baseURL: 'http://localhost:3000', // Your Node.js service address.
  timeout: 10000, // Request timeout
});

interface TranscriptionParams {
    SdkAppId: number;
    RoomId: string;
    RoomIdType?: number;
    UserId: string;
    UserSig: string;
  }

  interface StopParams {
    TaskId: string;
  }

// Start the AI transcription task
export function startAITranscription(params: TranscriptionParams) {
  return http.post('/start', params);
}

// Stop the AI transcription task
export function stopAITranscription(params: StopParams) {
  return http.post('/stop', params);
}