How to Build an AI Chatbot on Android with Flutter

Flutter has revolutionized cross-platform mobile development. With a single codebase, you can deploy to both Android and iOS—perfect for startups and enterprises alike. If you're looking to build an AI chatbot for Android using Flutter, this guide is for you.

This tutorial demonstrates how to create a fully functional Android AI chatbot using Flutter and ZEGOCLOUD's Conversational AI platform. You'll implement speech recognition, language processing, and text-to-speech—all through a clean Dart SDK. By the end, your Flutter app will deliver real-time voice conversations that work seamlessly across platforms.

Let's begin.

Quick Links

Resource	Repository
Backend Server + Web Client	github.com/ZEGOCLOUD/blog-aiagent-server-and-web
Flutter Client	github.com/ZEGOCLOUD/blog-aiagent-flutter

Prerequisites

Before you begin, ensure you have:

• Flutter SDK (latest stable version)
• Android Studio or VS Code with Flutter extensions
• ZEGOCLOUD Account — Create free account
• Backend deployment — Next.js on Vercel (covered in this guide)

Architecture Overview

System Components

ZEGOCLOUD's Conversational AI handles three core cloud services:

Service	Function
ASR (Automatic Speech Recognition)	Voice to text conversion
LLM (Large Language Model)	Response generation
TTS (Text-to-Speech)	Natural speech synthesis

System Flow

sequenceDiagram
    participant User as Flutter Client
    participant Server as Backend Server
    participant ZEGO as ZEGO Cloud
    participant LLM as LLM Service
    participant TTS as TTS Service

    User->>Server: 1. Request Token
    Server-->>User: 2. Return Token

    User->>ZEGO: 3. Login Room & Publish Audio Stream
    User->>Server: 4. Request to Start AI Agent
    Server->>ZEGO: 5. Register Agent (LLM/TTS config)
    Server->>ZEGO: 6. Create Agent Instance
    ZEGO-->>Server: 7. Return Agent Instance ID
    Server-->>User: 8. Return Agent Stream ID

    User->>ZEGO: 9. Play Agent's Audio Stream

    loop Conversation
        User->>ZEGO: User speaks (audio stream)
        ZEGO->>ZEGO: ASR: Speech to Text
        ZEGO->>LLM: Send text to LLM
        LLM-->>ZEGO: LLM response
        ZEGO->>TTS: Text to Speech
        TTS-->>ZEGO: Audio data
        ZEGO-->>User: AI voice + Subtitles
    end

    User->>Server: 10. Request to Stop AI Agent
    Server->>ZEGO: 11. Delete Agent Instance

Why Do We Need Both Server and Client?

When developing an AI chatbot for Android applications with Flutter, developers often ask: Why is a backend server necessary? This architecture decision is fundamental to building secure, production-ready applications.

The Three-Tier Architecture:

Tier	Responsibility	Trust Level
Backend Server	Token generation, AI agent lifecycle, API signing	High (your infrastructure)
Flutter Client	Audio I/O, stream management, UI rendering	Low (runs on user devices)
ZEGO Cloud	ASR, LLM, TTS processing	External (third-party service)

Security Considerations:

1. Credential Protection — Sensitive values like ZEGO_SERVER_SECRET and LLM API keys must remain on your server. Mobile apps can be reverse-engineered, exposing any embedded secrets.

2. Short-Lived Tokens — Instead of permanent credentials, clients request time-limited tokens (typically 1 hour). This containment strategy limits exposure if a token is intercepted.

3. Server-Side AI Configuration — System prompts, model selections, and voice settings live on your server. This enables dynamic AI behavior updates without requiring app store releases.

4. Monitoring & Controls — Your backend provides a single point for logging, rate limiting, abuse detection, and usage analytics.

Understanding this separation is critical when you build an AI chatbot on Android for production environments.

Step 1: Set Up the Backend

Your backend manages authentication and the AI agent lifecycle.

1.1 Environment Configuration

Create .env.local in your server root:

# ZEGO Credentials
NEXT_PUBLIC_ZEGO_APP_ID=your_app_id
ZEGO_SERVER_SECRET=your_32_char_secret

# AI Agent Settings
ZEGO_AGENT_ID=aiAgent1
ZEGO_AGENT_NAME=AI Assistant

# AI Personality
SYSTEM_PROMPT="You are my best friend whom I can talk to about anything. You're warm, understanding, and always there for me."

# LLM Configuration
LLM_URL=https://your-llm-provider.com/api/chat/completions
LLM_API_KEY=your_llm_api_key
LLM_MODEL=your_model_name

# TTS Configuration
TTS_VENDOR=ByteDance
TTS_APP_ID=zego_test
TTS_TOKEN=zego_test
TTS_CLUSTER=volcano_tts
TTS_VOICE_TYPE=zh_female_wanwanxiaohe_moon_bigtts

Variable	Purpose	Source
NEXT_PUBLIC_ZEGO_APP_ID	Application ID	ZEGOCLOUD Console
ZEGO_SERVER_SECRET	32-char secret	ZEGOCLOUD Console
SYSTEM_PROMPT	AI behavior	Customize per use case
LLM_*	Language model	Your LLM provider
TTS_*	Voice settings	Test or production TTS

1.2 Token Generation

// app/api/zego/token/route.ts
import { NextRequest, NextResponse } from 'next/server';
import crypto from 'crypto';

function generateToken(appId: number, userId: string, secret: string,
                       effectiveTimeInSeconds: number): string {
  const tokenInfo = {
    app_id: appId,
    user_id: userId,
    nonce: Math.floor(Math.random() * 2147483647),
    ctime: Math.floor(Date.now() / 1000),
    expire: Math.floor(Date.now() / 1000) + effectiveTimeInSeconds,
    payload: ''
  };

  const plainText = JSON.stringify(tokenInfo);
  const nonce = crypto.randomBytes(12);
  const cipher = crypto.createCipheriv('aes-256-gcm', secret, nonce);
  const encrypted = Buffer.concat([cipher.update(plainText, 'utf8'),
                                   cipher.final(), cipher.getAuthTag()]);

  const buf = Buffer.concat([
    Buffer.alloc(8).writeBigInt64BE(BigInt(tokenInfo.expire), 0),
    Buffer.from([0, 12]), nonce,
    Buffer.from([encrypted.length >> 8, encrypted.length & 0xff]), encrypted,
    Buffer.from([1])
  ]);
  return '04' + buf.toString('base64');
}

export async function POST(request: NextRequest) {
  const { userId } = await request.json();
  const token = generateToken(
    parseInt(process.env.NEXT_PUBLIC_ZEGO_APP_ID!),
    userId,
    process.env.ZEGO_SERVER_SECRET!,
    3600
  );
  return NextResponse.json({ token });
}

1.3 Deploy to Vercel

1. Push to GitHub
2. Import in Vercel
3. Add environment variables
4. Deploy

Server will be at https://your-project.vercel.app.

---

Step 2: Create the Flutter Application

Now we'll create an AI chatbot in Android using Flutter and Dart.

2.1 Create Flutter Project

flutter create aiagent_demo
cd aiagent_demo

2.2 Add Dependencies

Update pubspec.yaml:

dependencies:
  flutter:
    sdk: flutter

  # ZEGO Express SDK
  zego_express_engine: ^3.22.0

  # HTTP client
  http: ^1.2.0

  # Permission handling
  permission_handler: ^11.3.0

Run:

flutter pub get

2.3 Configure Permissions

Android (android/app/src/main/AndroidManifest.xml):

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

iOS (ios/Runner/Info.plist):

<key>NSMicrophoneUsageDescription</key>
<string>This app requires microphone access for voice chat</string>

2.4 App Configuration

// lib/config/app_config.dart
class AppConfig {
  // Must match backend NEXT_PUBLIC_ZEGO_APP_ID
  static const int appID = 1234567890;

  // Your Vercel deployment URL
  static const String serverURL = 'https://your-project.vercel.app';

  // Generate unique room ID
  static String generateRoomId() {
    final random = DateTime.now().millisecondsSinceEpoch % 100000;
    return 'room$random';
  }

  // Generate unique user ID
  static String generateUserId() {
    final random = DateTime.now().millisecondsSinceEpoch % 100000;
    return 'user$random';
  }

  // Generate stream IDs
  static String getAgentStreamId(String roomId) => 'agent_stream_$roomId';
  static String getUserStreamId(String roomId) => 'user_stream_$roomId';
}

2.5 API Service Layer

// lib/services/api_service.dart
import 'dart: convert';
import 'package:http/http.dart' as http;
import '../config/app_config.dart';

class ApiService {
  /// Get authentication token
  static Future<String?> getToken(String userId) async {
    try {
      final response = await http.post(
        Uri.parse('${AppConfig.serverURL}/api/zego/token'),
        headers: {'Content-Type': 'application/json'},
        body: jsonEncode({'userId': userId}),
      );

      if (response.statusCode == 200) {
        final data = jsonDecode(response.body);
        if (data['code'] == 0 && data['data'] != null) {
          return data['data']['token'] as String?;
        }
      }
      return null;
    } catch (e) {
      print('[ApiService] Token error: $e');
      return null;
    }
  }

  /// Start AI agent instance
  static Future<String?> startAgent({
    required String roomId,
    required String userId,
    required String userStreamId,
  }) async {
    try {
      final response = await http.post(
        Uri.parse('${AppConfig.serverURL}/api/zego/start'),
        headers: {'Content-Type': 'application/json'},
        body: jsonEncode({
          'roomId': roomId,
          'userId': userId,
          'userStreamId': userStreamId,
        }),
      );

      if (response.statusCode == 200) {
        final data = jsonDecode(response.body);
        if (data['code'] == 0 && data['data'] != null) {
          return data['data']['agentStreamId'] as String?;
        }
      }
      return null;
    } catch (e) {
      print('[ApiService] Start agent error: $e');
      return null;
    }
  }

  /// Stop AI agent
  static Future<bool> stopAgent(String roomId) async {
    try {
      final response = await http.post(
        Uri.parse('${AppConfig.serverURL}/api/zego/stop'),
        headers: {'Content-Type': 'application/json'},
        body: jsonEncode({'roomId': roomId}),
      );

      return response.statusCode == 200;
    } catch (e) {
      print('[ApiService] Stop agent error: $e');
      return false;
    }
  }
}

2.6 ZEGO Engine Manager

// lib/services/zego_express_manager.dart
import 'package:zego_express_engine/zego_express_engine.dart';
import '../config/app_config.dart';

class ZegoExpressManager {
  static final ZegoExpressManager _instance = ZegoExpressManager._internal();
  factory ZegoExpressManager() => _instance;
  ZegoExpressManager._internal();

  Function(String, ZegoRoomStateChangedReason, int)? onRoomStateChanged;
  bool _isInitialized = false;

  /// Initialize ZEGO Express Engine
  Future<void> initEngine() async {
    if (_isInitialized) return;

    // Configure engine
    final engineConfig = ZegoEngineConfig(advancedConfig: {
      'set_audio_volume_ducking_mode': '1',
      'enable_rnd_volume_adaptive': 'true',
    });
    await ZegoExpressEngine.setEngineConfig(engineConfig);

    // Create engine with profile
    final profile = ZegoEngineProfile(
      AppConfig.appID,
      ZegoScenario.HighQualityChatroom,
      enablePlatformView: false,
    );
    await ZegoExpressEngine.createEngineWithProfile(profile);

    // Setup event handlers
    _setupEventHandlers();

    // Configure audio (3A processing)
    await _configureAudioSettings();

    _isInitialized = true;
  }

  void _setupEventHandlers() {
    ZegoExpressEngine.onRoomStateChanged = (roomID, reason, errorCode, _) {
      onRoomStateChanged?.call(roomID, reason, errorCode);
    };
  }

  Future<void> _configureAudioSettings() async {
    final engine = ZegoExpressEngine.instance;
    await engine.enableAGC(true);  // Automatic Gain Control
    await engine.enableAEC(true);  // Echo Cancellation
    await engine.setAECMode(ZegoAECMode.AIBalanced);
    await engine.enableANS(true);  // Noise Suppression
    await engine.setANSMode(ZegoANSMode.Medium);
  }

  /// Login to room
  Future<int> loginRoom(String roomId, String userId, String token) async {
    final user = ZegoUser(userId, userId);
    final config = ZegoRoomConfig(0, true, token);
    final result = await ZegoExpressEngine.instance.loginRoom(
      roomId, user, config: config,
    );
    return result.errorCode;
  }

  /// Start publishing audio
  Future<void> startPublishing(String streamId) async {
    await ZegoExpressEngine.instance.muteMicrophone(false);
    await ZegoExpressEngine.instance.startPublishingStream(streamId);
  }

  /// Play remote stream
  Future<void> startPlaying(String streamId) async {
    await ZegoExpressEngine.instance.startPlayingStream(streamId);
  }

  /// Stop publishing
  Future<void> stopPublishing() async {
    await ZegoExpressEngine.instance.stopPublishingStream();
  }

  /// Stop playing
  Future<void> stopPlaying(String streamId) async {
    await ZegoExpressEngine.instance.stopPlayingStream(streamId);
  }

  /// Logout from room
  Future<void> logoutRoom(String roomId) async {
    await ZegoExpressEngine.instance.logoutRoom(roomId);
  }

  /// Destroy engine
  Future<void> destroyEngine() async {
    if (!_isInitialized) return;
    await ZegoExpressEngine.destroyEngine();
    _isInitialized = false;
  }
}

2.7 ViewModel (Provider Pattern)

// lib/viewmodels/chat_viewmodel.dart
import 'package:flutter/foundation.dart';
import '../services/api_service.dart';
import '../services/zego_express_manager.dart';

class ChatViewModel extends ChangeNotifier {
  final ZegoExpressManager _zegoManager = ZegoExpressManager();

  bool _isConnected = false;
  bool _isLoading = false;
  String? _errorMessage;
  String? _currentRoomId;
  String? _currentUserId;

  bool get isConnected => _isConnected;
  bool get isLoading => _isLoading;
  String? get errorMessage => _errorMessage;

  String get statusText {
    if (_isLoading) return 'Processing...';
    return _isConnected ? 'Connected' : 'Not connected';
  }

  ChatViewModel() {
    _setupCallbacks();
  }

  void _setupCallbacks() {
    _zegoManager.onRoomStateChanged = (roomId, reason, errorCode) {
      if (reason == ZegoRoomStateChangedReason.Logined) {
        _isConnected = true;
        notifyListeners();
      } else if (reason == ZegoRoomStateChangedReason.Logout ||
                 reason == ZegoRoomStateChangedReason.KickOut) {
        _isConnected = false;
        notifyListeners();
      }
    };
  }

  /// Start AI conversation
  Future<void> startCall() async {
    if (_isLoading) return;

    _isLoading = true;
    _errorMessage = null;
    notifyListeners();

    try {
      _currentRoomId = AppConfig.generateRoomId();
      _currentUserId = AppConfig.generateUserId();
      final userStreamId = AppConfig.getUserStreamId(_currentRoomId!);

      // Initialize
      await _zegoManager.initEngine();

      // Get token
      final token = await ApiService.getToken(_currentUserId!);
      if (token == null) throw Exception('Failed to get token');

      // Login room
      final loginResult = await _zegoManager.loginRoom(
        _currentRoomId!, _currentUserId!, token,
      );
      if (loginResult != 0) throw Exception('Login failed: $loginResult');

      // Publish stream
      await _zegoManager.startPublishing(userStreamId);

      // Start AI agent
      final agentStreamId = await ApiService.startAgent(
        roomId: _currentRoomId!,
        userId: _currentUserId!,
        userStreamId: userStreamId,
      );
      if (agentStreamId == null) throw Exception('Failed to start AI agent');

      // Play agent stream
      await _zegoManager.startPlaying(agentStreamId);

      _isConnected = true;
    } catch (e) {
      _errorMessage = e.toString();
      await _cleanup();
    } finally {
      _isLoading = false;
      notifyListeners();
    }
  }

  /// End call
  Future<void> endCall() async {
    if (_isLoading) return;

    _isLoading = true;
    notifyListeners();

    await _cleanup();

    _isLoading = false;
    notifyListeners();
  }

  Future<void> _cleanup() async {
    if (_currentRoomId != null) {
      await ApiService.stopAgent(_currentRoomId!);
      await _zegoManager.stopPlaying(
        AppConfig.getAgentStreamId(_currentRoomId!));
      await _zegoManager.stopPublishing();
      await _zegoManager.logoutRoom(_currentRoomId!);
    }
    _isConnected = false;
    _currentRoomId = null;
    _currentUserId = null;
  }

  @override
  void dispose() {
    _zegoManager.destroyEngine();
    super.dispose();
  }
}

2.8 Subtitle Display

Create subtitle models and views using ZEGO's official components. Download from ZEGO Flutter Subtitle Guide.

Key files:

• lib/audio/subtitles/model.dart — Message data model
• lib/audio/subtitles/view.dart — UI display widget
• lib/audio/subtitles/message_protocol.dart — Message parsing
• lib/audio/subtitles/message_dispatcher.dart — Event routing

2.9 Main UI (ChatPage)

// lib/widgets/chat_page.dart
import 'package:flutter/material.dart';
import 'package:zego_express_engine/zego_express_engine.dart';
import '../viewmodels/chat_viewmodel.dart';
import '../audio/subtitles/view.dart';
import '../audio/subtitles/model.dart';
import '../audio/subtitles/message_dispatcher.dart';

class ChatPage extends StatefulWidget {
  const ChatPage({super.key});

  @override
  State<ChatPage> createState() => _ChatPageState();
}

class _ChatPageState extends State<ChatPage> implements ZegoSubtitlesEventHandler {
  final ChatViewModel _viewModel = ChatViewModel();
  late ZegoSubtitlesViewModel _subtitlesModel;

  @override
  void initState() {
    super.initState();
    _subtitlesModel = ZegoSubtitlesViewModel();
    ZegoExpressEngine.onRecvExperimentalAPI = _onRecvExperimentalAPI;
    ZegoSubtitlesMessageDispatcher().registerEventHandler(this);
  }

  void _onRecvExperimentalAPI(String content) {
    ZegoSubtitlesMessageDispatcher.handleExpressExperimentalAPIContent(content);
  }

  @override
  void onRecvAsrChatMsg(ZegoSubtitlesMessageProtocol message) {
    _subtitlesModel.handleRecvAsrMessage(message);
  }

  @override
  void onRecvLLMChatMsg(ZegoSubtitlesMessageProtocol message) {
    _subtitlesModel.handleRecvLLMMessage(message);
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: SafeArea(
        child: ListenableBuilder(
          listenable: _viewModel,
          builder: (context, _) {
            return Column(
              children: [
                _buildControlPanel(),
                const Divider(height: 1),
                Expanded(child: ZegoSubtitlesView(model: _subtitlesModel)),
              ],
            );
          },
        ),
      ),
    );
  }

  Widget _buildControlPanel() {
    return Container(
      padding: const EdgeInsets.symmetric(vertical: 40, horizontal: 20),
      color: Colors.grey[100],
      child: Column(
        children: [
          const Text('ZEGO AI Agent',
              style: TextStyle(fontSize: 24, fontWeight: FontWeight.bold)),
          const SizedBox(height: 20),
          // Status indicator
          Row(
            mainAxisAlignment: MainAxisAlignment.center,
            children: [
              Container(
                width: 12, height: 12,
                decoration: BoxDecoration(
                  shape: BoxShape.circle,
                  color: _viewModel.isConnected ? Colors.green : Colors.grey,
                ),
              ),
              const SizedBox(width: 8),
              Text(_viewModel.statusText,
                  style: TextStyle(
                    fontSize: 16,
                    color: _viewModel.isConnected ? Colors.green : Colors.grey,
                  )),
            ],
          ),
          const SizedBox(height: 20),
          // Call button
          SizedBox(
            width: 120, height: 120,
            child: ElevatedButton(
              onPressed: _viewModel.isLoading
                  ? null
                  : (_viewModel.isConnected ? _viewModel.endCall : _viewModel.startCall),
              style: ElevatedButton.styleFrom(
                shape: const CircleBorder(),
                backgroundColor: _viewModel.isConnected ? Colors.red : Colors.green,
                foregroundColor: Colors.white,
              ),
              child: _viewModel.isLoading
                  ? const CircularProgressIndicator(color: Colors.white)
                  : Icon(_viewModel.isConnected ? Icons.call_end : Icons.call, size: 48),
            ),
          ),
        ],
      ),
    );
  }
}

2.10 App Entry Point

// lib/main.dart
import 'package:flutter/material.dart';
import 'package:permission_handler/permission_handler.dart';
import 'widgets/chat_page.dart';

void main() => runApp(const MyApp());

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'AI Agent Demo',
      debugShowCheckedModeBanner: false,
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.blue),
        useMaterial3: true,
      ),
      home: const PermissionWrapper(),
    );
  }
}

// Permission handling wrapper
class PermissionWrapper extends StatefulWidget {
  const PermissionWrapper({super.key});

  @override
  State<PermissionWrapper> createState() => _PermissionWrapperState();
}

class _PermissionWrapperState extends State<PermissionWrapper> {
  bool _permissionGranted = false;
  bool _permissionChecked = false;

  @override
  void initState() {
    super.initState();
    _checkPermission();
  }

  Future<void> _checkPermission() async {
    final status = await Permission.microphone.status;
    if (status.isGranted) {
      setState(() {
        _permissionGranted = true;
        _permissionChecked = true;
      });
    } else {
      final result = await Permission.microphone.request();
      setState(() {
        _permissionGranted = result.isGranted;
        _permissionChecked = true;
      });
    }
  }

  @override
  Widget build(BuildContext context) {
    if (!_permissionChecked) {
      return const Scaffold(
        body: Center(child: CircularProgressIndicator()),
      );
    }

    if (!_permissionGranted) {
      return Scaffold(
        body: Center(
          child: Column(
            mainAxisAlignment: MainAxisAlignment.center,
            children: [
              const Icon(Icons.mic_off, size: 64, color: Colors.grey),
              const SizedBox(height: 16),
              const Text('Microphone permission required',
                  style: TextStyle(fontSize: 18)),
              const SizedBox(height: 16),
              ElevatedButton(
                onPressed: () async {
                  await openAppSettings();
                  _checkPermission();
                },
                child: const Text('Open Settings'),
              ),
            ],
          ),
        ),
      );
    }

    return const ChatPage();
  }
}

---

Run the Application

Android:

flutter run

iOS:

flutter run -d ios

---

Conclusion

You've now built a cross-platform AI chatbot for Android and iOS using Flutter. Here's what you've accomplished:

• Secure token-based authentication
• Real-time voice streaming with ZEGOCLOUD
• AI-powered responses with natural TTS
• Live subtitle display for accessibility
• Single codebase for Android and iOS

Next steps:

• Customize SYSTEM_PROMPT for your brand voice
• Add visual avatars for enhanced UX
• Test on physical devices for optimal audio

The same architecture works for customer service bots, language tutors, AI companions, and more.

Ready to deploy? Start with ZEGOCLOUD and launch your Flutter AI chatbot today.

Questions? Visit the ZEGOCLOUD community for support and inspiration.

youtube link: https://www.youtube.com/shorts/8bwF3l-trEU