DEV Community

Mohamed Heni
Mohamed Heni

Posted on

Building an AI-Powered Short Video Automation Platform with Flask, React Native, and Playwright

#ai #automation #python #showdev

TL;DR: I built TikTonik — an AI platform that detects trending content, generates scripts, renders videos with FFMPEG, and publishes them to TikTok — all without human intervention. Here's the architecture and what I learned building it.

The Problem

Short-form video is the most effective content format right now, but producing it at scale is brutally manual:

  • Find a trend → Write a script → Record voiceover → Find B-roll → Edit → Render → Upload
  • Each video takes 1-3 hours for a decent output
  • Most creators burn out after maintaining this cadence for a few weeks
  • "AI video tools" are mostly wrappers that still require manual steps

I wanted to build a system that could go from "trend detected" to "video published" with zero human touching the pipeline.

The Architecture

TikTonik is organized as a multi-application workspace with four major components:

└─ LibTikTonk V2.2 (Backend) ── Flask API + Queue System + ThreadPoolExecutor
    ├─ AI Trend Pipeline  (LLM + trend analysis)
    ├─ FFMPEG/MoviePy   (video rendering)
    └─ Playwright Uploader (TikTok auth + post)

└─ frontend-window (Tauri Desktop)  ── Vite + React + Rust
└─ FrontendMobile (React Native/Expo)  ── iOS + Android
Enter fullscreen mode Exit fullscreen mode

1. Backend (LibTikTonkV2.2)

The core is a Flask application with a threaded task queue. When you submit a job:

  1. Trend Detection — LLM scans trends and generates video concepts
  2. Script Generation — AI writes a short-form script
  3. Voice Synthesis — Kokoro TTS converts script to voiceover
  4. Video Rendering — FFMPEG + MoviePy composites the final video
  5. Upload — Playwright automates TikTok upload with session cookies 
# Worker thread architecture
worker_thread = threading.Thread(target=queue_worker, daemon=True)
worker_thread.start()

# ThreadPoolExecutor for parallel video processing
def process_task(task):
    if task['type'] == 'create_video':
        result = process_create_video_task(task)
    elif task['type'] == 'upload':
        result = process_upload_task(task)
Enter fullscreen mode Exit fullscreen mode

2. Desktop Dashboard (Tauri + React)

Built with Vite + React + Tauri. Tauri wraps the web app as a native desktop app using Rust. Operators can view the job queue, configure AI limits, and monitor upload history — all from a native window with system tray integration.

3. Mobile Clients (React Native / Expo)

Two mobile apps using React Native (Expo):

  • TikTonk — Full client for viewing scheduled jobs, monitoring queue state, uploading from mobile
  • TickTonik 2.0 — Lightweight monitoring for quick status checks

Both connect via REST API to the Flask backend with Appwrite cloud sync.

The Hard Parts

Session-based TikTok Upload

TikTok's API is restricted, so the uploader uses Playwright browser automation with cached session cookies:

browser = playwright.chromium.launch(headless=True)
context = browser.new_context(storage_state="CookiesDir/tiktok_session.json")
page = context.new_page()
page.goto("https://www.tiktok.com/upload/")
Enter fullscreen mode Exit fullscreen mode

The challenge: TikTok changes its DOM structure frequently. The upload selector that worked last week breaks this week. I solved this by maintaining a cookie cache that preserves sessions between uploads, minimizing re-login frequency.

Queue-Based Parallel Processing 

task_queue = queue.Queue()
task_status = {}
task_lock = threading.Lock()

with task_lock:
    task_status[task_id] = {'status': 'processing'}

executor.submit(process_task, task)
Enter fullscreen mode Exit fullscreen mode

The lock prevents status update races. ThreadPoolExecutor handles parallel execution with configurable MAX_WORKERS.

Cross-Platform Startup 

python_cmd = "python3" if sys.platform.startswith("linux") else sys.executable
backend = subprocess.Popen([python_cmd, "main.py"], cwd=backend_dir)
frontend = subprocess.Popen(["npm", "run", "dev"], cwd=frontend_dir)
Enter fullscreen mode Exit fullscreen mode

Graceful shutdown with signal handlers ensures both processes clean up properly.

What I'd Do Differently

  1. Redis queue instead of in-memory for persistence across restarts
  2. Multi-tenancy — currently single-tenant, needs user isolation for SaaS
  3. Better error recovery — cleanup works but task states could be more descriptive
  4. Direct API upload — TikTok Business API would be more reliable than Playwright

Key Takeaways

  • Flask + ThreadPoolExecutor is capable for queue-based workloads. You don't always need Celery.
  • Playwright + session cookies works for platforms without APIs, but plan for DOM changes
  • Four-platform deployment (Flask API + Tauri Desktop + React Native iOS + Android) from one codebase is viable with shared API contracts
  • Video rendering is the bottleneck — FFMPEG is CPU-bound. GPU acceleration would be a major upgrade

Tech Stack

  • Backend: Python, Flask, Flask-Limiter, ThreadPoolExecutor
  • Desktop: Vite, React, Tauri (Rust)
  • Mobile: React Native (Expo)
  • AI: LLM script generation, Kokoro TTS, trend analysis
  • Automation: Playwright
  • Rendering: FFMPEG, MoviePy
  • Sync: Appwrite
  • Deployment: Docker

Comments / Discussion

Have you built an automated content pipeline? What's your approach to handling platform-specific upload APIs versus browser automation? I'd love to hear how others solve the "last mile" problem.

Top comments (0)