DEV Community

wintrover
wintrover

Posted on • Originally published at wintrover.github.io

Building a KYC Video Recording and Splitting System: React + FFmpeg + MediaRecorder API

Project Overview

I developed a system that records the entire user face recognition process during KYC (Know Your Customer) procedures and splits it into step-by-step segments for storage. This project utilized React, TypeScript, FFmpeg, and MediaRecorder API as core technologies, solving the challenging task of real-time video processing in web environments.

🎯 Key Feature Implementation

System Architecture Overview

1. Video Recording System

I implemented a system that records the entire process from KYC mission start to completion using the MediaRecorder API.

// Start video recording
const startRecording = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
  const mediaRecorder = new MediaRecorder(stream);

  mediaRecorder.ondataavailable = (event) => {
    if (event.data.size > 0) {
      recordedChunks.push(event.data);
    }
  };

  mediaRecorder.start();
};
Enter fullscreen mode Exit fullscreen mode

I designed a timeline data structure to accurately record start/end timestamps for each KYC step:

interface KYCTimeline {
  stepIndex: number;
  stepName: string;
  startTime: number;
  endTime: number;
  duration: number;
}
Enter fullscreen mode Exit fullscreen mode

2. Setting up FFmpeg in Web Environment

I solved several technical challenges to use FFmpeg in web browsers.

Cross-Origin Header Configuration

Essential header configuration for SharedArrayBuffer usage:

// vite.config.ts
export default defineConfig({
  server: {
    headers: {
      'Cross-Origin-Opener-Policy': 'same-origin',
      'Cross-Origin-Embedder-Policy': 'require-corp'
    }
  }
});
Enter fullscreen mode Exit fullscreen mode

FFmpeg Initialization 

import { FFmpeg } from '@ffmpeg/ffmpeg';
import { toBlobURL } from '@ffmpeg/util';

const ffmpeg = new FFmpeg();

const loadFFmpeg = async () => {
  const baseURL = 'https://unpkg.com/@ffmpeg/core@0.12.10/dist/esm';

  await ffmpeg.load({
    coreURL: await toBlobURL(`${baseURL}/ffmpeg-core.js`, 'text/javascript'),
    wasmURL: await toBlobURL(`${baseURL}/ffmpeg-core.wasm`, 'application/wasm')
  });
};
Enter fullscreen mode Exit fullscreen mode

3. Video Splitting and Processing Pipeline

I implemented a system that splits videos step-by-step based on timeline data.

Video Processing Flow 

const splitVideo = async (videoBlob: Blob, timeline: KYCTimeline[]) => {
  // Write video to FFmpeg virtual file system
  await ffmpeg.writeFile('input.webm', await fetchFile(videoBlob));

  const splitVideos = [];

  for (const [index, step] of timeline.entries()) {
    const startTime = formatTime(step.startTime);
    const duration = formatTime(step.duration);
    const outputName = `step${index}_${step.stepName}.webm`;

    // Split video with FFmpeg command
    await ffmpeg.exec([
      '-i', 'input.webm',
      '-ss', startTime,
      '-t', duration,
      '-c', 'copy',
      outputName
    ]);

    // Read split video
    const data = await ffmpeg.readFile(outputName);
    const blob = new Blob([data], { type: 'video/webm' });
    splitVideos.push({ name: outputName, blob });
  }

  return splitVideos;
};
Enter fullscreen mode Exit fullscreen mode

Time Format Conversion Utility 

const formatTime = (milliseconds: number): string => {
  const totalSeconds = Math.floor(milliseconds / 1000);
  const hours = Math.floor(totalSeconds / 3600);
  const minutes = Math.floor((totalSeconds % 3600) / 60);
  const seconds = totalSeconds % 60;
  const ms = milliseconds % 1000;

  return `${hours.toString().padStart(2, '0')}:${minutes.toString().padStart(2, '0')}:${seconds.toString().padStart(2, '0')}.${ms.toString().padStart(3, '0')}`;
};
Enter fullscreen mode Exit fullscreen mode

🔧 Major Trial and Error Process

Cross-Origin Policy and Security Header Relationship

1. OpenCV.js COEP Cross-Origin Issue

Problem: OpenCV.js CDN loading failed with net::ERR_BLOCKED_BY_RESPONSE.NotSameOriginAfterDefaultedToSameOriginByCoep error

Solution: Adopted local hosting approach instead of CDN

// Previous CDN approach (failed)
// <script src="https://docs.opencv.org/4.x/opencv.js"></script>

// Local hosting approach (success)
// After downloading public/opencv.js file
<script src="/opencv.js"></script>
Enter fullscreen mode Exit fullscreen mode

2. Preventing Duplicate Video Uploads

Problem: Same video being uploaded multiple times

Solution: Preventing duplicates through state management and reference comparison

const [isVideoUploaded, setIsVideoUploaded] = useState(false);
const uploadedBlobRef = useRef<Blob | null>(null);

const uploadVideo = async (videoBlob: Blob) => {
  // Prevent duplicate uploads
  if (isVideoUploaded && uploadedBlobRef.current === videoBlob) {
    return;
  }

  try {
    const formData = new FormData();
    formData.append('video', videoBlob, 'kyc-recording.webm');

    await fetch('/api/upload-video', {
      method: 'POST',
      body: formData
    });

    setIsVideoUploaded(true);
    uploadedBlobRef.current = videoBlob;
  } catch (error) {
    console.error('Video upload failed:', error);
  }
};
Enter fullscreen mode Exit fullscreen mode

3. Memory Management and Performance Optimization

Problem: Memory shortage and performance degradation when processing large videos

Solution: Proper resource management and cleanup

const processVideo = async (videoBlob: Blob) => {
  try {
    // Video processing logic
    const splitVideos = await splitVideo(videoBlob, timeline);

    // Upload split videos
    const uploadResults = await Promise.allSettled(
      splitVideos.map(video => uploadSplitVideo(video))
    );

    // Count success/failure
    const successful = uploadResults.filter(result => result.status === 'fulfilled').length;
    const failed = uploadResults.filter(result => result.status === 'rejected').length;

    if (failed > 0) {
      console.error(`Video upload failed: ${failed}, successful: ${successful}`);
    }
  } finally {
    // Memory cleanup
    await cleanupFFmpegFiles();
  }
};

const cleanupFFmpegFiles = async () => {
  try {
    const files = ['input.webm', ...timeline.map((_, i) => `step${i}_output.webm`)];
    for (const file of files) {
      try {
        await ffmpeg.deleteFile(file);
      } catch {
        // Ignore if file doesn't exist
      }
    }
  } catch (error) {
    console.error('FFmpeg file cleanup failed:', error);
  }
};
Enter fullscreen mode Exit fullscreen mode

🚀 Technology Stack and Architecture

Technology Stack Diagram

Frontend Technology Stack

  • React 19 + TypeScript: Component-based UI development
  • @tanstack/react-query: Server state management
  • @tanstack/react-router: Routing
  • Styled Components: CSS-in-JS styling
  • TailwindCSS: Utility-based styling

Video Processing Technologies

  • MediaRecorder API: Real-time video recording
  • FFmpeg.wasm: Video processing in web environment
  • OpenCV.js: Computer vision processing
  • MediaPipe: Face recognition and tracking

Development Tools

  • Vite: Build tool and development server
  • Jest: Unit testing
  • SonarQube: Code quality analysis
  • Sentry: Error monitoring

📈 Achievements and Learning Points

Development Process and Performance Metrics

Achievements

  1. Real-time video processing system in web environment completed
  2. Client-side video splitting using FFmpeg.wasm implemented
  3. Cross-Origin policy and SharedArrayBuffer related issues resolved
  4. Memory-efficient large file processing logic implemented

Key Learning Points

  1. Web Security Policies: Importance of COEP, COOP headers and external resource loading constraints
  2. WebAssembly Utilization: Web environment implementation for tasks requiring native performance
  3. Asynchronous Processing Optimization: Stable parallel processing using Promise.allSettled
  4. Resource Management: Memory leak prevention and proper cleanup in web environments

🔮 Future Improvement Directions

  1. Web Workers Utilization: Background processing to prevent main thread blocking
  2. Progressive Upload: Chunk-based upload for large files
  3. Real-time Compression: File size optimization through real-time video compression during recording
  4. Offline Support: Network instability handling using Service Workers

Through this project, I was able to acquire advanced video processing techniques in web environments, and gained deep understanding of browser security policies and WebAssembly utilization. I look forward to continuing to challenge myself with projects that push the boundaries of web technology.

Top comments (0)