Handling Fast Forwarding and Rewinding in Our Video App

Brian Twene — Mon, 07 Oct 2024 12:04:26 +0000

Handling Fast Forwarding and Rewinding in Our App

At Gontrel, we discovered that we needed to implement fast forwarding and rewinding controls for our posts, something that is minimal but still got the job done considering the amount of information we need to put on the screen. We took inspiration from how this was done on TikTok and especially YouTube shorts with the preview window that would allow users to see what part of the video they would be going to.

You can try it out in the live version of the app here

This post will go through how this can be implemented with the following which is part our tech stack:

React: To build dynamic and interactive UI components.
Ionic Framework's: This allows us to build the app for mobile platforms while still being able to use web technologies.
Tailwind CSS: Makes it easy to rapidly build interfaces by keeping you close to the code while styling.

Our Solution

The provided code is a simplified version (to show you how it works at its core) of our actual implementation and may not reflect the exact styling and complexity present in our main application.

Implementation Overview

This is built in a Post component that encapsulates a video player with custom controls, including a progress bar that allows users to seek through the video while displaying a real-time frame preview.

Prerequisites

To get the most out of this tutorial, you should have a basic understanding of React and Typescript. Even if you don't, ah sure its grand! Feel free to tag along 😄

1. Setting Up the Project

Before beginning, make sure that you have Node.js installed on your machine. If not head on to the Offical Node.js website to install it. You can check to make sure its installed corrected by using the following commands:

node -v
npm -v

First, we set up a React project and installed the necessary dependencies:

Scaffolding project with vite

npm create vite@latest video-app -- --template react-ts
cd video-app

Then install other dependencies

npm install @ionic/react react-icons 
# install these as dev dependencies
npm install -D tailwindcss postcss autoprefixer
# initalise tailwindcss
npx tailwindcss init -p

Next, follow the steps here to finish configuring Tailwind CSS by following the official Tailwind CSS installation guide for Vite.

Once that's done, start the dev server...

npm run dev

2. Implementing the Post Component

The Post component is responsible for rendering the video, custom controls (play/pause, mute/unmute), and the progress bar with frame previews. here is the full code for the component

Post.tsx

// Post.tsx
import React, { useEffect, useRef, useState } from 'react';
import { IoPause, IoPlay, IoVolumeHigh, IoVolumeMute } from 'react-icons/io5';
import {
  IonRange,
  RangeChangeEventDetail,
  RangeKnobMoveEndEventDetail,
} from '@ionic/react';

interface PostProps {
  videoSrc: string;
}

export const calculateVideoTime = (
  currentSeekValue: number,
  duration: number
): string => {
  // Calculate the current time based on the seek value
  const currentTime = (currentSeekValue / 100) * duration;

  // Convert duration to minutes and seconds
  const durationMinutes = Math.floor(duration / 60);
  const durationSeconds = Math.floor(duration % 60);
  const formattedDuration = `${durationMinutes}:${
    durationSeconds < 10 ? '0' : ''
  }${durationSeconds}`;

  // Convert current time to minutes and seconds
  const currentMinutes = Math.floor(currentTime / 60);
  const currentSeconds = Math.floor(currentTime % 60);
  const formattedCurrentTime = `${currentMinutes}:${
    currentSeconds < 10 ? '0' : ''
  }${currentSeconds}`;

  // Return both times
  return `${formattedCurrentTime}/${formattedDuration}`;
};

const generatePreview = (video: HTMLVideoElement, time?: number): string => {
  try {
    const canvas = document.createElement('canvas');
    const context = canvas.getContext('2d');

    if (context) {
      canvas.width = video.videoWidth;
      canvas.height = video.videoHeight;
      if (time) video.currentTime = time;
      context.drawImage(video, 0, 0, canvas.width, canvas.height);
      return canvas.toDataURL('image/jpeg');
    }

    return '';
  } catch (e) {
    console.log('error: ', e);
    return '';
  }
};

const Post: React.FC<PostProps> = ({ videoSrc }) => {
  const videoRef = useRef<HTMLVideoElement>(null);
  const seekVideoRef = useRef<HTMLVideoElement>(null);
  const [progress, setProgress] = useState<number>(0);
  const [seekValue, setSeekValue] = useState(0);
  const [framePreview, setFramePreview] = useState<string>('');
  const [isSeeking, setIsSeeking] = useState(false);
  const [mute, setMute] = useState(true);
  const [play, setPlay] = useState(false);

  const togglePlay = async () => {
    const video = videoRef.current;
    if (!video) return;

    if (play) {
      video.pause();
      setPlay(false);
    } else {
      await video.play();
      setPlay(true);
    }
  };

  const onSeekingStop = (event: CustomEvent<RangeKnobMoveEndEventDetail>) => {
    const video = videoRef.current;
    const seekVideo = seekVideoRef.current;
    const value = event.detail.value as number;
    if (video && seekVideo) {
      const newTime = (value / 100) * video.duration;
      video.currentTime = newTime;
      setFramePreview(generatePreview(seekVideo));
    }
    setIsSeeking(false);
  };

  const onSeekingMove = (e: CustomEvent<RangeChangeEventDetail>) => {
    const videoElement = videoRef.current;
    const seekVideoElement = seekVideoRef.current;
    const value = e.detail.value as number;
    setSeekValue(value);

    if (videoElement && seekVideoElement) {
      let seekTime = parseFloat(
        (((e.detail.value as number) / 100) * videoElement.duration).toFixed(2)
      );

      if (seekTime < 0) {
        seekTime = 0;
      }
      setFramePreview(generatePreview(seekVideoElement, seekTime));
    }
  };

  useEffect(() => {
    const videoElement = videoRef.current;

    if (videoElement) {
{/* as the video is playing, update the progress state */}
      const handleTimeUpdate = () => {
        if (!isSeeking) {
          const progress =
            (videoElement.currentTime / videoElement.duration) * 100;
          setProgress(progress);
        }
      };

      videoElement.addEventListener('timeupdate', handleTimeUpdate);

      return () => {
        videoElement.removeEventListener('timeupdate', handleTimeUpdate);
      };
    }
  }, [videoRef, isSeeking]);

  return (
    <div className="h-full flex flex-col">
      <div className="flex-grow relative">
        <video
          className="h-full"
          loop
          ref={videoRef}
          src={videoSrc}
          muted={mute}
        />

        <video
          className="hidden"
          ref={seekVideoRef}
          src={videoSrc}
          muted={mute}
        />

        {!isSeeking && (
          <div className="text-2xl flex gap-4 controls items-center p-2 bg-slate-500 rounded-lg m-4">
            <button onClick={togglePlay}>
              {play && <IoPause />}
              {!play && <IoPlay />}
            </button>

            <button onClick={() => setMute(!mute)}>
              {mute && <IoVolumeMute />}
              {!mute && <IoVolumeHigh />}
            </button>
          </div>
        )}
      </div>
      <div className="mx-4 relative">
        {isSeeking && (
          <div className="flex flex-col items-center gap-3 absolute preview-container">
            <div className="preview w-28 h-16 rounded-lg">
              {framePreview && <img src={framePreview} alt="Video preview" />}
            </div>

            <span className="text-white">
              {videoRef.current &&
                calculateVideoTime(seekValue, videoRef.current.duration)}
            </span>
          </div>
        )}

        <IonRange
          className={isSeeking ? 'activated' : ''}
          min={0}
          max={100}
          value={isSeeking ? seekValue : progress}
          onIonInput={onSeekingMove}
          onIonKnobMoveEnd={onSeekingStop}
          onIonKnobMoveStart={() => setIsSeeking(true)}
        />
      </div>
    </div>
  );
};

export default Post;

App.css

/* App.css */
.controls {
  position: absolute;
  bottom: 10px; /* Adjust based on design */
}

.controls button {
  border-radius: 8px;
  background-color: transparent;
  color: white;
}

.preview-container {
  top: -100px; /* Adjust based on design */
  left: 50%;
  transform: translateX(-50%);
}

.preview {
  border: 1px solid transparent;
  background-clip: padding-box;
  background-image: linear-gradient(
    135deg,
    rgba(255, 255, 255, 0.6),
    rgba(255, 255, 255, 0.2)
  );
}

.preview img {
  max-width: 100%;
  max-height: 100%;
  padding: 1px;
  border-radius: 8px;
}

.activated ion-range {
  --height: 8px;
  --bar-height: 8px;
}

.activated ion-range::part(bar-active) {
  transition: none;
}

ion-range::part(bar-active) {
  transition: right 0.4s ease;
}

.video-feed {
  display: flex;
  flex-direction: column;
  gap: 20px;
}

3. Rendering the Post Component with a Sample Video

Next the Post component is put in a div in the App.tsx. A sample video link is passed to load the video. You can download this video from here and add it to the ./assets folder

App.tsx

// App.tsx
import React from 'react';
import Post from './Post';
import './App.css';
import SampleVideo from "./assets/mov_bbb.mp4"

const App: React.FC = () => {
  return (
    <div className="video-feed">
      <Post videoSrc={SampleVideo} />
    </div>
  );
};

export default App;

4. Understanding the Code

Let's break down the key parts of our implementation:

Generating Frame Previews

To provide real-time frame previews during seeking, a second video element (seekVideoRef) is used. This allows us to capture frames without interrupting the playback of the main video.

const generatePreview = (video: HTMLVideoElement, time?: number): string => {
  try {
    const canvas = document.createElement('canvas');
    const context = canvas.getContext('2d');

    if (context) {
      canvas.width = video.videoWidth;
      canvas.height = video.videoHeight;
      if (time) video.currentTime = time;
      context.drawImage(video, 0, 0, canvas.width, canvas.height);
      return canvas.toDataURL('image/jpeg');
    }

    return '';
  } catch (e) {
    console.log('error: ', e);
    return '';
  }
};

Hidden Video Element: We have a second video element (seekVideoRef) that's hidden from view. When the user seeks, we set the currentTime of this hidden video to the seeking time, capture the frame, and display it as a preview.
Canvas Drawing: Using the Canvas API, we draw the current frame of the hidden video and convert it to a Data URL, which is then displayed as an image.

Handling User Interactions

We manage the seeking state and update the progress accordingly:

Play/Pause Toggle: Users can play or pause the video using the play/pause button.
Mute/Unmute Toggle: Users can mute or unmute the video using the volume button.
Seeking: As users seek through the progress bar, we update the frame preview in real-time.

const onSeekingMove = (e: CustomEvent<RangeChangeEventDetail>) => {
  const videoElement = videoRef.current;
  const seekVideoElement = seekVideoRef.current;
  const value = e.detail.value as number;
  setSeekValue(value);

  if (videoElement && seekVideoElement) {
    let seekTime = parseFloat(
      (((e.detail.value as number) / 100) * videoElement.duration).toFixed(2)
    );

    if (seekTime < 0) {
      seekTime = 0;
    }
    setFramePreview(generatePreview(seekVideoElement, seekTime));
  }
};

onIonInput (onSeekingMove): This event fires continuously as the user drags the progress. We calculate the corresponding time in the video, generate a frame preview, and update the state.
onIonKnobMoveEnd (onSeekingStop): When the user releases the progress bar, we set the video's currentTime to the new position and resume playback.

Displaying the Frame Preview

During seeking, a small preview window appears above the progress bar, showing the frame corresponding to the current seek position.

{isSeeking && (
  <div className="flex flex-col items-center gap-3 absolute preview-container">
    <div className="preview w-28 h-16 rounded-lg">
      {framePreview && <img src={framePreview} alt="Video preview" />}
    </div>

    <span className="text-white">
      {videoRef.current &&
        calculateVideoTime(seekValue, videoRef.current.duration)}
    </span>
  </div>
)}

5. Styling with Tailwind CSS

Some normal CSS used for better readability.

Key Styles

Controls Positioning: Positioned the play/pause and mute/unmute buttons at the bottom of the video.
Preview Container: Positioned above the progress bar, styled with a gradient border and rounded corners.
Progress Bar: Customized the IonRange component to have a thicker progress bar when active and smooth transitions.

/* App.css */
.controls {
  position: absolute;
  bottom: 10px; /* Adjust based on design */
}

.controls button {
  border-radius: 8px;
  background-color: transparent;
  color: white;
}

.preview-container {
  top: -100px; /* Adjust based on design */
  left: 50%;
  transform: translateX(-50%);
}

.preview {
  border: 1px solid transparent;
  background-clip: padding-box;
  background-image: linear-gradient(
    135deg,
    rgba(255, 255, 255, 0.6),
    rgba(255, 255, 255, 0.2)
  );
}

.preview img {
  max-width: 100%;
  max-height: 100%;
  padding: 1px;
  border-radius: 8px;
}

.activated ion-range {
  --height: 8px;
  --bar-height: 8px;
}


.video-feed {
  display: flex;
  flex-direction: column;
  gap: 20px;
}

Check out a demo of it here

Conclusion

Implementing fast forwarding and rewinding with smooth interactions and real-time frame previews significantly enhances the user experience in video-centric applications. By leveraging Ionic's IonRange, React, and Tailwind CSS, we were able to create a responsive and efficient video progress bar that handles user interactions gracefully, while preventing the screen from not being too cluttered

Key Takeaways:

Customizable Progress Bar: Using Ionic's IonRange allows for creating a highly customizable and responsive progress bar.
Real-time Frame Previews: Implementing a hidden video element to capture frame previews provides immediate visual feedback without interrupting the main video playback.

We hope this deep dive into our fast forward and rewind implementation provides valuable insights for your own projects. Feel free to reach out with any questions or feedback!

Happy Coding!

Stay tuned for more insights and updates from our engineering team.

DEV Community: Gontrel