The Problem
Every video editor knows the pain: scrubbing through a 45-minute interview to find the good parts, manually removing "ums" and "uhs," aligning captions frame by frame. It takes 3-4 hours for a 10-minute video.
I wanted to build a tool that lets you edit video by editing text — delete the filler words from a transcript, and the video follows automatically. No cloud uploads, no GPU required, no subscription.
Here's how I built ClipScribe Pro using faster-whisper and FFmpeg.
Architecture Overview
The system has three stages:
- Transcription — Convert audio to timestamped text via faster-whisper
- Text-based editing — Let users delete/rearrange text, track which segments to keep
- Video assembly — Use FFmpeg to cut and concatenate the corresponding video segments
class ClipScribePipeline:
def __init__(self):
self.transcriber = WhisperTranscriber()
self.editor = TextEditor()
self.renderer = VideoRenderer()
def process(self, video_path):
segments = self.transcriber.transcribe(video_path)
edited = self.editor.edit(segments)
output = self.renderer.render(video_path, edited)
return output
Stage 1: Why faster-whisper and Not OpenAI Whisper
OpenAI's whisper is the gold standard. But the original loads the entire model into GPU memory and runs synchronously.
faster-whisper uses CTranslate2:
- Uses int8 quantization (half the memory, similar accuracy)
- Runs 3-4x faster than the original
- Supports CPU-only inference
- Has a batched mode for long audio files
from faster_whisper import WhisperModel
class WhisperTranscriber:
def __init__(self, model_size="base", device="cpu", compute_type="int8"):
self.model = WhisperModel(model_size, device=device, compute_type=compute_type)
def transcribe(self, audio_path, language="en"):
segments, info = self.model.transcribe(audio_path, language=language, beam_size=5, word_timestamps=True, vad_filter=True, vad_parameters=dict(min_silence_duration_ms=500))
result = []
for segment in segments:
words = []
if segment.words:
for word in segment.words:
words.append({"text": word.word, "start": word.start, "end": word.end, "confidence": word.probability})
result.append({"text": segment.text, "start": segment.start, "end": segment.end, "words": words})
return result
Model size comparison:
| Model | Size | Accuracy | Speed (10min) |
|---|---|---|---|
| tiny | 75MB | 74.0% | 45s |
| base | 150MB | 78.0% | 1m 20s |
| small | 500MB | 83.0% | 2m 30s |
| medium | 1.5GB | 86.0% | 4m 10s |
| large-v3 | 3.1GB | 89.0% | 7m 30s |
Stage 2: Text-Based Editing
Let users edit the transcript and track which video parts to keep.
class SegmentEditor:
def __init__(self, segments):
self.segments = segments
self.keep = [True] * len(segments)
def delete_filler_words(self, filler_words=None):
if filler_words is None:
filler_words = ["um", "uh", "ah", "like", "you know"]
for i, seg in enumerate(self.segments):
words = seg.get("words", [])
for w in words:
if w["text"].strip().lower() in filler_words:
w["remove"] = True
def get_active_segments(self):
return [seg for i, seg in enumerate(self.segments) if self.keep[i]]
Stage 3: Video Rendering with FFmpeg
class VideoRenderer:
def render(self, original_video, keep_segments, output_path):
cmd = ["ffmpeg", "-y", "-f", "concat", "-safe", "0", "-i", concat_file, "-c", "copy", output_path]
subprocess.run(cmd, check=True, capture_output=True)
return output_path
Using -c copy = no re-encoding = instant cuts, zero quality loss.
Auto-Caption Generation
class CaptionGenerator:
def generate_srt(self, segments, output_path="captions.srt"):
lines = []
idx = 1
for seg in segments:
words = seg.get("words", [])
if not words:
continue
chunks = self._chunk_words(words, 5)
for chunk in chunks:
start = self._format_time(chunk[0]["start"])
end = self._format_time(chunk[-1]["end"])
text = " ".join(w["text"].strip() for w in chunk)
lines.append(str(idx))
lines.append(f"{start} --> {end}")
lines.append(text)
lines.append("")
idx += 1
with open(output_path, "w", encoding="utf-8") as f:
f.write("\n".join(lines))
return output_path
def _format_time(self, seconds):
ms = int((seconds - int(seconds)) * 1000)
h = int(seconds // 3600)
m = int((seconds % 3600) // 60)
s = int(seconds % 60)
return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"
Putting It All Together
Desktop app using tkinter:
python
class ClipScribeApp:
def __init__(self):
self.root = tk.Tk()
self.root.title("ClipScribe Pro")
self.transcriber = WhisperTranscriber(model_size="small", device="cpu", compute_type="int8")
self.renderer = VideoRenderer()
self._build_ui()
def _build_ui(self):
ttk.Button(self.root, text="Open Video", command=self.open_video).pack(pady=10)
self.transcript_text = tk.Text(self.root, height=20, width=80)
self.transcript_text.pack(padx=10, pady=5)
ttk.Button(self.root, text="Remove Filler Words", command=self.clean_filler).pack(pady=5)
ttk.Button(self.root, text="Export Video", command=self.export).pack(pady=10)
## Performance Results
| Metric | Before (Manual) | After (ClipScribe) |
|--------|----------------|-------------------|
| Filler word cleanup | 20-30 min | 2-3 min |
| Silence removal | 10-15 min | Automatic (30s) |
| Caption generation | 15-20 min | 30 seconds |
| Total time (10-min video) | ~3 hours | ~45 minutes |
## Key Takeaways
1. **faster-whisper with int8 is the sweet spot** for CPU transcription
2. **Frame-accurate cuts with no re-encoding** using FFmpeg concat demuxer
3. **Text-based editing** reduces cognitive load enormously
4. **VAD filter is essential** for removing silence
## Try It Yourself
The complete ClipScribe Pro source is included with the desktop app:
[Download ClipScribe Pro](https://aixhdd.com/product/clipscribe-pro/) — $29 lifetime license, runs entirely offline, no GPU required.
Originally published at [AIXHDD.com](https://aixhdd.com/product/clipscribe-pro/)
Top comments (0)