Temporal Video Inpainting: Reconstructing Keyframes Occluded by AI Watermarks
Generative video models such as Kling AI produce cinematic fidelity, but embed permanent branding watermarks and timestamps in corner quadrants.
Standard computer vision approaches like single-frame PatchMatch or spatial diffusion inpainting fail on video: they treat each frame independently, creating noticeable temporal flicker, boundary shimmering, and blurring during camera pans.
Why Spatial Inpainting Fails on Dynamic Video
When an object or camera moves across keyframes $t-1$, $t$, and $t+1$, pixels that are occluded by a watermark at time $t$ were almost certainly exposed and visible in adjacent frames due to natural parallax.
Spatial inpainting discards this temporal ground truth and attempts to hallucinate textures from scratch. In contrast, temporal reconstruction computes bidirectional optical flow vectors to project unoccluded background pixels into the occluded region.
Frame (t - 1) [ Background Visible ] ---- Forward Flow Warping ---->
Frame (t) [ Watermark Occlusion ] <-- Optical Flow Blend ---> Restored Frame
Frame (t + 1) [ Background Visible ] <--- Backward Flow Warping ---
Optical Flow Pixel Projection Algorithm
Given current frame $I_t$, reference frame $I_{t-1}$, and binary watermark mask $M$:
- Compute dense forward optical flow $V_{t \to t-1} = \text{Farneback}(I_t, I_{t-1})$.
- Warp unoccluded pixels from $I_{t-1}$ into the coordinate space of $I_t$: $$I_{\text{projected}}(x, y) = I_{t-1}(x + V_x, y + V_y)$$
- Blend the projected pixels using edge-preserving cross-bilateral weights: $$I_{\text{restored}} = (1 - M) \odot I_t + M \odot I_{\text{projected}}$$
Working Python Implementation
import cv2
import numpy as np
def temporal_reconstruct_quadrant(curr_frame, prev_frame, mask):
# Reconstructs occluded pixels using dense optical flow from adjacent temporal frames.
gray_curr = cv2.cvtColor(curr_frame, cv2.COLOR_BGR2GRAY)
gray_prev = cv2.cvtColor(prev_frame, cv2.COLOR_BGR2GRAY)
# Compute dense optical flow
flow = cv2.calcOpticalFlowFarneback(
gray_curr, gray_prev, None,
pyr_scale=0.5, levels=3, winsize=15,
iterations=3, poly_n=5, poly_sigma=1.2, flags=0
)
h, w = curr_frame.shape[:2]
flow_map_x, flow_map_y = np.meshgrid(np.arange(w), np.arange(h))
remap_x = (flow_map_x + flow[:, :, 0]).astype(np.float32)
remap_y = (flow_map_y + flow[:, :, 1]).astype(np.float32)
# Warp reference frame pixels into current frame coordinates
warped_prev = cv2.remap(prev_frame, remap_x, remap_y, cv2.INTER_LINEAR)
# Binary mask normalized
norm_mask = (mask > 0).astype(np.float32)[:, :, np.newaxis]
# Composite reconstruction
restored = (curr_frame * (1.0 - norm_mask) + warped_prev * norm_mask).astype(np.uint8)
return restored
Practical Performance Considerations
- Scene Boundary Detection: Always compute color histogram differences before temporal warping; if a shot cut occurs between $t$ and $t-1$, fall back to spatial Telea inpainting.
- Latency Optimization: Restricting optical flow calculation strictly to the bounding box of the watermark quadrant (e.g. bottom-right $256 \times 128$ window) accelerates throughput to over 60 FPS on standard CPUs.
For free online watermark removal without local GPU requirements, check out the Analytix9 Kling AI Watermark Remover.
Top comments (0)