If you pause a video, you see a complete, sharp image.
It’s easy to assume that the video file contains that image somewhere as a complete picture.
Usually, it doesn't.
A compressed video is much closer to a collection of instructions for reconstructing frames than it is to a folder full of photographs. Once you understand that, a few annoying video behaviors suddenly make a lot more sense:
- Why scrubbing sometimes jumps slightly away from the exact position
- Why an extracted frame can look worse than the frame you saw during playback
- Why cutting one second from a video can require re-encoding the entire thing
- Why some trims are instant and lossless while others take much longer
The interesting part is that all of these behaviors come from basically the same mechanism.
A video isn't really a stack of photos
Raw camera footage is, in a sense, a sequence of images.
But storing every frame as a completely independent image would make compressed video absurdly large.
Imagine a static shot where almost nothing changes between frames. If the camera records 30 frames per second, there is very little reason to store the entire background 30 times.
Instead, modern video codecs such as H.264, HEVC, and AV1 take advantage of the similarity between neighboring frames.
Some frames contain a complete image.
Others mostly describe how the image has changed.
And that's where I-frames, P-frames, and B-frames come in.
The three main frame types
I-frames
An I-frame, or intra-coded frame, is a complete image that can be decoded independently.
It's the closest thing to a traditional photograph inside a compressed video.
If a decoder has an I-frame, it has enough information to reconstruct that frame without needing another frame first.
The downside is that I-frames are relatively expensive to store.
That's why a video doesn't normally contain a complete I-frame for every single frame.
P-frames
P-frames are predicted frames.
Instead of storing another complete image, a P-frame describes changes relative to previously decoded reference frames.
For a mostly static scene, that can save a huge amount of data.
Suppose someone is sitting in front of a stationary background. Between two frames, the wall, desk, and most of the person's body might be almost identical.
There's no reason to encode all of that information again.
The encoder can describe the changes instead.
This is why most frames in a typical compressed video aren't self-contained pictures.
B-frames
B-frames take prediction a step further.
They can use information from frames before and after them.
The "after" part sounds strange at first. How can a frame refer to something that hasn't happened yet?
The encoder and decoder don't necessarily process frames in the same order that you watch them.
The encoder can arrange the necessary reference frames so that the decoder has the information it needs when reconstructing the final playback sequence.
B-frames can therefore compress very efficiently because they have more information available for prediction.
You normally never notice any of this while watching a video.
Your player handles the reconstruction automatically.
It becomes interesting when you stop doing normal playback.
This is where the GOP comes in
These frames aren't just thrown into the file randomly.
They're organized into groups called GOPs — Groups of Pictures.
A simplified GOP might look something like this:
I B B P B B P B B P I
The exact structure varies by codec and encoder, but the important idea is that an I-frame provides a new independent starting point, followed by frames that can depend on other frames.
GOP length is a trade-off.
More I-frames generally means more independently stored pictures and therefore more data.
Longer GOPs can improve compression efficiency because the encoder doesn't have to store as many expensive I-frames.
But longer GOPs also make certain operations less convenient.
And that's when things like seeking and editing start getting weird.
Why does video seeking sometimes feel inaccurate?
Say you drag the timeline to exactly 01:23.450.
You expect the player to show you precisely that frame.
But that frame might be a P-frame or B-frame.
It can't necessarily be decoded by itself.
The player may need to find an earlier I-frame and then decode all the necessary frames between that keyframe and your requested position.
Conceptually:
I → B → B → P → B → P → B → B → [your frame]
The player starts from the I-frame and reconstructs forward until it reaches the requested frame.
That's computational work.
Some software prioritizes accuracy and does this reconstruction.
Other implementations may prioritize speed and seek to a nearby keyframe first, which can make the scrubber appear to land slightly before or after where you expected.
So when a video player doesn't seem perfectly precise while scrubbing, it isn't necessarily broken.
It may simply be dealing with the way the video was encoded.
Why can an extracted frame look worse than the video?
This one is particularly unintuitive.
You pause a video and see a fairly sharp image.
Then you extract that exact frame as a JPEG or PNG.
And somehow the exported still looks softer or more blocky.
What happened?
The frame you extracted might have been a predicted frame.
During playback, that frame exists as part of a rapidly changing sequence. Your visual system doesn't spend much time inspecting the individual frame.
But when you turn it into a standalone image, all of its compression artifacts become much easier to notice.
Fast motion can make this even more obvious.
A predicted frame following a large amount of movement has more changes to encode. The encoder can compress those changes aggressively because, during normal playback, the viewer isn't expected to stop and inspect that single frame.
Once you extract it as a still, that assumption disappears.
Now you're staring directly at the frame.
This is one reason why moving the extraction point forward or backward by a few frames can sometimes produce a noticeably better still.
You're not necessarily finding a "better moment."
You're simply landing on a frame that happened to survive compression a little better.
The surprising part: trimming a video
Here's where this frame structure becomes much more important.
Suppose you have a ten-minute video and want to remove the first second.
It sounds like a trivial operation.
Why should an editor have to do anything complicated? Just remove the first second and keep everything else, right?
The problem is that your cut might happen in the middle of a GOP.
Imagine the beginning of the remaining video looks like this:
I → B → B → P → B → P → B → ...
↑
cut here
If the first frame you're keeping depends on information from a frame you just removed, that frame is no longer valid on its own.
The editor can't simply throw away the first second and copy the remaining compressed data unchanged.
It needs to create a new valid sequence.
That usually means:
- Decode the relevant video data
- Apply the cut
- Encode the resulting frames again
- Create new reference frames and predictions
And that's why a seemingly tiny edit can trigger a full export.
But sometimes trimming is instant
There is an important exception.
If your cut happens to land exactly on an existing I-frame, the remaining video may already have a valid independent starting point.
In that case, software can sometimes perform a stream copy or remux instead of re-encoding the video.
The actual picture data doesn't need to change.
The software can essentially rearrange the existing encoded streams inside a new container.
That's extremely fast and doesn't introduce another generation of lossy compression.
The catch is that you don't get to choose arbitrary frame-accurate cut points when doing this.
If the cut falls between keyframes, you generally lose that advantage.
This is why two apparently identical "trim this video" operations can behave completely differently.
The difference may simply be where the keyframes happen to be.
One simple way to think about it
A useful mental model is to imagine a video like this:
[I] [changes] [changes] [changes] [I] [changes] [changes] [changes] [I]
The I-frames are complete starting points.
The other frames are instructions that depend on information around them.
During playback, your video player quietly reconstructs everything.
During editing, you start interacting with those dependencies.
That's why:
| What you see | What's happening |
|---|---|
| Scrubbing isn't perfectly precise | The player may need to seek from a nearby keyframe |
| An extracted frame looks rough | The frame may be heavily compressed or predicted |
| A tiny trim takes a long time | The cut may fall inside a GOP and require re-encoding |
| Some trims are nearly instant | The cut may align with existing keyframes |
| Motion-heavy frames look worse as stills | More changes have to be represented in the predicted frame |
Once you know about GOPs, these behaviors stop looking random.
Does a longer GOP mean worse video quality?
Not necessarily.
A longer GOP doesn't automatically mean lower visual quality.
It's primarily a compression and access trade-off.
Fewer I-frames can reduce the amount of data required, while predicted frames take advantage of similarities between neighboring frames.
The downside is that random access becomes more complicated, because there are fewer independent starting points.
This matters for things like:
- Video editing
- Seeking
- Streaming
- Frame extraction
- Scrubbing
- Fast random access
For ordinary playback, you might never notice the difference.
Why codecs are designed this way
It can be tempting to look at all of this and think: wouldn't it be easier if every frame were just a complete image?
Yes — but the resulting files would be enormous.
The whole point of inter-frame compression is that consecutive frames are usually highly redundant.
A stationary background doesn't need to be encoded from scratch 30 times every second.
A talking head doesn't require the entire scene to be completely re-described for every frame.
Video compression takes advantage of that redundancy.
The price is that not every frame is independently useful.
That's not a flaw.
It's the fundamental trade-off that makes compressed video practical.
The useful takeaway
You don't need to understand GOP structures to watch YouTube or record a video on your phone.
But once you start extracting frames or editing video, this little bit of codec knowledge becomes surprisingly useful.
If a frame looks worse than expected, try a few nearby frames.
If a simple trim takes much longer than expected, the cut probably isn't landing on a convenient keyframe.
And if one video can be trimmed almost instantly while another takes ages to export, the difference may have nothing to do with the length of the video at all.
It can come down to something you normally never see:
where the keyframes are.
That's one of the interesting things about compressed video. What looks like a simple sequence of pictures on screen is actually a carefully constructed dependency graph underneath.
Top comments (0)