How bad is my video? A video quality artifacts walk-through
Video compression has been a hot topic for many years and now with the multiplication of content providers (read Netflix, Disney+ or others) being able to deliver high quality content at medium/low bandwidth is getting increasingly important. After all, we pay for a service and expect to get our 4Kmovies on the train right?
But as anything in life, everything comes at a cost. To be able to deliver content we need to be able to compress it, and to compress it, at some point, we usually need to drop data, and with the drop of data, the quality drops.
So how do the big guys decide how much should be dropped? A very good source of information (which I recommend to follow) has been the Netflix Tech blog.
In this (rather short) article I will go through some of the most common artifacts we could see in videos. I will complete later on with an overview of objective metrics (read without the need for human intervention) available out there to try to detect those (and actually used out there by the big guys).
What is a video artifact?
A video artifact is a noticeable distortion (for the viewer) in the quality of a video, usually due to image capture limitations or to lossy data compression (where we drop information).
In the part below I will list the most common artifacts and explain briefly some possible causes. Some are usually compression related (blocking, banding, ringing), capture related (noise, aliasing, rolling-shutter) or transmission related (ghosting, freezing, data-error)
Which artifacts do we usually see?
Noise
By far the most common artifact. Image noise is random variation of brightness (luminance) or color (chrominance) information in images. It is is usually due to electronic noise (from the image capture or transmission). Image noise can range from almost imperceptible (usually on photograph taken in good light), to radioastronomical images that are almost entirely noise, from which a small amount of information can be derived by sophisticated processing. If you want to know more, check out this Wikipedia article.
In the image below noise is the small colored dots present in the image.
Blocking
The second most common, it usually appears in videos with higher compression settings and is a consequence of block-oriented compression where we drop a little too much data. We can clearly see the blocks used during the compression process in the image below. These artifacts happen because during the compression process, the image is split into squares which are compressed (relatively) independently, the big drop in information generates big differences between these blocks which are then visible in the end result.
Some de-blocking filtering can be applied to reduce this effect but usually adds blurriness to the end result.
Banding / posterization
Banding is a subtle result of posterization, where the conversion of a continuous gradation of tone to several regions of fewer tones, causes abrupt changes from one tone to another. It may be a deliberate or an unintended artifact of color quantization (where we do not have enough possible values to represent the fine differences in colors and end up with “jumps” in color becuase of that). We usually see those during nice sunset or dark scenes, most often because the video is encoded with 8 bits and a slight change in color could not be represented correctly. Banding can be seen in the image below in the sky around the sun.
Ringing
Ringing is somewhat a blanket term for edge artifacts. It is usually induced by some image enhancement filters. Ringing includes mosquito noise (shimmering blur of dots around edges), edge enhancement artifacts (strong contrast between edge colors), overshoot, or ring-like ringing caused by the Gibbs phenomenon. In the picture below we can see the mosquito effecy shown with the white arrows as well as some edge enhancement artifacts around the black curls. The image after shows ringing effect from edge enhancement.
Aliasing
It is an effect that causes different signals to become indistinguishable (or aliases of one another) when sampled. It also often refers to the distortion or artifact that results when a signal reconstructed from samples is different from the original continuous signal. With videos it often manifests as Moiré patterns. The Moiré pattern often arises because the image contains repetitive detail which exceeds the resolution of the camera (see the door below).
Another type of aliasing could be seen when the encoding process adds new textures which are not present in the original image (as the grass below)
or even the stroboscopic effect (fun aliasing occurring when continuous cyclic motion is represented by a series of short or instantaneous samples) which can lead to fun effects like non-propelled flying helicopters:
Ghosting
This appends when a replica of a transmitted image, offset in position, is visible in an other image. It is often caused when a TV signal travels by two different paths to a receiving antenna, with a slight difference in timing or when temporal filters (often for noise reduction) keep multiple replicas of an object in the image. You can see it below with the duplicated player.
Interlace artifacts
These are artifacts created by video interlacing. When watching interlaced video on a progressive monitor with without deinterlacing, they can see sort of combing in movement between two fields of one frame.
Block loss artifacts
Data errors in the compressed bit-stream, possibly due to transmission errors appear sometime due to missing or corrupted data preventing the decoder to properly reconstruct the image. The way it shows is different between decoders are most have some kind of error-correction (where the error is compensated for using surrounding or extra data) or error-concealment mechanism (where the error is hidden by replacing the corrupted data).
It often shows with blocks on the image (as below) or replacing the coirrupted blocks with colored blocks.
Freezing
This happens when some images in a video are missing/corrupted/delayed and creates a gap in the sequence of images. Again here the way this is experienced depends on the decoder as this could be compensated using interpolation or displaying a previous frame longer than needed on the screen. Some decoders display a black or gray image instead of the missing frame. The video below contains both freezing and ghosting.
Other (not-really) artifacts
Below are some other indicator which are not really artifacts but give a good indication of a video quality and detecting these could help:
Rolling shutter
These are pretty rare and are due to limitation of the readout-rate of image sensors which end up in a distorted image. This is the because the pixels in active-pixel sensors are read line by line and by the time a line is completed the object in the next one has moved, resulting in a distorted view of the object. It can be seen in the helicopter image or the distorted train on the left of the next image.
Flickering
Flicker is a visible change in luminous flux of an illuminated light source which can be due to fluctuations of the light source itself, or due to external causes. It can also happen due to correction algorithms oscillations. You can see a light source flicker in the animation below (this specific one likely due to the frequency of electricity being lower than the capture frequency of the video)
Blur
Even if not really an artifact, it is worth to mention as it often needs to be detected and corrected. Blur is when something is unclear or less distinct, an example can be seen below.
Brightness / contrast
When brightness or contrast are off for the image then some details are lost or emphasized and the image can look strange.
Closing words
Now, I must apologize, because next time you watch a video, you will start to pinpoint those errors you did not pay attention to earlier. But now you can also show-off and be a video content professional… Happy hunting!
Top comments (0)