Computer Vision is so fun!

I work as a Software Engineer at Endtest.

In this article, I will show you how easy it is to use Computer Vision.

The term itself sounds intimidating, it might make you think that you need a PhD in Machine Learning.

Why is it useful?

Computer Vision can be used to detect objects in images.

If you have an Arduino, you can build your own self-driving toy car that detects the road signs.

It can also be used to detect differences between images.

This second use case can be extremely useful for developers who want to do automated visual testing.

Let's focus on that one.

Detecting differences between images

The entire concept is dead simple, we just compare each pixel and we calculate the percentage of different pixels.

We’ll just need to make sure our system has Python, OpenCV, scikit-image, Pillow and imutils.

You can find instructions for installing OpenCV here.

As for the rest, you can just use pip:

pip install scikit-image

pip install imutils

pip install pillow

First, we need to make sure that the images have the same size.

The quickest way to achieve that is just to resize the bigger one.

import argparse
import imutils
import cv2
from PIL import Image
from PIL import ImageFilter
from PIL import ImageDraw

image1 = Image.open("/path/to/image1.png")
image2 = Image.open("/path/to/image2.png")

image1_width, image1_height = image1.size
image2_width, image2_height = image2.size

image1_surface = image1_width * image1_height
image2_surface = image2_width * image2_height

if image1_surface != image2_surface:
   if image1_surface > image2_surface:
      image1 = image1.resize((image2_width, image2_height), Image.ANTIALIAS)
      image1.save("/path/to/image1.png")

   if image2_surface > image1_surface:
      image2_surface = image2_surface.resize((image1_width, image1_height), Image.ANTIALIAS)
      image2.save("/path/to/image2.png")

Now our images have the same size.

Time to compare them:

image1 = cv2.imread("/path/to/image1.png")
image2 = cv2.imread("/path/to/image2.png")

i1 = image1
i2 = image2

assert i1.mode == i2.mode, "Different kinds of images."

pairs = izip(i1.getdata(), i2.getdata())
   if len(i1.getbands()) == 1:
      dif = sum(abs(p1 - p2) for p1, p2 in pairs)
   else:
      dif = sum(abs(c1 - c2) for p1, p2 in pairs for c1, c2 in zip(p1, p2))

ncomponents = i1.size[0] * i1.size[1] * 3
diff = (dif / 255.0 * 100) / ncomponents

diff = Decimal(diff)
diff = round(diff, 2)
print "Difference between images is " + str(diff) + "%." 

Difference between images is 7.11%.

We can also generate a third image that shows us the differences.

This can be done by generating a mask and adding it on top of the first image.

difference = cv2.subtract(image1, image2)
Conv_hsv_Gray = cv2.cvtColor(difference, cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(Conv_hsv_Gray, 0, 255,cv2.THRESH_BINARY_INV |cv2.THRESH_OTSU)

difference[mask != 255] = [0, 0, 255]

image1[mask != 255] = [0, 0, 255]
image2[mask != 255] = [0, 0, 255]

cv2.imwrite("/path/to/image3.png", image1)

Benefits

There have been documented cases where elements have disappeared from pages and no one noticed it for weeks.

Implementing a visual check for your site can't hurt.

And think of how much fun you can have by testing your implementation on all of those Spot the differences challenges.

POC vs Reality

What I showed you was a basic example.

If you would apply this on an actual system, you would also need to make small tweaks.

For example, let's say that your images are identical, but one of them is 2px higher than the other.

You would need a function to focus the 2 images before comparing them.

Almost forgot

If you want to perform Automated Testing without reinventing the wheel, you can use Endtest.