OpenCV for counting horn flies on dairy cattle images

---

Object detection & Image classification are two of the most interesting and widely used applications of Computer Vision. Over the years, the multiple fold improvement in computing power and the introduction of some very interesting algorithms has made both these problems very easy to tackle.

Based on this confidence, we were very excited when one of our clients brought forth a very interesting idea. They wanted to count horn flies on dairy cattle. We did a little digging to find out that horn flies are a menace to cattle, specifically dairy cattle. They have been found to impact the health and milk production of cows tremendously. Horn flies were found to have single-handedly resulted in losses of almost $ 1.36 billion per year. Several methods to control horn flies’ population are in use currently. But they are expensive and sometimes excess use makes them ineffective in some horn fly populations. Therefore, the use of such methods has to be done judiciously, and only when the horn fly population has crossed a certain limit.

This brings us the next roadblock and our problem statement.

Counting horn flies settled on a cow.

Manual methods are inefficient, inaccurate, and cumbersome. The count can greatly vary by the time a person completes counting the flies. Capturing high-quality images, and then dividing them into grids to assist with manual counting is not a great improvement.

Can we count horn flies settled on a cow using Computer Vision?

This was the question posed to us by our client. It was intriguing, challenging, and perfect to whet our appetite for interesting problems.

OpenCV to detect & count flies

Before diving deep into machine learning-based solutions, we tried using OpenCV to see how we could manipulate the images to get a fly count.

Fig.1 — A cow with numerous horn flies settled on its back and a very big mud patch which can make detection difficult (Image is property of Land O’Lakes Inc.)

As expected, there were numerous challenges when using OpenCV for fly counting.

• inconsistent contrast due to sunlight
• multiple cows in a single image
• cows of variable colors — white, black, brown
• the relative size of fly object and image

Still, we used a pipeline to see how the results would look

• grayed the image using cv2.cvtColor
• blurred the image using cv2.GaussianBlur
• thresholded the image using cv2.adaptiveThreshold
• morphed the image using cv2.morphologyEx with cv2.MORPH_OPEN argument
• found edges using the Sobel edge detection method extracted ROIs
• detected contours and filtered them based on mean pixel values and contour areas — this would give us the fly contours

This worked well for clear images, where the distinction between flies and cow background was sharp as shown in Fig. 2

Fig. 2 A cow image with a sharp contrast between flies and cow background

But for images such as Fig.1, this methodology failed spectacularly.

There were quite a few reasons why even a fantastic library like OpenCV couldn’t tackle the challenges posed by this problem statement.

• even with adaptive threshold, it was impossible to properly binarize all images due to variable shading on cow body, variable cow skin colors, and contrast with fly colors.
• improper binarization led to improper edge detection and contour detection
• even with good binarization, there were several contours that matched the criterion for fly contours, and it was difficult to find a metric that would help isolate the fly contours only

Conclusion

OpenCV worked well for some test images. We were also able to tweak the parameters a little to make it work for a particular set of images. But when tested on some difficult images, the count was way off. Fig. 3 is a good example of images where it worked well. And Fig. 4 is a good example of why this approach is not suitable for building an actual application that can be used confidently in the field by farm owners. Due to the critical nature of the decision making that depends upon the application’s output, such an approach was not deemed fit.

This was expected.

Fig.3 An image where detection worked well except for the small patch of sunshine on the cow’s body (Image is property of Land O’Lakes Inc.)

Fig. 4 An image where detection failed. (Image is property of Land O’Lakes Inc.)

While playing with OpenCV for this problem statement, it was expected that it wouldn’t fully work. But this was essential to showcase to the client that using Computer Vision is the right way forward. It established confidence that a computational approach for such an application is not sci-fi anymore but is very much in the realm of possibilities.

Maybe OpenCV is not the answer but machine learning could be. We’ll find out.