Computer vision caught my eye at the Whitney Biennial last year, where I watched Triple Chaser, Forensic Architecture's investigation of the Triple-Chaser tear gas grenades and how law enforcement officials use them to suppress civilians. Their team used computer vision to detect Triple-Chaser canisters in footage from around the world.
Object detection requires thousands of images to be accurately trained, and Forensic Architecture couldn't get their hands on that many images of the actual Triple-Chaser. By zeroing in on footage where enforcement used these grenades, they were able to build a digital model of the Triple-Chaser grenade. Then they generated thousands of photorealistic 'synthetic' images, situating the Triple-Chaser in semblances of real-world environments. From the project page:
"In this way, ‘fake’ images help us to search for real ones, so that the next time Safariland munitions are used against civilians, we’ll know."
US border agents fired Triple Chasers at civilians in November 2018, and Forensic Architecture proved the presence and use of other Safariland products by law enforcement officers during civil unrest in Puerto Rico in 2018.
Once they had thousands of images to work with, they trained an object detector and then deployed it to find more images, exposing more instances of governments using Triple-Chasers to suppress civilians.
The Safariland Group is owned by Warren B. Kanders, who was the vice chair of the board of trustees of the Whitney Museum of American Art before stepping down because of this exposure.
That intersection of code, art, and activism really made an impact. I love the idea of building things with code for the greater good and I was inspired to learn more about it. At the Codeland conference last year, I attended a workshop on Real-Time Object Detection with TensorFlow.js led by Nicholas Bourdakos, a software developer at IBM. Within a couple of hours, I built my own app using custom-trained models to detect objects.
Everyone needed a lot of photos of their object to thoroughly train the model, and since we were creating this at the workshop, we needed something handy, like our own two thumbs. Our mission was to train our models to detect a thumbs up and a thumbs down. It was quite a sight to see a room full of people snapping countless photos of their thumbs. The more photos we had, the better trained our models would be, but even with just a few hundred photos, it worked!
I was surprised that we could build this kind of app within such a short timeframe and I definitely want to explore it more in the future, ideally for something that could have a positive social impact like this IBM Visual Insights and IBM Spectrum Discover: Automation with Computer Vision (Covid-19 Use Case) video, which shows how to detect Covid-19 from x-rays using object detection (skip to 7:50).
Do it Yourself:
The entire tutorial of this Object Detection Workshop is documented on IBM Cloud Annotations' website.
Other Resources:
Cloud Annotations Github
Top comments (0)