Every Devoxx Belgium edition produces 4000+ pictures by our event photographer Dimitris (and in the past by An). These pictures are then published into Flickr albums and can be freely used by speakers, attendees and the event organiser.
Finding a specific speaker in the different albums often feels like going down a visual rabbit hole. After scrolling through many Flickr pages and wasting a bit of time, you often find the hidden gem.
This time consuming effort was crying for an automated approach!
Since many years I was searching for a technical solution which would provide me an API that I could integrate in the different conference apps.
A first low level Python solution
In 2018 Alex van Boxel introduced me to Lien Wuyts. She was interested in doing her dissertation exactly on this subject. With the support from Alex, Jan-Kees and myself she worked on a Python prototype where an existing paparazzi model was used to find speaker faces.
A first rough solution was demo'ed during Devoxx Belgium 2018 in an informal BOF session. Together with Jan-Kees she also provided more details during a Devoxx interview.
An Out of the Box Solution
The solution from Lien remained an experimental prototype. Luckily Jan-Kees van Andel was still interested in this project and he suggested the JPoint team would tackle this opportunity during a team-building weekend. After that weekend he informed me they had a working solution based on AWS Recognition.
It works!
Tim van Eijndhoven (a colleague of Jan-Kees) took ownership of the project and he provided a REST interface which I could use from within the new Angular CFP.DEV web app.
The integration was very straight forward. I post the speaker avatar image using a REST interface and the "face-recognition backend" returns me a list of Flickr URL's. These are then listed in the CFP.DEV admin for review. Each picture also has a similarity percentage and includes even the rectangle coordinates where the face was found within the picture.
My PHP CFP.DEV plugin then displays these speaker images on a related Wordpress page.
Wordpress page
Amazon Rekognition also provides highly accurate facial analysis and facial search capabilities that you can use to detect, analyze, and compare faces for a wide variety of user verification, people counting, and public safety use cases.
The Amazon Rekognition model is a scary beast because it works frighteningly well. It finds speakers faces in various unsuspected places. During several Devoxx Belgium program committee meetings we were testing the system with all sorts of pictures and each time we were amazed of the results it presented.
Where's Wally?
Or... where's Mario Fusco? The recognition API returned pictures of speakers where we often had no idea if (s)he was included. After searching for several minutes you could find the speaker, often hidden away with only a portion of the face visible 😱
Or some pictures were returned (for example here of James Birnie) where a partial speaker face was shown on the twitter wall.
Or Louis Jacomet was found because he photobombed our end-of-devoxx-belgium pizza picture 😂🍕
You can find other gems by visiting the Devoxx Belgium speakers page.
Privacy & GDPR
Because of the amazing accuracy we decided to only use this for the speakers. Every user that submits a talk needs to accept the CFP terms & conditions. As a result this gives us permission to publish their talk on YouTube and share their event photo's on Flickr 👍🏼
Project Details
The "Face recognition" project was obviously presented at Devoxx Belgium 2019 by both Tim van Eijndhoven and Roy Braam.
Make sure to watch the presentation if you want more technical details.
Arun Gupta was of course very excited to interview Tim and Roy about their AWS solution.
Thanks again guys for the effort, much appreciated.
What's next?
We now have a nice working solution which obviously could be offered as a service by JPoint, will see what happens on that front 😎
But let's brain-storm for a minute what else could we do with this face recognition solution if we exclude the privacy part:
We could have a camera at the entrance of each room and track which attendee goes to which talk. Based on this info we could recommend talks to other attendees who go to similar sessions. Of course this could already be done just by analysing their schedule favourites instead of tracking faces, which is too scary.
We could allow attendees to upload their photo and we then return all the related Flickr photos where they're included.
Roy van Rijn suggested on Twitter that this solution could also be used in situations where a person wants to enforce his "right to be forgotten". We can now easily find all pictures where a person is included and censure him/her out.
We could have a camera at the main entrance and welcome speakers when they enter the venue. Or use it as a notification to inform me that the speaker has arrived. That would actually be cool but we can already do this when they scan their eTicket.
What else could we do moving forward?
Let me know in the comments below 😎👌
Top comments (0)