Earlier this week, I found myself staring at a noisy heart rate signal, wondering how it could possibly be useful. As the sole founder and engineer of EmoPulse, I've grown accustomed to dealing with imperfect data, but this particular signal seemed like a lost cause. And yet, as I delved deeper into the issue, I stumbled upon an unexpected insight: this noisy signal could become our anti-spoof primitive.
The underlying lesson here is that sometimes, the things that seem like flaws or imperfections can actually become our greatest strengths. In the case of our remote photoplethysmography (rPPG) implementation, the noise and variability of the heart rate signal made it seem like a poor candidate for any serious application. However, as we began to explore its properties, we realized that this very noise could be used to detect spoofing attempts. It's a counter-intuitive observation, to say the least: the thing that makes our signal imperfect is also what makes it secure.
Our custom rPPG implementation, which extracts heart rate from face color changes, is just one part of our broader behavioral perception infrastructure layer. We use MediaPipe's 478 facial landmarks and 52 ARKit blendshapes to extract 47 biometric and behavioral signals from any standard RGB camera, all on-device in the browser via WebAssembly. The output is a structured state vector posted to a Flask /state endpoint on our server, which runs on a $0/month Oracle ARM box in Chicago. As we worked to develop our liveness scoring system, we discovered that the rPPG signal's noise could be used to detect anomalies in the data. Specifically, we found that a BPM standard deviation above 12 BPM, combined with other penalty signals like gaze stability and micro-expression count, could be used to identify spoofing attempts with remarkable accuracy.
This insight has significant implications for our journey at EmoPulse. As we continue to develop and refine our technology, we're forced to confront the trade-offs between security, accuracy, and usability. Our liveness scoring system, which runs on the server-side and uses a sliding window of the last 5 ticks per session, is just one example of how we're working to balance these competing demands. As we move forward, we'll need to continue exploring the properties of our signals and finding creative ways to leverage their imperfections.
What will be the next unexpected benefit to arise from our imperfect signals, and how will it change the course of our development?
Top comments (0)