DEV Community: Tarush Nagpal

Building a Real-Time Face Recognition System with ESP32-CAM (in a Weekend)

Tarush Nagpal — Thu, 15 Jan 2026 09:16:59 +0000

How I built a local, real-time face detection and tracking system that knows who’s around and when. Using just ₹800 of hardware.

ESP32-CAM → MJPEG stream → Python server → face recognition → web dashboard

The Itch That Needed Scratching

Last Friday evening, after a conversation with my friend Palash, I couldn’t shake an idea.

I had an ESP32-CAM lying on my desk, and I wanted to build something useful: a system that could tell who was around and when, without cloud APIs, subscriptions, or training datasets.

Not surveillance. Not production-grade. Just something practical, local, and hackable.

By Sunday night, it worked.

What We’re Building

A fully local face recognition system that:

Streams live video from an ESP32-CAM
Detects and identifies faces automatically
Tracks detection timelines over time
Displays everything in a simple web dashboard

No cloud. No external APIs. Everything runs on your local network.

What this is and isn’t

This is:

A privacy-friendly, local presence-detection system
Ideal for personal projects and experimentation

This is not:

Production surveillance software
Robust under extreme lighting or crowded scenes

Why ESP32-CAM?

The ESP32-CAM is absurdly good value for what it offers:

Cost: ~$10
Built-in camera: 2MP, good enough for faces
WiFi: No cables
Programmable: Full Arduino support
Low power: Can run 24/7

The tradeoff is limited resolution and FPS—but that’s fine. All the intelligence lives on the server.

Part 1: Streaming Video from the ESP32-CAM

We start with the simplest possible requirement: live video.

Not recording. Not RTSP. Just streaming.

Why MJPEG?

MJPEG is perfect here. It’s simply a continuous stream of JPEG images sent over HTTP. No complex codecs, no decoding pipelines.

Hardware Setup

You’ll need:

ESP32-CAM (AI-Thinker model)
USB-to-Serial adapter (for flashing)
5V power supply (3.3V is not enough)

ESP32-CAM Streaming Code

Here’s a minimal streaming setup. This exposes a /stream endpoint that serves MJPEG.

#include "esp_camera.h"
#include <WiFi.h>
#include "esp_http_server.h"

const char* ssid = "YOUR_WIFI";
const char* password = "YOUR_PASSWORD";

// AI-Thinker pin configuration
// https://randomnerdtutorials.com/esp32-cam-ai-thinker-pinout/

static esp_err_t stream_handler(httpd_req_t *req) {
  while (true) {
    camera_fb_t *fb = esp_camera_fb_get();
    if (!fb) continue;

    httpd_resp_send_chunk(req, (const char *)fb->buf, fb->len);
    httpd_resp_send_chunk(req, "\r\n--frame\r\n", 12);

    esp_camera_fb_return(fb);
  }
}

void setup() {
  Serial.begin(115200);

  camera_config_t config;
  config.pixel_format = PIXFORMAT_JPEG;
  config.frame_size = FRAMESIZE_SVGA; // 800x600
  config.jpeg_quality = 10;

  esp_camera_init(&config);

  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) delay(500);

  startCameraServer();

  Serial.print("Stream available at: http://");
  Serial.println(WiFi.localIP());
}

Live MJPEG stream from the ESP32-CAM - this is what we'll be processing

Part 2: The Brain - Python + Face Recognition

Now that we have video, we need to:

Consume the MJPEG stream
Extract frames
Detect faces
Identify who they belong to
Track detections over time

Libraries Used

We're using face_recognition built on top of dlib.

pip install face-recognition opencv-python flask requests

On my machine, this processes ~8–12 FPS using the HOG model. CNN is more accurate but slower.

Consuming the MJPEG Stream

MJPEG is just JPEG images separated by boundaries.

JPEGs start with 0xFFD8 and end with 0xFFD9. We extract frames by scanning for these markers.

import requests
import cv2
import numpy as np

def consume_stream(url):
    response = requests.get(url, stream=True)
    bytes_data = bytes()

    for chunk in response.iter_content(chunk_size=1024):
        bytes_data += chunk

        # Find JPEG markers
        a = bytes_data.find(b'\xff\xd8')  # Start
        b = bytes_data.find(b'\xff\xd9')  # End

        if a != -1 and b != -1:
            jpg = bytes_data[a:b+2]
            bytes_data = bytes_data[b+2:]

            # Decode and process
            image = cv2.imdecode(
                np.frombuffer(jpg, np.uint8), 
                cv2.IMREAD_COLOR
            )
            process_frame(image)

Face Detection & Encoding

import face_recognition

def process_frame(image):
    # Convert BGR to RGB (OpenCV uses BGR)
    rgb_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

    # Find faces
    face_locations = face_recognition.face_locations(rgb_image)
    face_encodings = face_recognition.face_encodings(
        rgb_image, 
        face_locations
    )

    for encoding in face_encodings:
        person_id = find_or_create_person(encoding)
        record_detection(person_id, datetime.now())
        save_face_image(person_id, image, face_location)

Automatic Identification (No Manual Labels)

When a new face appears, we compare it against known encodings. If it doesn’t match anyone, we create a new person ID.

def find_or_create_person(face_encoding):
    # Compare with known persons
    for person_id, known_encodings in persons.items():
        matches = face_recognition.compare_faces(
            known_encodings, 
            face_encoding, 
            tolerance=0.5
        )

        if True in matches:
            return person_id

    # New person! Create entry
    new_id = f"person_{next_id()}"
    persons[new_id] = [face_encoding]
    return new_id

Duplicates can occur under poor lighting or extreme angles. I added a manual merge feature to fix this.

Each person typically stabilizes at ~5–10 encodings with a max of 20.

Part 3: Visualising

Bucketing Detections

We bucket detections into 30-second intervals:

def get_bucketed_detections(person_id, hours=24):
    detections = person_detections[person_id]
    cutoff = datetime.now() - timedelta(hours=hours)
    recent = [ts for ts in detections if ts > cutoff]

    buckets = defaultdict(int)
    for ts in recent:
        # Round to 30-second bucket
        bucket = ts.replace(
            second=(ts.second // 30) * 30, 
            microsecond=0
        )
        buckets[bucket] += 1

    return buckets

This gives us smooth graphs instead of spiky mess.

The Dashboard

Flask + Chart.js gets you a clean UI quickly:

new Chart(ctx, {
    type: 'line',
    data: {
        labels: timestamps,
        datasets: [{
            label: 'Detections per 30s',
            data: counts,
            borderColor: '#667eea',
            fill: true
        }]
    }
});

Buttons let you toggle between 1 hour, 6 hours, 24 hours, and 7 days.

The Results

After a weekend of hacking:

Real-time processing: ~10 FPS
Fully local: No cloud, no data leaves the network
Automatic identification: No training step
Historical tracking: Clear presence patterns

Detection frequency over 7 days - you can clearly see patterns of when someone is around

The "Aha!" Moments

Moment 1: Realizing MJPEG is just JPEG markers was liberating. No complex video codecs.

Moment 2: The face_recognition library is stupid simple. No TensorFlow, no model training, just works.

Moment 3: 30-second bucketing makes graphs actually useful. Raw timestamps are too noisy.

The Gotchas

Things that bit me:

ESP32-CAM needs 5V: 3.3V will make it reboot randomly
Face detection is CPU-intensive: ~0.3-1 second per frame with HOG model
Lighting matters: Needs decent light for face detection
Face encodings need diversity: Store multiple encodings per person from different angles

What's Next?

Now that the detection system is working, I'm planning to take this further - building a physical enclosure with some 3D printed parts and adding an AI agent that can actually interact based on who it detects. But that's a story for another post.

Try It Yourself

The full code is on GitHub: [https://github.com/tarushnagpal/esp32-cam-face-recognition]

Requirements:

ESP32-CAM (~$10)
Python 3.8+
A weekend

Final Thoughts

This project shows how far you can get with simple, cheap hardware and some weekend coding. A face detection system that runs entirely locally, tracks presence over time, and costs less than lunch. Not bad for a few hours of work.

Coming Soon: Taking this further with 3D printing and AI agents

Resources

Making Nike Run Club Searchable

Tarush Nagpal — Tue, 16 Sep 2025 14:19:57 +0000

https://nrc.tarushnagpal.com

Find Any Nike Run Club Guided Run Instantly

Nike’s Run Club (NRC) app is beloved by runners for its Audio-Guided Runs, where top coaches (and even stars like Eliud Kipchoge) talk you through workouts. These runs cover everything from easy jogs to tough speed sessions. The NRC app now has hundreds of them (306 to be exact), but frustratingly, there’s no built-in search or filter. Most runners end up endlessly scrolling just to find the one they want.

Here are just a few examples of people complaining (and there are many more):

So I built NRC Run Search to solve exactly this. The site lets you:

Search by keyword: Type any word from a run’s title or description (e.g. “tempo”, “Bennet”, “intervals”) and instantly filter the library. Because Nike’s coaches have already prepared amazing guided workouts, we’re just making them easier to find.

How It Was Built

There were two main problems while building this:

I didn’t want to manually build the dataset by clicking on each run, copying the info, and cleaning it up.
The share link in the app kept changing every time I clicked it, and it wasn’t tied to the run’s actual ID.

Solving Problem 1

I used HTTP Toolkit to set up an HTTP proxy.
This let me route my iPhone traffic through the proxy running on my laptop (same Wi-Fi).
And voilà — all the app’s requests showed up in the proxy!

From there, I just copied the contents of each request. (Sure, I could’ve scripted it, but honestly, it was only ~12 pages, so this worked fine.)

The dataset now lives in my open-source app here: GitHub link.

Solving Problem 2

Remember the rotating link issue? Turns out NRC uses singular.net to generate deep link URLs — probably for shortening + tracking (seems a bit overkill).

The good news: the payload of that call already contained the actual deep link to the app.

Here’s the code that generates the deep link.

And that’s it — both problems solved, which gave us nrc.tarushnagpal.com.

Messing around with Proxy objects in Javascript!

Tarush Nagpal — Wed, 04 Jan 2023 21:28:32 +0000

The Problem

Let's say you are working with a few isolated components that have no idea who their children are nor their parents (Because they could be any and many), now each of these components rely on certain information that updates continuously from a (say) library.

So now each component is expected to go on and say

...
library.addEventListener('someUpdate', doSomething);
...

But when your number of events and your number of components keep going up, you start approaching memory leak scenarios and just memory issues on browsers on lower tier machines/phones.

The solution?

What if we build a single store that listens to all of these events so you have only one listener for each type of property! BUT, how does each isolated component actually talk to the store, Another event listener? That was the problem we were trying to solve! 🤦‍♂️

Introducing,

The Proxy object.

I am not going to go into depth here but you can check out MDN.

Basically, a Proxy defined as,

const target = {
  message1: "hello",
  message2: "everyone",
};

const handler1 = {};

const proxy1 = new Proxy(target, handler1);

can handle events happening to the target and perform certain actions before or after actually doing what the event expected it to do.

In simpler terms,

For a backend developer -> It's a middleware
For a OOP developer -> It's an overridden getter/setter/(Plus a few more)
For a frontend developer -> It's a Proxy!

Solving the problem

Now let's actually solve the problem.
Let's say we want two components to listen for resize events on a div. Now instead of each component adding their own individual listeners we are going to create a store.ts that defines a proxy like this

/**
 * Listener Store
 */

let store = {
  windowHeight: 0,
  windowWidth: 0,
};

let storeProxy = new Proxy(store, {
  set(obj, prop, value) {
    /**
     * Can also put validation here
     */
    obj[prop] = value;
    return true;
  },
});

Then creates a resize observer which updates the value of the proxy.

const resizeObserver = new ResizeObserver((entries) => {
  const entry = entries[0];
  const { width, height } = entry.contentRect;
  storeProxy.windowHeight = height;
  storeProxy.windowWidth = width;
});

resizeObserver.observe(document.getElementById('app'));

And we will also add a variable called callOnStoreUpdate which will be an array of functions to be called whenever a store is updated, hence making the entirety of store.ts

/**
 * Listener Store
 */

/**
 * Holds an array of functions to call when any value
 * in store updates
 */
let toCallOnUpdate = [];

let store = {
  windowHeight: 0,
  windowWidth: 0,
};

let storeProxy = new Proxy(store, {
  set(obj, prop, value) {
    /**
     * Can also put validation here
     */
    if (obj[prop] === value) return true;
    obj[prop] = value;
    toCallOnUpdate.forEach((fun) => fun(obj, prop, value));
    return true;
  },
});

const resizeObserver = new ResizeObserver((entries) => {
  const entry = entries[0];
  const { width, height } = entry.contentRect;
  storeProxy.windowHeight = height;
  storeProxy.windowWidth = width;
});

resizeObserver.observe(document.getElementById('app'));

export const callOnStoreUpdate = (fun: (obj, prop, value) => void) => {
  toCallOnUpdate.push(fun);
};

I think now you might have understood where we were getting at.

We simply in how many ever components we want go and add

/**
 * Sample component!
 */
import { callOnStoreUpdate } from '../listeners';

const onResize = (obj, prop, value) => {
  switch (prop) {
    case 'windowHeight':
      console.log('This is component.ts reporting the window height!', value);
      break;
    case 'windowWidth':
      console.log('This is component.ts reporting the window width!', value);
  }
};

callOnStoreUpdate(onResize);

And boom 💥! We have managed to capture updates to a variable without the need to ever add more than one event listener.

I will be exploring a bit more into Proxy and just making sure I understood everything correctly, but I feel like this is an under-utilised web api which can really do crazy things!

This was a problem that we faced over at Dyte, check us out we're pretty cool.

Also a full working sample. Stackblitz

Making an Advent of Code solution for beginners!

Tarush Nagpal — Sun, 02 Dec 2018 12:10:56 +0000

What is Advent of Code? a fun series of coding challenges all through december! (If you don't know about them, Check it out!)

So I was going through most of the solutions for advent of code and some seemed really complex with no descriptions so I thought to myself why not make a beginner friendly solution for the Advent of Code where I explain the concepts and give out the code in simple python!

The Repo!

I'm no expert in python and this would be a fun way to get comfortable with the language as well and help the community.

This is the first time I'm doing something like this, if you have any tips or suggestions please Let me know! Any help is appreciated.

The Django Rest Custom User Model and Authentication

Tarush Nagpal — Sun, 22 Jul 2018 09:39:01 +0000

I was recently building a website for one of my clients by using the Django rest + React framework ( which is beautiful by the way ) and encountered a problem.

The Problem

If any of you have worked with Django Rest you'll know the multiple issues that pop up while making a custom user model, and specifically how the changes you make don't reflect upon a save.

I'll show you the best way (in my opinion) to build a custom user model in DRF (Django Rest Framework), without a username.

Step 1

Create an app called user ( Or whatever you want to call it ) and in the models.py make a User model which extends AbstractBaseUser

    from django.db import models
    from django.contrib.auth.models import AbstractBaseUser,BaseUserManager
    import datetime

    class User(AbstractBaseUser):

        username = None
        email = models.EmailField(_('email address'), unique=True)
        name = models.CharField(max_length=100)
        date_of_birth = models.DateField(default=datetime.date.today)

        USERNAME_FIELD = 'email'
        REQUIRED_FIELDS = [ 'date_of_birth','name' ]

        def __str__(self):              # __unicode__ on Python 2
            return self.email

Step 2

And now we add a very important line in the model

    objects = UserManager()

This basically tells Django how it's supposed to store the Object and with what attributes depending on permissions.

Step 3

We add the usermanager

    class UserManager(BaseUserManager):

    use_in_migrations = True

    def create_user(self, email, name, date_of_birth, password=None):
        user = self.model(
            email=self.normalize_email(email),
            date_of_birth=date_of_birth,
            name=name,
        )
        user.set_password(password)
        user.save(using=self._db)
        return user

    def create_staffuser(self, email, name, date_of_birth, password):
        user = self.create_user(
            email,
            password=password,
            date_of_birth=date_of_birth,         
            name=name,
        )
        user.staff = True
        user.save(using=self._db)
        return user

    def create_superuser(self, email, name, date_of_birth, password):
        user = self.create_user(
            email,
            password=password,
            date_of_birth=date_of_birth,
            name= "True",
        )
        user.staff = True
        user.admin = True
        user.save(using=self._db)
        return user

Step 4

Build the CustomRegisterSerializer

    from rest_auth.registration.serializers import RegisterSerializer
    class CustomRegisterSerializer(RegisterSerializer):

        email = serializers.EmailField(required=True)
        password1 = serializers.CharField(write_only=True)
        name = serializers.CharField(required=True)
        date_of_birth = serializers.DateField(required=True)

        def get_cleaned_data(self):
            super(CustomRegisterSerializer, self).get_cleaned_data()

            return {
                'password1': self.validated_data.get('password1', ''),
                'email': self.validated_data.get('email', ''),
                'name': self.validated_data.get('name', ''),
                'date_of_birth': self.validated_data.get('date_of_birth', ''),
            }

    class CustomUserDetailsSerializer(serializers.ModelSerializer):

        class Meta:
            model = User
            fields = ('email','name','date_of_birth')
            read_only_fields = ('email',)

Create the custom view which just has all user objects as a query set

    from rest_auth.registration.views import RegisterView

    class CustomRegisterView(RegisterView):
        queryset = User.objects.all()

Finally update the settings.py to apply these

ACCOUNT_USER_MODEL_USERNAME_FIELD = None
ACCOUNT_AUTHENTICATION_METHOD = 'email'

ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_UNIQUE_EMAIL = True
ACCOUNT_USERNAME_REQUIRED = False
ACCOUNT_USER_EMAIL_FIELD = 'email'
ACCOUNT_LOGOUT_ON_GET = True

AUTH_USER_MODEL = 'users.User'

REST_AUTH_SERIALIZERS = {
    "USER_DETAILS_SERIALIZER": "users.serializers.CustomUserDetailsSerializer",
}
REST_AUTH_REGISTER_SERIALIZERS = {
    "REGISTER_SERIALIZER": "users.serializers.CustomRegisterSerializer",
}

And just add the view to the URL and you're good to go!

I honestly believe that the Django Rest + React stack is one of the best ones in the game right now, but the Django Rest Framework does need a little bit more work to make it perfect for most developers.

Hit me up if you need help! Or have some projects that need people to work with :D