I Built an Adaptive Hand Cricket Game Using React, MediaPipe, and TensorFlow.js

Ayush Shah — Tue, 15 Sep 2026 04:54:14 +0000

I wanted to build something fun, not another CRUD app or chatbot.

So I built Adaptive Hand Cricket: a browser-based game where you play hand cricket through your webcam, your hand gestures are recognized in real time, and the computer gradually becomes harder by combining randomness, live gesture reading, and machine learning.

How it works

The game uses:

React + Vite for the frontend
MediaPipe for real-time hand landmark detection
TensorFlow.js for learning player patterns
Webcam input through getUserMedia
localStorage to remember previous batting behavior

The supported gestures are:

1 -> index
2 -> index + middle
3 -> index + middle + ring
4 -> index + middle + ring + pinky
5 -> all fingers
6 -> thumb
7 -> thumb + index
8 -> thumb + index + middle
9 -> thumb + index + middle + ring

Instead of simply counting fingers, I classify the exact finger pattern.

The game loop

Every delivery follows:

Computer commits to a move
        ↓
        3
        2
        1
        ↓
      SHOW
        ↓
Human gesture is locked
        ↓
Both moves are revealed
        ↓
Score / wicket is calculated

This keeps the reveal synchronized like actual hand cricket.

Making the opponent adaptive

A completely random computer got boring quickly, so I added three decision strategies:

RANDOM
ML
PEEK

Difficulty automatically increases with the player's score:

0-39   -> EASY
40-99  -> MEDIUM
100+   -> HARD

Each level changes how often the opponent uses each strategy.

For example:

EASY: {
  peekChance: 0.10,
  mlChance: 0.30,
  randomChance: 0.60
}

Peek strategy

Humans often form their gesture before the countdown reaches zero.

On some deliveries, the computer is secretly allowed to inspect the gesture at a random point during:

3... 2... 1...

If it gets a valid reading, it can use that number.

Slightly evil. Very effective.

Learning player patterns

The more interesting strategy uses TensorFlow.js.

Every batting move is stored:

7, 4, 2, 7, 8, 4, 7...

The model learns from:

the player's last 5 moves
overall frequency of numbers 1-9
recent frequency of numbers 1-9

This produces 63 input features.

The network is intentionally small:

63 inputs
   ↓
Dense 64
   ↓
Dense 32
   ↓
9-way Softmax

The output might look like:

4 -> 18%
7 -> 42%
8 -> 12%
...

So the model predicts 7 as the player's next move.

Training starts after enough moves have been collected and retrains periodically rather than after every ball.

What surprised me

The machine learning was not actually the hardest part.

The trickier problems were:

thumb detection across different hand orientations
synchronizing countdown and capture timing
preventing invalid camera frames from affecting gameplay
managing game transitions cleanly
keeping browser ML from interfering with real-time video processing

A small game ended up touching computer vision, React state management, probability, feature engineering, browser ML, and real-time interaction design.

What I'd improve next

I'd like to add:

a short multi-frame voting window instead of using one camera frame
proper ML-vs-random prediction accuracy tracking
player profiles
a comparison between neural networks and simpler approaches like Markov chains
better gesture classification for rotated hands

Tech Stack

React
Vite
MediaPipe Tasks Vision
TensorFlow.js
WebRTC / getUserMedia
localStorage

It started as a simple question:

Can I play hand cricket against my webcam?

It turned into:

Can the computer learn how predictable I am?

DEV Community: Ayush Shah