How I Built a Computer Vision Chess Board Detector

MD ABUBAKAR — Tue, 09 Dec 2025 11:26:24 +0000

I Built a Chess Scanner That Converts Any Chess Image Into a FEN + Analyzes Games Like Chess.com

👉 Live website: https://chessscanner.seedrlite.in/

🧠 Introduction

Chess engines are extremely powerful — but only when they have a FEN position or a PGN to analyze.
Most of the time, people see a board in a photo, a screenshot, or a video frame… and wish they could instantly analyze it.

So I built a web application called Chess Scanner that:

✅ Detects the chessboard from any image
✅ Identifies all 12 piece types (White/Black × King/Queen/Rook/Bishop/Knight/Pawn)
✅ Generates a perfect FEN string
✅ Lets you edit the detected board
✅ Analyzes PGNs and classifies moves (Good, Great, Brilliant, Miss, Blunder, etc.)
✅ Plays best moves using Stockfish

This post explains how I built it, the challenges I faced, and how it works under the hood.

♟ 1. The Idea

My goal was simple but ambitious:

“Upload any chess image → get instant FEN → analyze it like on chess.com.”

There are apps that do individual tasks, but none that combine:

Computer vision
FEN extraction
Real-time board editing
Game analysis
Chess.com-like move classification

I wanted this all in one clean web app.

🏗 2. Architecture Overview

Frontend (React + Tailwind)
        ↓ Sends image
Backend (FastAPI + YOLOv8)
        ↓ Detects board + pieces
Board Processor → Generates FEN
Stockfish Engine → Analysis
Frontend Visualization → Move types + best moves

Tech Stack

React + Tailwind for UI
FastAPI (Python) for backend
YOLOv8 (Ultralytics) for board & piece detection
Custom dataset (Roboflow)
Stockfish 17.1 for move evaluation
SQLite authentication system
JWT auth
Cloud VPS (Ubuntu, Nginx)

🔍 3. Training the Vision Model

Dataset

I created my own labeled dataset using Roboflow:

Board Detection Model
- Segmentation + bounding box
- Trained on real + synthetic chessboard images
Piece Detection Model
- 12 classes (white/black for each piece)
- Augmented with flip/rotate/lighting variations
- Special training on dark boards, unconventional themes, and low-light images

Challenges

Some chess sets have similar shapes (e.g., black pawns vs black bishops in bad lighting).
Perspective distortion makes squares uneven.
Large boards in mobile screenshots require high-resolution inference.

Results

My best YOLOv8 model achieves:

mAP50: ~0.98
mAP50-95: ~0.84
High accuracy even with dark backgrounds

🧩 4. Converting Detection Output to FEN

This is the magical part.

Once pieces are detected:

The board segmentation mask is used to warp the image into a perfect 8×8 grid.
Each square is assigned a piece (if detection center falls inside).
Empty-square compression creates the FEN row string.
The final FEN is returned to frontend.

Example output:

7N/5p2/1p3P2/p7/kp6/1p6/1P6/1K6 w - - 0 1

You can then analyze or edit the board directly.

♜ 5. PGN Analyzer + Move Classification

Another major feature I added:

✔ Analyze any PGN

I process each move with Stockfish and compare:

Eval before
Eval after
Best engine move
Centipawn loss
Sacrifices
Move difficulty

Then I classify moves in categories inspired by Chess.com:

Brilliant
Great Move
Good Move
Book Move
Inaccuracy
Mistake
Blunder
Miss (Missed tactic)

This gives players a clean, human-friendly understanding of their game quality.

🎨 6. UI/UX

The UI focuses on clarity and speed:

Drop zone for uploading images
Real-time board preview
FEN editor
Move list with evaluation graph
Best move suggestions
Dark/Light themes
Full mobile support

I also added a Chess Scanner logo featuring a board + camera icon.

🚀 7. Deployment

Technical deployment stack:

Ubuntu 22.04 VPS
Nginx → Gunicorn → FastAPI
Project served as /api/infer endpoint
React frontend deployed with Nginx static hosting
SSL via Certbot
YOLO model files optimized with half precision

Everything runs smoothly even on a mid-tier VPS.

📈 8. What’s Next?

Future improvements:

🔹 Real-time video scanning
🔹 Automatic PGN generation from detected positions
🔹 Opening explorer integration
🔹 ELO estimation based on move accuracy
🔹 Cloud account system + saved games
🔹 Mobile app (React Native, already under development)

🔗 Live App: Chess Scanner

🏁 Conclusion

Building a chess scanner that works reliably on ANY image was a challenging but rewarding project.

The combination of:

YOLOv8
Smart board geometry
Stockfish evaluation
React UX

makes the tool extremely powerful for:

Chess learners
Coaches
Content creators
Daily puzzle solvers

If you want to try it or want the repo, feel free to reach out.

🚀 EngageSwap — A Free Platform to Promote Your Website and Earn Coins

MD ABUBAKAR — Wed, 22 Oct 2025 04:26:59 +0000

🚀 What is EngageSwap?

EngageSwap is a free website-promotion platform I built to help creators, small businesses, and developers get real visitors without spending on ads.

Promote your website for free and get real visitors today.

Try EngageSwap Now

You can:

💰 Earn coins by visiting other websites
🌐 Spend coins to promote your own
🔁 Enjoy a fair exchange system that rewards genuine engagement

Everything happens automatically through our smart system that ensures authentic visits , quiz-based validation , and anti-bot checks .

💡 Why I Built This

While testing my own marketing campaigns, I realized how difficult it is for new websites to get real visitors.
Buying ads is expensive. SEO takes months. So I wanted to create a mutual-growth ecosystem where everyone benefits — whether they want traffic or exposure .

🛠️ Tech Stack

Frontend: React + Tailwind
Backend: Node.js + Express + MySQL
Auth & Payments: OTP-based login + Razorpay integration
Hosting: Nginx reverse proxy on Ubuntu VPS

🌟 Key Features

✅ Real-time click tracking
✅ Quiz-based verification to avoid fake visits
✅ Referral & reward system
✅ Wallet-based coin economy
✅ Admin dashboard to manage users, campaigns & analytics

🔗 Try It Out

👉 Visit EngageSwap 🚀

Sign up, add your website, and start promoting instantly.
You can even earn coins by simply browsing — it’s completely free.

💬 Feedback Welcome!

This is still a growing project, and I’d love your feedback!
Would you use a system like this for your own site?
Drop your thoughts in the comments 👇

DEV Community: MD ABUBAKAR