I Built an S3-Compatible Storage Server in Go — Here's What I Learned

Aman Kumar — Mon, 08 Jun 2026 10:45:43 +0000

Why Build Another S3 Clone?

Two reasons:

I wanted to understand distributed storage — not just read about it, but actually build it
AWS bills are scary — paying ₹25k/month for storage hurts when you're a solo developer

So I spent 2 weeks building my own S3-compatible storage server in Go. Here's what happened.

The Stack

Component	What I Used
Language	Go (1.22+)
Database	PostgreSQL (10 tables, auto-migrations)
Caching	Redis
Container	Docker + Docker Compose
Orchestration	Kubernetes manifests
Metrics	Prometheus
API	S3-compatible REST

Features I Built

🔐 JWT Auth + RBAC

I implemented JWT with access/refresh token rotation. Users can have different roles with different permissions — admin, read-only, bucket-specific access. IAM-style policy engine built from scratch.

💾 Erasure Coding (Reed-Solomon)

Your file is split into 4 data shards + 2 parity shards. Even if 2 disks die, your data survives. No expensive RAID hardware needed.

📤 Multipart Upload

Large files are uploaded in parallel parts. Each part is independently stored and reassembled on download. Essential for files > 100MB.

📊 Prometheus Metrics

Every request is tracked — latency histograms, error rates, storage usage, active connections. Grafana dashboard ready.

🔍 Content Deduplication

SHA-256 hashing ensures duplicate files are stored once. Saves significant space in real-world usage.

🚦 Graceful Shutdown

SIGTERM/SIGINT handling with connection draining. Zero-downtime deployments.

The Architecture

┌──────────────────────────────────────────┐
│            HTTP/REST API                 │
├──────────────────────────────────────────┤
│      Auth & Authorization Layer          │
├──────────────────────────────────────────┤
│        Business Logic Layer              │
├──────────────────────────────────────────┤
│           Storage Layer                  │
│  Chunking · Dedup · Erasure · Replication│
├──────────────────────────────────────────┤
│     Infrastructure                       │
│  PostgreSQL │ Redis │ Local Disk Store   │
├──────────────────────────────────────────┤
│     Observability                        │
│  Metrics │ JSON Logs │ Audit Trails      │
└──────────────────────────────────────────┘

Quick Start

git clone https://github.com/aman179102/distributed-file-storage-service.git
cd distributed-file-storage-service
docker compose up -d

curl http://localhost:8080/health

That's it. Your own S3. Running. Free.

What I Learned Building This

Distributed systems aren't magic — they're just good engineering decisions stacked together
Go is perfect for this — concurrency, performance, simple deployment
PostgreSQL can handle more than you think — 10 tables, smooth migrations
Building > Reading — I learned more in 2 weeks of building than 2 months of tutorials

What's Next

[ ] S3 API compatibility (ListObjects, DeleteObjects, etc.)
[ ] Web UI for file management
[ ] S3-to-S3 replication (backup to real AWS)
[ ] MinIO client compatibility

Get The Code

🔗 GitHub: Link

If you found this useful, star the repo ⭐ — it helps more people discover it.

Drop a comment if you've built something similar or have questions. Let's learn together. 🚀

From ML Model to Deployable AI App – MoodSense AI

Aman Kumar — Sat, 11 Apr 2026 11:24:13 +0000

I built an end-to-end NLP project that detects mood from text and turns it into a usable application.

Key features:

Multi-class mood classification
Confidence score + probability distribution
Recommendation system
FastAPI backend + Gradio UI
Live deployment

Live: Click Me

GitHub: Click Me

Big takeaway: ML is just one part — productization is what really matters.

From CLI to GUI: Building an AI Podcast Studio (PodVoice)

Aman Kumar — Sun, 29 Mar 2026 08:18:05 +0000

I originally built PodVoice as a CLI tool to convert Markdown into multi-speaker audio using Coqui XTTS.

While it worked well, usability was a challenge.

So I built PodVoice Studio — a web-based GUI on top of it.