The latest articles on DEV Community by Uddipan Bhattacharyya (@uddipan_bhattacharyya). https://dev.to/uddipan_bhattacharyya https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4049767%2F496d7c81-a01a-4d8c-a256-dac0806a94b8.jpg https://dev.to/uddipan_bhattacharyya en Uddipan Bhattacharyya Mon, 27 Jul 2026 15:17:43 +0000 https://dev.to/uddipan_bhattacharyya/the-day-i-learned-to-respect-database-indexes-the-hard-way-29k9 https://dev.to/uddipan_bhattacharyya/the-day-i-learned-to-respect-database-indexes-the-hard-way-29k9 <p>In 2021, I graduated from my university and joined a startup as its first employee. That time, I learned a lot of things. I picked up the full stack from scratch, Flutter, Node.js, MongoDB, all of it. I learned fast and I was building things on my own within a few months.</p> <p>But there was one thing nobody told me to learn, and I never thought about it myself, database optimization.</p> <h2> The Launch Was Three Days Away </h2> <p>The company got a big project, and we worked on it for months. One day, we found out the launch was only three days away.</p> <p>The app had three levels of users. The lowest level users were added by us directly, and as part of that work, I uploaded more than 4 lakh user accounts into the database before the launch.</p> <p>The plan for the event was simple. On launch day, the mid level users would sign up on their own, live, in front of everyone.</p> <h2> The Server Started Crashing </h2> <p>Launch day came. The mid level users started registering.</p> <p>After about 50 people signed up, the server began to crash.</p> <p>We had no way of knowing how many people would try to register that day, so we never scaled the database server beforehand. To make it worse, we were running the database on an AWS free tier EC2 instance.</p> <p>While the server kept crashing, I checked everything I could think of. That's when I noticed it, the CPU was sitting at 100%.</p> <p>Then it hit me. I had forgotten to add an index for one important collection.</p> <p>With more than 4 lakh records in that collection, every single query, even a basic one like checking if a user exists, had to go through the whole collection one by one. No index meant no shortcut, just the CPU doing all the work, and it couldn't keep up.</p> <h2> Somehow, We Pulled It Off </h2> <p>There was no time to panic. People were there, the launch was happening in real time, and we had to fix it fast. We scaled up the database server and added the missing index, and somehow it worked. The event went through successfully.</p> <p>But that moment stayed with me long after the day was over.</p> <h2> What I Learned From It </h2> <p>Reading about indexes in a tutorial is one thing. Watching a live server crash in front of 200 people because of a missing one is a completely different lesson.</p> <p>Since that day, I always check for indexes first, on every project, no matter how small it is or how few users it has. I never want to feel that panic again.</p> <p>It has been 5 years now, and the project is still running, served more than 5 million users on it today. After that day, we kept optimizing many things, we added Redis and a few other tools, and now we run the whole system at a very low cost, around 0.05 rupees per user in our monthly bill.</p> database mongodb performance startup