DEV Community: siddarthpatelkama

What Exhausting 50,000 Firestore Reads Taught Me About Database Design

siddarthpatelkama — Wed, 02 Sep 2026 04:56:03 +0000

How I redesigned the data layer of my anti-proxy attendance system around access patterns, security, offline synchronization, and efficient Firestore usage.

When I first built my attendance system, I wasn't thinking much about database architecture.

I had a working MVP.

Students could log in.
Teachers could create attendance sessions.
Students could scan dynamic QR codes.
Attendance records were stored in Firestore.

It worked.

Until I started using it heavily.

Then I learned a lesson that only becomes obvious after something breaks:

Your database structure isn't just about how you store data. It's about how your application needs to retrieve it.

And in my case, that lesson started with 50,000 Firestore reads.

The Problem Wasn't Just the Quota

In an earlier part of this series, I wrote about how I unexpectedly exhausted my Firestore free-tier reads while developing and testing the system.

That incident pushed me to build offline capabilities and start reducing unnecessary database operations.

But it also exposed a deeper problem.

I had been thinking about Firestore mainly as:

"Where should I store this data?"

I needed to start thinking:

"What data will my application need to retrieve, how often, and in what combination?"

That shift changed how I looked at the entire data layer.

I Stopped Thinking in Tables

Coming from SQL-style database thinking, my first instinct was to model everything around entities.

Students.

coordinaters.

Attendance.

meetings.

Sessions.

That isn't necessarily wrong.

But Firestore isn't a relational database with tables and joins.

It is a document-oriented database built around collections and documents, with support for nested structures and subcollections.

So instead of asking:

"What are all my entities?"

I started asking:

"What does the application actually need to read?"

That sounds like a small difference.

It wasn't.

I Started With Access Patterns

Consider what happens when a student tries to mark attendance.

The system needs to establish several things:

Who is this user?
Is the authentication valid?
Is the dynamic QR valid?
Is this device associated with the account?
Is there an active attendance session?
Has this student already marked attendance?
Should an attendance record be created?

The important thing here is that these aren't independent database operations.

They're part of one application workflow.

That meant the database structure had to support the workflow rather than forcing the application to repeatedly search for unrelated pieces of information.

Attendance Isn't Just a Property of a Student

One of the most important modeling decisions was treating attendance as its own record.

A student isn't simply:

Student
attendance = 87%

That number doesn't tell me much.

I need to know:

Which meeting?
Which session?
Which date?
Was the attendance marked successfully?
What authentication flow was used?
Was the request associated with the expected device?
When was it recorded?

Attendance is really an event.

And events deserve their own records.

This also gives the application something much more useful to work with.

Instead of asking:

"What is this student's attendance?"

the system can ask:

"What attendance events belong to this student?"

That distinction becomes increasingly important as the application grows.

The Database Became Part of the Security Pipeline

This was another thing I didn't appreciate when I started.

My dynamic QR system already had multiple layers of verification.

But authentication doesn't end when the QR is scanned.

The request still needs to move through the backend before attendance is accepted.

The general flow became:

Student
↓
User Authentication
↓
Dynamic QR Validation
↓
Device Verification
↓
Backend Validation
↓
Firestore
↓
Attendance Record

The database isn't responsible for deciding whether a student is trustworthy.

But it stores the state that the backend needs to make that decision.

That makes the data model part of the security architecture.

And this is one reason I became much more careful about where authentication-related state lived.

I Learned Not to Read Everything

This sounds obvious.

But it's surprisingly easy to accidentally build an application that retrieves far more data than it needs.

For example, imagine a dashboard that only needs the current attendance session.

There is no reason for the application to retrieve an entire history of attendance records just to determine what is happening right now.

The same principle applies everywhere:

Don't ask:
"How much data can I retrieve?"

Ask:
"What's the smallest amount of data this operation needs?"

Firestore supports document-level queries, filtering, sorting, limits, and pagination mechanisms, which makes this style of access-pattern-driven design possible.

This became particularly important after my 50,000-read incident.

Every unnecessary read stopped feeling harmless.

Offline Support Changed the Data Flow

Then there was another complication.

The attendance system couldn't assume that the network would always be available.

That meant the architecture eventually needed to support a flow like:

User Action
↓
Local State
↓
Pending Attendance
↓
Network Available?
↙ ↘
YES NO
↓ ↓
Firestore Keep Locally
↓
Sync Later

Now the database wasn't simply receiving requests from the frontend.

There was a synchronization problem too.

An attendance action could exist locally before it existed remotely.

That forced me to think about things like:

pending records
duplicate submissions
synchronization
conflict handling
what happens when the network comes back
how the backend should validate synced data

The database architecture therefore had to coexist with the offline architecture.

Security and Performance Pulled in Different Directions

This was probably one of the more interesting tradeoffs.

Security wants more validation.

Performance wants fewer operations.

Offline support wants local state.

Consistency wants reliable synchronization.

And Firestore usage needs to remain efficient.

You can't optimize one dimension without considering the others.

For example:

More validation
↓
More reads / writes
↓
Potentially higher latency

Less validation
↓
Fewer operations
↓
Potentially weaker verification

So the goal wasn't:

"Use the fewest Firestore operations possible."

The goal was:

"Use the database operations that actually contribute to the correctness and security of the workflow."

That's a much better optimization target.

The 50,000 Reads Incident Changed How I Debug

Before the quota incident, I mostly looked at whether the feature worked.

After it, I started asking different questions.

When something happened in the UI:

What triggered this read?

Why was this document needed?

Could this have been cached?

Did the application already have this information?

Could I retrieve a smaller result?

Is this happening once or repeatedly?

That mindset is useful beyond Firestore.

A slow application isn't always slow because the database itself is slow.

Sometimes the application is simply asking the database the same question over and over.

I Started Treating the Data Layer as Architecture

This became the biggest lesson from the whole process.

Initially, my architecture looked conceptually like:

Frontend
↓
Backend
↓
Firestore

But the real system became closer to:

                ┌──────────────┐
                │ Authentication│
                └──────┬───────┘
                       ↓

┌────────────┐ ┌───────────────┐
│ Client │ ───→ │ Backend │
└─────┬──────┘ └───────┬───────┘
│ │
│ ↓
│ ┌─────────────┐
│ │ Firestore │
│ └──────┬──────┘
│ │
↓ ↓
Local State ←────── Synchronization

Firestore was no longer just "the place where I save attendance."

It became one component in a larger system involving:

authentication
authorization
attendance sessions
device state
offline synchronization
backend validation
caching
read/write optimization

That changed how I designed everything around it.

What I'd Do Differently If I Started Again

If I rebuilt the system from scratch, I would design the data layer before building most of the UI.

Not because I want a perfect schema from day one.

Because I'd want to understand the application's major access patterns first.

I'd write down questions like:

What does a student need to read?

What does a teacher need to read?

What must the backend validate?

What data changes frequently?

What data can be cached?

What must work offline?

What needs to be queried by date?

What needs to be queried by student?

What needs to be queried by attendance session?

Then I'd design the documents and queries around those questions.

That would have saved me a lot of unnecessary iteration.

The Bigger Lesson

I originally thought my biggest Firestore lesson would be:

"Don't exceed your free-tier quota."

It wasn't.

The bigger lesson was:

Database design starts with application behavior, not just data structure.

Once I understood how the application actually read and wrote data, several architectural decisions became easier.

The 50,000-read incident forced me to care about database usage.

Offline support forced me to care about synchronization.

Device authentication forced me to care about persistent identity state.

And together, they pushed me toward thinking about Firestore as part of the architecture rather than just a storage layer.

That's probably the biggest change in how I build systems now.

I don't start by asking:

"How should I store this?"

I start by asking:

"How will the system use this?"

And then I design the data around that answer.

What's Next?

So far, this series has gone from:

Dynamic QR authentication → Firestore quota problems → offline capability → device-bound authentication → database architecture

But there was another problem hiding in the system.

Even when the database and backend were working correctly, the application could still feel slow.

Especially when the backend had to wake up.

That led me to another engineering problem:

How do you make a backend with cold starts feel fast to the user?

That's what I'll cover in the next part.

The QR Code Wasn't Enough: How I Added Device-Bound Authentication

siddarthpatelkama — Sun, 09 Aug 2026 05:34:45 +0000

How I added another layer of verification to make account sharing and proxy attendance harder.

The Problem

In my previous article, I explained how I used dynamic QR authentication to reduce one of the simplest forms of proxy attendance:

screenshot and share the QR code.

But that introduced another question.

What if the student shares their account instead?

Consider this scenario:

Classroom
Student A
   │
   └── Has valid account + trusted device

Hostel
Student B
   │
   └── Gets Student A's credentials
   │
   └── Logs in from another phone
   │
   └── Attempts attendance

The QR code can be valid.

The credentials can be valid.

The user can be authenticated.

And the attendance request can still be illegitimate.

So I needed another verification layer.

Authentication Isn't the Same as Device Trust

Traditional authentication primarily answers:

Who is this user?

For this attendance system, I also wanted to answer:

Is this request coming from the device associated with this account?

That led me to implement device-bound authentication.

The idea is straightforward:

User Account
     │
     ▼
Trusted Device
     │
     ▼
Attendance Request
     │
     ▼
Backend Verification

The device becomes another signal that the backend can use when deciding whether an attendance request should continue.

The Device Registration Flow

At a high level, the flow looks like this:

First Login
    │
    ▼
Generate / Retrieve Device Identifier
    │
    ▼
Associate Device With Account
    │
    ▼
Store Device Association
    │
    ▼
Future Requests
    │
    ▼
Backend Compares Device Identity
    │
    ├── Match ────────► Continue
    │
    └── Mismatch ─────► Reject

The important part is that the final decision happens on the backend.

The client shouldn't be able to simply say:

"Trust me, this is my device."

The backend needs to validate the request against the device association it already knows about.

The 1-to-1 Device Lock

In my implementation, the relevant student flow uses a strict one-to-one device association.

The repository uses the persistent identifier:

uba_emergency_device_id

to maintain that device association.

Conceptually:

Student A
    │
    └── Device A ✅

But:

Student A
    │
    ├── Device A ✅
    │
    └── Device B ❌

The important distinction is that having valid credentials doesn't automatically make a new device trusted.

Where Device Verification Fits

Device binding doesn't replace dynamic QR authentication.

It adds another layer to it.

The attendance request becomes something closer to:

Attendance Request
        │
        ▼
User Authentication
        │
        ▼
Dynamic QR Validation
        │
        ▼
Device Verification
        │
        ▼
Backend Validation
        │
        ▼
Attendance Recorded

Each layer addresses a different problem.

Dynamic QR

Makes previously captured QR codes much less useful.

Authentication

Establishes the student's account identity.

Device verification

Makes casual account sharing across devices harder.

Backend validation

Keeps the final decision on the server.

Why Not Just Trust the Client?

This is an important principle I learned while building the system:

The client should provide information. The backend should decide whether that information can be trusted.

A device identifier coming from the browser isn't magically secure.

A determined attacker may be able to manipulate client-side identifiers depending on the environment.

That's why I don't treat device binding as an unbreakable security mechanism.

It's another signal.

The goal is not:

"Make proxy attendance impossible."

The goal is:

"Make casual proxy attendance significantly harder without making legitimate attendance painful."

The Recovery Problem

Device binding immediately creates another problem.

What happens when the legitimate user changes devices?

Phones get:

Lost
Replaced
Reset
Repaired
Reconfigured

Browser storage can also disappear.

A security mechanism without recovery can become worse than the problem it was designed to solve.

So a real implementation has to consider:

Trusted Device
      │
      ├── Device lost
      ├── New phone
      ├── Storage reset
      └── Device replacement
              │
              ▼
        Recovery Process

This is one of the areas where authentication becomes a system-design problem rather than just an if statement.

Security vs Usability

Every additional security layer introduces friction.

More checks can mean:

More security

but also:

More ways to accidentally block legitimate users.

For this project, I think about the trade-off as:

Security
   ×
Usability
   ×
Recoverability

All three matter.

A system that blocks every suspicious request but also locks legitimate students out isn't a good attendance system.

Likewise, a system that is extremely convenient but accepts every request isn't doing much to prevent proxy attendance.

What This Doesn't Protect Against

It's important not to oversell device binding.

It can help reduce:

Basic account sharing
Casual multi-device access
Credential-based proxy attendance
Repeated use of one student's account from another device

But it doesn't automatically prevent:

Sophisticated attackers
Compromised devices
Manipulation of client-side identifiers
Someone physically using the registered device

Security is about the threat model.

For my project, I'm primarily trying to prevent ordinary proxy attendance, not defend against a nation-state with a lab full of reverse engineers.

The Bigger Lesson

When I started this project, I thought proxy attendance was mainly a QR-code problem.

It wasn't.

It became an identity problem.

The QR tells the system about the attendance session.

Authentication tells it about the student.

Device verification provides another signal about the device making the request.

Backend validation brings those signals together.

That led me to a broader lesson:

Security rarely comes from one perfect mechanism. It comes from multiple layers that compensate for each other's weaknesses.

What's Next?

Device-bound authentication solved another part of the proxy-attendance problem.

But it also introduced more engineering questions.

How should these validations interact with the database?

How do you minimize Firestore reads while performing multiple checks?

How should suspicious device changes be logged?

And how do you design the backend so that adding security doesn't make every attendance request unnecessarily expensive?

Those are the problems I'm exploring as I continue building the system.

Final Thoughts

I started with a simple question:

"How do I stop someone from sharing a QR code?"

Then it became:

"What stops them from sharing an account?"

And eventually:

"How do I know the request is coming from a trusted device?"

That's been one of the most interesting parts of building this project.

The architecture wasn't designed perfectly on day one.

It evolved as each new problem exposed another weakness.

And honestly, that's probably the most realistic way to build software.

Project

🔗 GitHub: https://github.com/siddarthpatelkama/UBA-veltech-attendance-system

💼 LinkedIn: https://www.linkedin.com/in/siddarthpatelkama

I Didn't Plan to Build an Offline-First Attendance System. Firestore Forced Me To.

siddarthpatelkama — Fri, 24 Jul 2026 19:39:11 +0000

How accidentally exhausting Firestore's free-tier limit transformed a rushed MVP into a more resilient anti-proxy attendance system.

Introduction

When I started building an attendance system for my university's UBA (Unnat Bharat Abhiyan) club, I had one clear instruction from our coordinator:

"Build an MVP as soon as possible."

Like most student developers, I focused on getting the features working first.

The application already had everything I needed for the MVP:

Dynamic QR-based attendance
Firebase Authentication
Firestore database
Anti-proxy validation
Admin dashboard
Meeting management

My daily workflow looked something like this:

Code
↓
Run
↓
Test
↓
Scan QR
↓
Debug
↓
Repeat

Everything seemed to be progressing smoothly.

What I wasn't paying attention to was something happening quietly in the background.

Every refresh.

Every API call.

Every test.

Every debugging session.

All of them were consuming Firestore document reads.

The Problem I Never Expected

One day, while testing the application, things suddenly stopped working.

Attendance requests started failing.

Firestore wasn't responding.

Like most developers, my first thought was:

"I definitely broke something."

I checked my recent commits.

I reviewed the API routes.

I verified Firebase Authentication.

I even looked through my Firestore Security Rules.

Everything looked perfectly fine.

Then I opened the Firebase Console.

That's when I saw it.

Firestore Reads: 50,000 / 50,000

I had exhausted the entire free-tier quota.

Not because thousands of students were using my application.

Not because it had gone viral.

Simply because I had spent days developing and debugging it.

Ironically, I had become my own biggest user.

The Easy Fix Wasn't the Right Fix

Technically, I could have waited for the daily quota to reset.

Problem solved.

But this wasn't just another side project.

It was an attendance system intended to be used during actual UBA meetings.

If Firestore became unavailable during a meeting, attendance couldn't simply stop.

So instead of asking,

"How do I reduce Firestore reads?"

I asked a different question.

"How do I make the system continue working even when Firestore isn't available?"

That single question completely changed the architecture of the project.

Offline-First Was Never the Plan

Before this happened, offline support wasn't even on my roadmap.

Every attendance request depended on Firestore.

Every scan required the backend.

Which meant one dependency controlled whether the entire application worked.

I didn't like that.

So I redesigned the workflow.

Instead of immediately writing attendance to Firestore, the application could now continue functioning locally whenever the backend wasn't available.

Attendance records would be stored securely on the device and synchronised automatically once connectivity or database access returned.

The project unexpectedly became offline-first.

But There Was One Catch...

Building offline support for a normal CRUD application is relatively straightforward.

Building it for an anti-proxy attendance system isn't.

Offline mode couldn't become an easy way to bypass the security model.

The system still had to guarantee:

Attendance integrity
Duplicate prevention
Secure local storage
Reliable synchronisation
Consistent validation after syncing

The goal wasn't simply to make the application work offline.

It was to ensure the same anti-proxy workflow remained intact whether the application was online or offline.

That made the redesign much more interesting than simply adding a cache.

Solving the Symptom vs Solving the Cause

The offline architecture solved the immediate problem.

But after everything was working again, I realised something.

Running out of Firestore reads wasn't the real problem.

My application was.

I had built the MVP quickly.

I hadn't built it efficiently.

That led me to review every interaction between the application and Firestore.

Questions I had never asked before suddenly became important.

Do I really need this document read?
Why am I fetching the same document multiple times?
Can this data be cached?
Can I combine multiple reads into one request?
Is every database write actually necessary?

Those questions ended up improving the architecture far more than any new feature.

Optimising Firestore Usage

Once the offline system was stable, I shifted my attention to optimisation.

I reduced unnecessary Firestore reads.

I optimised database queries.

I removed redundant writes wherever possible.

I introduced Render's free cache to store frequently accessed data, reducing repeated requests for information that rarely changed.

These weren't flashy improvements.

Users would never notice them directly.

But together, they made the application significantly more efficient and much less dependent on Firestore's free-tier limits.

What I Learned

When I started this project, I thought software engineering was mostly about building features.

This experience taught me something different.

Sometimes the most important engineering decisions don't happen while adding new features.

They happen when your assumptions break.

Running out of Firestore reads forced me to think beyond "making it work."

It pushed me to think about resilience.

Reliability.

Efficiency.

And designing systems that continue working even when one of their dependencies doesn't.

Looking back, hitting Firestore's limit was frustrating.

Today, I'm actually grateful it happened.

Without that unexpected limitation, I probably would never have redesigned the application into something far more resilient than the MVP I originally planned.

Final Thoughts

This project began as a simple anti-proxy attendance system for my university.

It unexpectedly became a lesson in building software that can survive constraints.

Sometimes architecture evolves because of careful planning.

Sometimes it evolves because reality forces you to rethink everything.

For me, it took exactly 50,000 Firestore document reads to learn that lesson.

And honestly, I wouldn't build it any other way now.

Thanks for Reading! 🚀

Have you ever hit a cloud free-tier limit, API quota, or infrastructure constraint that completely changed the way you designed your application?

I'd love to hear your experience in the comments. Some of the best engineering lessons come from the problems we never planned for.

How I Designed an Anti-Proxy Attendance System Using Dynamic QR Authentication

siddarthpatelkama — Mon, 29 Jun 2026 12:45:23 +0000

"Anyone can generate a QR code. Designing one that actually prevents misuse is the hard part."

When people hear "QR-based attendance," the first question is usually:

"Can't someone just take a photo of the QR code and send it to their friend?"

The answer is: yes, if the system is designed poorly.

That question became one of the biggest engineering challenges while building my attendance platform.

Instead of thinking about QR codes as images, I started thinking about them as temporary authentication tokens.

The Problem

Most QR attendance systems work like this:

Teacher generates a QR code.
Students scan it.
Attendance is marked.

Simple.

Unfortunately, it's also vulnerable.

A student can:

Photograph the QR code.
Send it through WhatsApp.
Another student scans it from somewhere else.
Attendance gets recorded.

The QR code itself becomes the vulnerability.

Rethinking the QR Code

Instead of creating a QR code that simply points to an attendance page, I designed it to behave more like a short-lived authentication token.

Every QR code is generated dynamically and remains valid only for a limited period (in my implementation, I set this to 11 seconds).

Once it expires, the system rejects it.

This dramatically reduces the opportunity for someone to reuse an older QR code.

Authentication Beyond the QR

Scanning the QR code alone isn't enough.

The backend performs additional validation before marking attendance.

Rather than trusting the client, the server becomes the source of truth.

This reduces opportunities for client-side manipulation and keeps the attendance workflow under server control.

Designing for the Real World

One challenge I encountered was balancing security with usability.

If the QR expires too quickly, students may struggle to scan it in time.

If it stays active for too long, it becomes easier to misuse.

Finding the right balance required thinking about actual classroom conditions rather than only technical implementation.

Lessons Learned

One lesson stood out throughout this project:

Security isn't usually one big feature.

It's the result of multiple smaller decisions working together.

A dynamic QR code helps.

Server-side validation helps.

Short-lived sessions help.

Each layer reduces risk.

What's Next?

This article only covers one piece of the system.

In future posts, I'll explore:

Backend architecture
Firestore data modeling
Authentication flow
Scaling the platform
Engineering decisions behind the project

Building software has taught me that writing code is only part of engineering.

Designing systems that continue to work under real-world conditions is where the interesting problems begin. In this project, I used libraries like qrcode on the backend to generate dynamic QR codes and html5-qrcode on the frontend to scan and verify them efficiently, while SHA-256 hashing was used on the backend to securely generate and validate the QR token payloads.

⭐ GitHub: https://github.com/siddarthpatelkama

💼 LinkedIn: https://www.linkedin.com/in/siddarthpatelkama

Building an Anti-Proxy -Offline-First Attendance Platform with Next.js, Express & Firebase

siddarthpatelkama — Fri, 26 Jun 2026 12:10:14 +0000

Traditional attendance systems seem simple until you look at how they work in practice. Manual roll calls consume valuable class time, paper attendance sheets are inefficient, and many QR-based systems can still be exploited through proxy attendance.

I wanted to explore whether a better solution was possible.

This project is my attempt at designing and building an offline-first anti-proxy attendance platform from scratch as a solo developer. The goal wasn't just to digitize attendance, but to create a system that remains reliable in real-world environments while prioritizing security, scalability, and a smooth user experience.

The platform is built using Next.js 16, React 19, TypeScript, Express.js, Firebase Authentication, Firestore, Firebase Admin SDK, Tailwind CSS, Capacitor, Progressive Web Apps (PWA), Sentry, and dynamic QR-based authentication.

Rather than focusing only on features, this article explores the engineering decisions behind the project: why I chose an offline-first architecture, how dynamic QR validation improves security, the reasoning behind the technology stack, the challenges I faced during development, and the lessons I learned while building a production-style application that is already being used in my college club.

The project is still evolving, and I continue to improve it whenever I have time. My aim is to keep documenting that journey by sharing not only what works, but also the mistakes, trade-offs, and architectural decisions that shaped the system.

If you're interested in full-stack development, system design, Next.js, Firebase, TypeScript, or building software that solves practical problems, I hope you'll find something valuable in this article.

⭐ GitHub: https://github.com/siddarthpatelkama

💼 LinkedIn: https://www.linkedin.com/in/siddarthpatelkama

I'm always open to feedback, suggestions, and discussions with fellow developers. If you have ideas for improving the project or questions about the implementation, I'd love to hear from you.