DEV Community: Jacob Alcock

Model Collapse: The AI Feedback Loop Problem Nobody Wants to Talk About

Jacob Alcock — Mon, 10 Nov 2025 13:31:41 +0000

AI models are eating their own tail, and it's going to be a problem.

The entire premise of modern LLMs is that they're trained on human-generated content. Books, articles, research papers, Stack Overflow answers, GitHub repositories - billions of tokens of actual human knowledge. But that assumption is breaking down faster than anyone wants to admit.

The Core Issue

As we approach the end of 2025, the web is saturated with AI-generated content:

Stack Overflow answers copy-pasted from ChatGPT
GitHub repos with AI-generated documentation and comments
Blog posts churned out by content farms using GPT
Social media posts from bots
Technical articles written entirely by LLMs

Yet AI companies still scrape the web for training data. They can't reliably distinguish human content from synthetic content. Which means the next generation of models will inevitably train on the outputs of previous models.

This is model collapse. And it's not theoretical - it's measurable, reproducible, and already happening.

How Model Collapse Works

The feedback loop is straightforward:

Gen 1: Train on 95% human data, 5% AI slop → minor quality issues
Gen 2: Train on 80% human data, 20% AI content → noticeable degradation
Gen 3: Train on 60% human data, 40% AI outputs → significant problems
Gen 4: Train on majority AI-generated content → model collapse

Each generation compounds the problems:

Loss of diversity - outputs converge toward homogeneous, repetitive patterns
Amplified biases - quirks from previous models get magnified
Increased hallucinations - errors stack across generations
Tail knowledge disappears - rare but critical information gets filtered out first

It's the same principle as photocopying a photocopy. Each iteration degrades the original.

Why You Should Care

Code quality degradation

If Copilot trains on AI-generated code that was itself generated by an earlier model, code suggestions degrade. You're not getting patterns from experienced developers anymore - you're getting averaged-out slop that "looks" like code.

Security implications

AI-assisted security tools trained on AI-generated vulnerability analyses will miss things. If the training data is full of hallucinated CVE details or incorrect exploit explanations, the model learns wrong information.

Knowledge erosion

Niche technical knowledge - the kind buried in obscure forum posts, old mailing lists, and forgotten documentation - disappears first. AI models optimise for common patterns. Rare but critical knowledge gets filtered out.

Trust degradation

You can't tell anymore if that blog post explaining a security vulnerability was written by someone who actually found and tested it, or by an LLM that pieced together fragments from six different sources and hallucinated the rest.

Proposed Solutions (And Why They're All Flawed)

Watermarking

Embed cryptographic signatures in AI outputs to filter them during training. Google and OpenAI are researching this. Problem: watermarks can be stripped. It's an arms race.

Provenance tracking

Track the origin of all training data. Only use verified human content. Problem: doesn't scale. The entire value proposition of LLMs is training on massive web-scale datasets.

Curated datasets

Stop scraping the web entirely. Build human-verified, high-quality datasets. Problem: expensive, slow, and fundamentally limits what the model can learn.

Adversarial filtering

Train models to detect and exclude AI-generated text. Problem: classic adversarial arms race. Detection improves, generation improves to evade detection, repeat forever.

Controlled synthetic mixing

Carefully balance the ratio of real to synthetic data. Problem: requires knowing the exact contamination threshold, which varies by domain and model architecture.

None of these solve the core issue. And we might already be past the point of no return. The web is saturated with AI slop. Even if filtering started today, there are years of contamination already baked into datasets.

The Actual Problem

We're running a one-way experiment on the future of LLMs, and nobody knows the safe parameters.

No one knows what percentage of AI contamination causes collapse. No one knows if current models are already degraded. No one knows how to reverse contamination once it's in the dataset.

LLMs were built on the assumption of abundant, renewable human knowledge. But that assumption was wrong. We're strip-mining the web for training data, and the mine doesn't refill. Every piece of human writing that gets replaced with AI slop permanently degrades the training pool.

The Economic Incentive Problem

The economics make this worse. AI companies have no incentive to solve this:

Scraping is free (legally questionable, but free)
Filtering costs money
Competition doesn't care about data quality 5 years from now
Investors reward shipping features, not long-term dataset integrity

Publishers can't win either. Paywalling content to prevent scraping also blocks legitimate human readers. Not paywalling means getting drained by RAG systems that plagiarise without attribution.

Content creators lose traffic and revenue to AI summaries. So they either stop producing content (reducing the pool of human knowledge) or start using AI to produce more content faster (contaminating the pool).

It's a race to the bottom, and every participant is incentivised to make it worse.

What Actually Needs to Happen

The realistic options are limited:

Legislation requiring training data transparency - companies must disclose what they trained on and prove licensing rights
Mandatory AI content labeling - cryptographic signatures that can't be easily stripped
Royalty systems for scraped content - similar to how music licensing works
Incentivise human-generated content - platforms that verify and reward genuine human writing

None of this will happen voluntarily. The industry is too profitable and moving too fast. Regulation would need to come first, and regulators barely understand the technology.

More likely: we hit model collapse in 3-5 years, everyone scrambles to fix it retroactively, and we end up with some half-botched solution that only partially works.

Final Thoughts

Model collapse is not a hypothetical future problem. It's happening now, measurably, in controlled experiments. The only question is whether we're already seeing it in production models.

The feedback loop is real. The economic incentives ensure it will continue. And the proposed solutions all have fundamental flaws that make them unlikely to work at scale.

I'm not saying LLMs are doomed. I'm saying the current trajectory is unsustainable, and nobody with the power to fix it has an incentive to do so. The companies building these models are optimising for next quarter's revenue, not training data quality in 2030.

This will either get fixed through heavy-handed regulation, or we'll collectively find out what happens when AI models train on increasingly degraded synthetic data. My money is on the latter.

The snake is already eating its tail. We're just waiting to see how far down it gets before someone notices.

Research:

Is Model Collapse Inevitable? (Matthias Gerstgrasser et al., 2024)
The Curse of Recursion (Ilia Shumailov et al., 2024)
AI models collapse when trained on recursively generated data (Ilia Shumailov et al., 2024)
New research warns of potential ‘collapse’ of machine learning models

How to Write Secure Firebase Rules

Jacob Alcock — Mon, 10 Nov 2025 13:30:40 +0000

Firebase Security Rules are the only thing protecting your data from unauthorised access. This guide covers how to write rules that actually secure your app.

Understanding the Basics

Firebase Security Rules work by matching paths and applying conditions. If the condition evaluates to true, the request is allowed. If false, it's denied.

Cloud Firestore Rules Structure

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    // Your rules go here
    match /collection/{document} {
      allow read, write: if <condition>;
    }
  }
}

Realtime Database Rules Structure

{
  "rules": {
    "path": {
      ".read": "<condition>",
      ".write": "<condition>"
    }
  }
}

Key Concepts

Match blocks: Define which paths the rule applies to
Allow statements: Specify what operations are permitted
Conditions: Boolean expressions that grant or deny access
Variables: request (incoming request data) and resource (existing data)

Rule Methods

Firestore rules support granular methods:

read: Covers both get (single document) and list (queries)
write: Covers create, update, and delete
get: Read a single document
list: Read queries and collections
create: Write new documents
update: Modify existing documents
delete: Remove documents

// Granular control
match /posts/{postId} {
  allow get: if true;  // Anyone can read a single post
  allow list: if request.auth != null;  // Only authenticated users can query
  allow create: if request.auth != null;  // Only authenticated users can create
  allow update: if request.auth.uid == resource.data.authorId;  // Only author can update
  allow delete: if request.auth.uid == resource.data.authorId;  // Only author can delete
}

Common Secure Patterns

Pattern 1: User Can Only Access Their Own Data

Use case: User profiles, private documents, personal settings

Firestore:

match /users/{userId} {
  allow read, write: if request.auth != null && request.auth.uid == userId;
}

Realtime Database:

{
  "rules": {
    "users": {
      "$userId": {
        ".read": "$userId === auth.uid",
        ".write": "$userId === auth.uid"
      }
    }
  }
}

Pattern 2: Public Read, Authenticated Write

Use case: Blog posts, public content, product listings

Firestore:

match /posts/{postId} {
  allow read: if true;
  allow create: if request.auth != null;
  allow update, delete: if request.auth != null
                         && request.auth.uid == resource.data.authorId;
}

Realtime Database:

{
  "rules": {
    "posts": {
      "$postId": {
        ".read": true,
        ".write": "auth != null && (!data.exists() || data.child('authorId').val() === auth.uid)"
      }
    }
  }
}

Pattern 3: Role-Based Access Using Custom Claims

Use case: Admin panels, multi-role applications

Setup custom claims (server-side):

const admin = require('firebase-admin');

// Set custom claims
await admin.auth().setCustomUserClaims(uid, { admin: true });

Firestore rules:

match /adminData/{document} {
  allow read, write: if request.auth.token.admin == true;
}

match /posts/{postId} {
  allow read: if true;
  allow write: if request.auth.token.editor == true
               || request.auth.token.admin == true;
}

Realtime Database:

{
  "rules": {
    "adminData": {
      ".read": "auth.token.admin === true",
      ".write": "auth.token.admin === true"
    }
  }
}

Pattern 4: Data Validation

Use case: Ensuring data format and required fields

Firestore:

match /posts/{postId} {
  allow create: if request.auth != null
                && request.resource.data.keys().hasAll(['title', 'content', 'authorId'])
                && request.resource.data.title is string
                && request.resource.data.title.size() > 0
                && request.resource.data.title.size() < 200
                && request.resource.data.authorId == request.auth.uid;

  allow update: if request.auth != null
                && request.auth.uid == resource.data.authorId
                && request.resource.data.authorId == resource.data.authorId; // Prevent changing author
}

Realtime Database:

{
  "rules": {
    "posts": {
      "$postId": {
        ".write": "auth != null && newData.hasChildren(['title', 'content', 'authorId'])",
        "title": {
          ".validate": "newData.isString() && newData.val().length > 0 && newData.val().length < 200"
        },
        "authorId": {
          ".validate": "newData.val() === auth.uid && (!data.exists() || data.val() === newData.val())"
        }
      }
    }
  }
}

Pattern 5: Attribute-Based Access (Data-Driven Roles)

Use case: Shared documents, team access, permission-based systems

Firestore:

match /projects/{projectId} {
  allow read: if request.auth != null
              && request.auth.uid in resource.data.members;

  allow write: if request.auth != null
               && request.auth.uid in resource.data.admins;
}

Realtime Database:

{
  "rules": {
    "projects": {
      "$projectId": {
        ".read": "auth != null && data.child('members').child(auth.uid).exists()",
        ".write": "auth != null && data.child('admins').child(auth.uid).exists()"
      }
    }
  }
}

Using Functions for Reusable Logic

Functions make rules more maintainable and readable.

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {

    // Check if user is authenticated
    function isSignedIn() {
      return request.auth != null;
    }

    // Check if user owns the resource
    function isOwner(userId) {
      return request.auth.uid == userId;
    }

    // Check if user has a specific role
    function hasRole(role) {
      return isSignedIn() && request.auth.token[role] == true;
    }

    // Validate required fields
    function hasRequiredFields(fields) {
      return request.resource.data.keys().hasAll(fields);
    }

    // Use the functions
    match /users/{userId} {
      allow read: if isSignedIn();
      allow write: if isOwner(userId);
    }

    match /posts/{postId} {
      allow create: if isSignedIn()
                    && hasRequiredFields(['title', 'content', 'authorId'])
                    && isOwner(request.resource.data.authorId);

      allow update: if isOwner(resource.data.authorId);
      allow delete: if isOwner(resource.data.authorId) || hasRole('admin');
    }
  }
}

Handling Subcollections

In Firestore, rules don't cascade to subcollections. You must explicitly define rules for each level.

match /users/{userId} {
  allow read: if request.auth.uid == userId;

  // Subcollection requires its own rules
  match /privateData/{document} {
    allow read, write: if request.auth.uid == userId;
  }

  // Another subcollection
  match /posts/{postId} {
    allow read: if true;  // Public read
    allow write: if request.auth.uid == userId;  // Only owner can write
  }
}

Important: A match like /users/{userId}/{document=**} will match ALL nested subcollections recursively. Use this carefully.

// This matches /users/{userId}/anything/at/any/depth
match /users/{userId}/{document=**} {
  allow read: if request.auth.uid == userId;
}

Realtime Database: Cascading Rules

In Realtime Database, rules CASCADE. Parent rules override child rules.

{
  "rules": {
    "users": {
      // This grants read access to all user data
      ".read": "auth != null",
      "$userId": {
        // This CANNOT restrict the read access granted above
        ".read": "$userId === auth.uid",  // This is IGNORED
        ".write": "$userId === auth.uid"
      }
    }
  }
}

Correct approach: Don't grant broad access at parent levels.

{
  "rules": {
    "users": {
      "$userId": {
        ".read": "$userId === auth.uid",
        ".write": "$userId === auth.uid"
      }
    }
  }
}

Testing Your Rules

Use FireScan

Try out my purpose built tool for auditing firebase infrastructure. It’s completely free, open-source and available for anyone to use. Check it out here.

Use the Firebase Emulator

Install and run locally:

npm install -g firebase-tools
firebase init emulators
firebase emulators:start

Use the Rules Simulator in Firebase Console

Navigate to Firestore/Realtime Database → Rules → Playground

Select operation type (get, list, create, etc.)
Choose authenticated or unauthenticated
Specify the path
Run simulation

This is useful for quick checks but not a substitute for proper testing.

Common Mistakes to Avoid

1. Using `if true` in Production

// NEVER DO THIS
match /{document=**} {
  allow read, write: if true;
}

2. Relying Only on `request.auth != null`

// This allows ANY authenticated user to access ANY data
match /users/{userId} {
  allow read, write: if request.auth != null;  // Too permissive
}

// Better: verify the user matches
match /users/{userId} {
  allow read, write: if request.auth != null && request.auth.uid == userId;
}

3. Forgetting Realtime Database Cascade Rules

{
  "rules": {
    "data": {
      ".read": true,  // Grants read to everything below
      "private": {
        ".read": false  // This is IGNORED, read was already granted above
      }
    }
  }
}

4. Not Validating Data on Create/Update

// Bad: No validation
match /posts/{postId} {
  allow create: if request.auth != null;
}

// Good: Validate required fields and author
match /posts/{postId} {
  allow create: if request.auth != null
                && request.resource.data.keys().hasAll(['title', 'content', 'authorId'])
                && request.resource.data.authorId == request.auth.uid;
}

5. Allowing Field Modification That Shouldn't Change

// Bad: User can change the author
match /posts/{postId} {
  allow update: if request.auth.uid == resource.data.authorId;
}

// Good: Prevent changing the author field
match /posts/{postId} {
  allow update: if request.auth.uid == resource.data.authorId
                && request.resource.data.authorId == resource.data.authorId;
}

6. Overusing `get()` and `exists()`

Each get() or exists() call in your rules counts as a read operation and costs money. You're also limited to 10 calls per request.

// Bad: Multiple get() calls
match /posts/{postId} {
  allow read: if get(/databases/$(database)/documents/users/$(request.auth.uid)).data.role == 'reader'
              || get(/databases/$(database)/documents/users/$(request.auth.uid)).data.role == 'admin';
}

// Better: Use custom claims or structure data differently
match /posts/{postId} {
  allow read: if request.auth.token.reader == true
              || request.auth.token.admin == true;
}

Version Control Your Rules

Keep your rules in source control alongside your code.

Add to .gitignore if needed:

# Don't ignore rules files
!firestore.rules
!database.rules.json

Example firestore.rules:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    // All your rules here
  }
}

Deploy with Firebase CLI:

firebase deploy --only firestore:rules
firebase deploy --only database

Deployment Checklist

Before deploying rules to production:

Remove all if true or if false test rules
Verify authentication checks on all sensitive paths
Test rules using the emulator with unit tests
Check for cascading rule issues (Realtime Database)
Validate required fields on create/update operations
Ensure users can't modify fields they shouldn't (like authorId)
Review get() and exists() usage (limit of 10 per request)
Test with authenticated and unauthenticated contexts
Version control your rules
Use firebase deploy --only firestore:rules (don't deploy everything)

Complete Example: Blog Application

Here's a complete, production-ready ruleset for a blog app:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {

    // Helper functions
    function isSignedIn() {
      return request.auth != null;
    }

    function isOwner(uid) {
      return isSignedIn() && request.auth.uid == uid;
    }

    function isAdmin() {
      return isSignedIn() && request.auth.token.admin == true;
    }

    // User profiles
    match /users/{userId} {
      allow read: if isSignedIn();
      allow create: if isOwner(userId)
                    && request.resource.data.keys().hasAll(['displayName', 'email'])
                    && request.resource.data.email == request.auth.token.email;
      allow update: if isOwner(userId)
                    && request.resource.data.email == resource.data.email; // Prevent email change
      allow delete: if isOwner(userId) || isAdmin();
    }

    // Blog posts
    match /posts/{postId} {
      allow read: if resource.data.published == true || isOwner(resource.data.authorId) || isAdmin();
      allow create: if isSignedIn()
                    && request.resource.data.keys().hasAll(['title', 'content', 'authorId', 'published', 'createdAt'])
                    && isOwner(request.resource.data.authorId)
                    && request.resource.data.title is string
                    && request.resource.data.title.size() > 0
                    && request.resource.data.title.size() <= 200
                    && request.resource.data.createdAt == request.time;
      allow update: if isOwner(resource.data.authorId)
                    && request.resource.data.authorId == resource.data.authorId  // Prevent author change
                    && request.resource.data.createdAt == resource.data.createdAt;  // Prevent timestamp change
      allow delete: if isOwner(resource.data.authorId) || isAdmin();

      // Comments subcollection
      match /comments/{commentId} {
        allow read: if true;
        allow create: if isSignedIn()
                      && request.resource.data.keys().hasAll(['text', 'authorId', 'createdAt'])
                      && isOwner(request.resource.data.authorId)
                      && request.resource.data.text.size() > 0
                      && request.resource.data.text.size() <= 1000;
        allow update: if isOwner(resource.data.authorId)
                      && request.resource.data.authorId == resource.data.authorId;
        allow delete: if isOwner(resource.data.authorId) || isAdmin();
      }
    }
  }
}

Final Thoughts

Default to denying access. Only grant permissions where specifically needed.
Always verify authentication with request.auth != null and check user ownership.
Validate data on create and update operations.
Prevent field tampering by ensuring critical fields don't change on update.
Use custom claims for roles instead of repeated get() calls.
Test your rules with the emulator and unit tests before deploying.
Version control your rules and review changes like code.
Understand cascading (Realtime Database) vs explicit subcollection rules (Firestore).

Firebase Security Rules are powerful but require careful implementation. Take the time to write them correctly, test them thoroughly, and audit them regularly.

Your rules are the only thing standing between your data and unauthorised access. Make them count.

Firebase Security Is Broken. Here's the Tool I Built to Fix It.

Jacob Alcock — Fri, 07 Nov 2025 09:00:00 +0000

A couple of months ago I was doing a few penetration tests recently when I encountered Firebase configurations. Each time, I found myself stringing together a bunch of cURL commands and one-off Python scripts to check for common misconfigurations. After the third engagement, I realised this was pretty inefficient.

I was looking for a tool where I could just set the configuration and run enumeration checks. Something like msfconsole but for Firebase. I couldn't find anything that fit the bill, so I built it myself.

The Problem

Firebase is incredibly popular - it powers millions of apps. But its security model is... tricky. The core issue is that Firebase uses declarative security rules. A single || operator in the wrong place can expose your entire database.

During pentests, I kept seeing the same patterns:

RTDB nodes readable without authentication
Firestore collections with open read rules
Cloud Storage buckets listing all files
Cloud Functions without proper auth checks

The Tea app breach is a perfect example - misconfigured Firestore rules exposed sensitive user data. This wasn't a sophisticated attack, it was just someone checking if default or weak rules were still in place.

What I Wanted

Coming from a pentesting background, I needed something that:

Works with minimal information (i.e. Just the projectID and web API key)
Tests comprehensively
Is safe by default (Won't accidentally damage production data)
Handles authentication properly
Scales to large wordlists

None of the existing tools checked all these boxes.

Introducing FireScan

FireScan is a tool designed for penetration testers and developers to audit the security posture of Firebase projects. It provides an interactive console to enumerate databases, test storage rules, check function security, and much more, all from a single, easy-to-use interface.

$ firescan
███████╗██╗██████╗ ███████╗███████╗ ██████╗ █████╗ ███╗   ██╗
██╔════╝██║██╔══██╗██╔════╝██╔════╝██╔════╝██╔══██╗████╗  ██║
█████╗  ██║██████╔╝█████╗  ███████╗██║     ███████║██╔██╗ ██║
██╔══╝  ██║██╔══██╗██╔══╝  ╚════██║██║     ██╔══██║██║╚██╗██║
██║     ██║██║  ██║███████╗███████║╚██████╗██║  ██║██║ ╚████║
╚═╝     ╚═╝╚═╝  ╚═╝╚══════╝╚══════╝ ╚═════╝╚═╝  ╚═╝╚═╝  ╚═══╝

FireScan v1.0 - The Firebase Security Auditor

firescan > set projectID my-app-12345
firescan > set apiKey AIza...
firescan > auth --create-account
✓ Successfully authenticated
firescan > scan --all
Key Features

Example

Here's a real scenario from a recent test without the real data:

firescan > set projectID example-app-abc123 
firescan > set apiKey AIzaSy... 
firescan > auth --create-account
firescan > scan --firestore -l all
[✓] Scanning... [Checked: 200/200 | Found: 4]

[Firestore] Vulnerability Found! ├── Timestamp: 2025-01-15T10:23:45Z ├── Severity: High ├── Type: Firestore └── Path: users

[Firestore] Vulnerability Found! ├── Timestamp: 2025-01-15T10:23:47Z ├── Severity: High ├── Type: Firestore └── Path: messages

firescan > extract --firestore --path users 
{ "documents": 
    [ { "DOCUMENT_ID": "user_12345", "email": "john.doe@example.com", "name": "John Doe", ... } ] 
}

In under 2 minutes, I found two readable collections and extracted the data. Without FireScan, this would have taken me 20 minutes of manual curl commands.

Try It Out

https://github.com/JacobDavidAlcock/firescan

DEV Community: Jacob Alcock

Model Collapse: The AI Feedback Loop Problem Nobody Wants to Talk About

The Core Issue

How Model Collapse Works

Why You Should Care

Proposed Solutions (And Why They're All Flawed)

The Actual Problem

The Economic Incentive Problem

What Actually Needs to Happen

Final Thoughts

How to Write Secure Firebase Rules

Understanding the Basics

Cloud Firestore Rules Structure

Realtime Database Rules Structure

Key Concepts

Rule Methods

Common Secure Patterns

Pattern 1: User Can Only Access Their Own Data

Pattern 2: Public Read, Authenticated Write

Pattern 3: Role-Based Access Using Custom Claims

Pattern 4: Data Validation

Pattern 5: Attribute-Based Access (Data-Driven Roles)

Using Functions for Reusable Logic

Handling Subcollections

Realtime Database: Cascading Rules

Testing Your Rules

Use FireScan

Use the Firebase Emulator

Use the Rules Simulator in Firebase Console

Common Mistakes to Avoid

1. Using if true in Production

2. Relying Only on request.auth != null

3. Forgetting Realtime Database Cascade Rules

4. Not Validating Data on Create/Update

5. Allowing Field Modification That Shouldn't Change

6. Overusing get() and exists()

Version Control Your Rules

Deployment Checklist

Complete Example: Blog Application

Final Thoughts

Firebase Security Is Broken. Here's the Tool I Built to Fix It.

The Problem

What I Wanted

Introducing FireScan

Example

Try It Out

1. Using `if true` in Production

2. Relying Only on `request.auth != null`

6. Overusing `get()` and `exists()`