DEV Community

Cover image for Your Scraper Didn’t Fail. It Just Started Lying. So I Built Molt.
Prince Panchani
Prince Panchani

Posted on

Your Scraper Didn’t Fail. It Just Started Lying. So I Built Molt.

#brightdatachallenge #opensource #automation #ai

A website renames a CSS class.

Your scraper still returns HTTP 200.

The job still says completed.

The row count is unchanged.

No exceptions. No alerts. No obvious failure.

Except the data is now wrong.

A field that used to contain real numbers quietly starts returning 0.

And your monitoring system says:

Everything is healthy.

🎥 Watch Molt Catch a Silent Scraper Failure

This is the problem I built Molt to solve.

The failure mode nobody was watching

Most scraper monitoring focuses on things like:

  • Is the endpoint reachable?
  • Did the request return 200?
  • Did the scraper finish?
  • Did we get rows?
  • Did the schema change?

Those checks are useful.

But they miss the most dangerous failure:

The scraper succeeds while the data becomes wrong.

My own chaos test demonstrated exactly that.

Two numeric fields moved to a different element.

The scraper didn't return null.

It returned:

rows  60

score   ███████████████░░░░░  75

status  BROKEN

2 of 8 fields returned only zeros:
comment_count
download_count
Enter fullscreen mode Exit fullscreen mode

Before:

comment_count   60.5
download_count  20251.5
Enter fullscreen mode Exit fullscreen mode

After:

comment_count   0
download_count  0
Enter fullscreen mode Exit fullscreen mode

The job was still successful.

HTTP was still 200.

The row count was still 60.

And download_count = 0 is a perfectly plausible number.

That's what makes this failure so dangerous.

A null check won't catch it.

An HTTP check won't catch it.

A row-count check won't catch it.

The distribution catches it.

A median collapsing from roughly 20,251 to 0 is a very different signal from an empty result.

So I built Molt.

Molt treats scraper breakage more like an SRE incident than a failed request.

It doesn't just ask:

"Did the scraper run?"

It asks:

"Does the data still look like the data we trusted before?"

When it doesn't, Molt can:

  1. Detect the anomaly
  2. Diagnose what changed
  3. Generate a targeted repair
  4. Wait for human approval
  5. Apply the repair
  6. Re-run the scraper
  7. Verify that the data actually recovered

The goal isn't simply to detect a broken scraper.

The goal is to close the loop from detection to verified recovery.

The lifecycle

Stage What happens Bright Data surface
🔍 Detect Run scraper, snapshot output, compare against baseline bdata scraper run
🧠 Diagnose Measure drift and identify affected fields Molt
🛠️ Heal Generate a targeted repair and run the real CLI bdata scraper heal
👀 Review Compare baseline, broken output and proposed repair Molt
✅ Approve Human approves or rejects the change bdata scraper approve
🔄 Verify Run again and confirm the data actually recovered bdata scraper run

The important part:

Healing is not automatic and blind.

There is a human approval gate before the mutation happens.

The part I'm most proud of

Most "self-healing scraper" demos I've seen eventually depend on a human saying:

"The comment_count selector broke. Please fix it."

Molt doesn't need that description.

It generates the healing instruction from the evidence itself.

For example:

comment_count still fills but its values changed scale
(typical value was 60.5, now 0).

download_count still fills but its values changed scale
(typical value was 20,251.5, now 0).

Re-capture comment_count and download_count from the
current markup, keeping the existing field names.

Fields category, date, summary, tags, title and version
are unaffected and still extracting normally — leave
them as they are.
Enter fullscreen mode Exit fullscreen mode

That last sentence matters more than it looks.

Tell the healer what NOT to touch.

If six fields are healthy and two are broken, the repair should target the two broken fields.

Not rewrite the entire scraper.

That's how you avoid turning a small production incident into a bigger one.

Why Bright Data Scraper Studio is critical to Molt

Bright Data isn't simply the data source in this architecture.

It is the foundation of the product.

Molt's core workflow depends on Scraper Studio's ability to heal an existing collector while preserving the Collector ID that downstream systems already depend on.

Without that capability, Molt's self-healing workflow doesn't exist.

And mutations go through the real Bright Data CLI.

For example:

node node_modules/@brightdata/cli/dist/index.js scraper heal \
  c_mt101cvbc0o34ghzh \
  "comment_count still fills but its values changed scale..." \
  --url https://molt-chaos.vercel.app
Enter fullscreen mode Exit fullscreen mode

Every invocation is captured:

  • argv
  • stdout
  • stderr
  • exit code
  • duration

And streamed into a live terminal drawer in the UI.

So when someone asks:

"What command actually ran?"

There is no hand-waving.

You can see it.

Architecture 

flowchart LR
    A[Check] -->|Drift found| B[Diagnose]
    A -->|Healthy| A
    B --> C[Heal]
    C --> D{Awaiting Approval}
    D -->|Approve| E[Verify]
    D -->|Reject| B
    E -->|Recovered| F[Resolved]
    E -->|Still Broken| B
Enter fullscreen mode Exit fullscreen mode

The project is split into deliberately small boundaries:

packages/health
  Pure drift detection.
  Rows in → health verdict out.
  No I/O, clock, randomness or network.

packages/brightdata
  The ONLY I/O boundary.
  Drives the real bdata CLI and handles credential redaction.

packages/diagnose
  Drift evidence → targeted heal prompt.
  Pure and capped at 1000 characters.

packages/store
  libSQL + explicit SQL.
  No ORM.

packages/core
  Incident state machine + orchestration engine.

apps/sentinel
  The molt CLI.

apps/web
  Public site, documentation, playground and fleet cockpit.

apps/chaos
  A deliberately breakable website used to demonstrate healing.
Enter fullscreen mode Exit fullscreen mode

Two design decisions made the rest of the system much easier to reason about.

1. Health detection is pure

No network.

No filesystem.

No clock.

No randomness.

Every drift rule is tested against deterministic fixtures.

So I can write a test that says:

"A field returning 0 from a non-zero baseline is broken, not degraded."

And know that test won't randomly fail because some website changed.

2. All Bright Data I/O lives behind one boundary

Everything else talks through injected ports.

That means the entire incident lifecycle can be tested offline:

  • approval
  • rejection
  • failed healing
  • successful healing
  • healing that didn't actually fix the data
  • empty results
  • command crashes

No API key.

No network.

No credits.

Try it without an account

The playground has four modes.

🧪 Preflight

Run the same size and robots.txt checks used before Molt spends a scraper-create call.

🔬 Drift Replay

Provide your own baseline/current rows or use the bundled example.

Then watch the actual:

buildSnapshot
      ↓
compareSnapshots
      ↓
diagnose
Enter fullscreen mode Exit fullscreen mode

pipeline execute directly in the browser.

It even generates the same heal prompt that would be passed to:

bdata scraper heal
Enter fullscreen mode Exit fullscreen mode

No network.

No credits.

No Bright Data account.

🌐 Live Check

Runs against real Bright Data infrastructure.

🏗️ Create a Collector

Creates a real Bright Data collector and therefore uses real infrastructure/credits.

The playground makes that distinction explicit before anything potentially billable runs.

What went wrong while building it

This project caught bugs in itself.

Which felt appropriate.

The 1.63 MB page

My original target was a 1.63 MB changelog page.

Scraper creation failed twice.

The intent analyser couldn't ingest a document that large.

I switched to PostgreSQL's security advisories page at around 67 KB.

It worked on the first attempt.

The chaos site fooled the crawler

My chaos site linked between different layout versions so I could compare them visually.

Scraper Studio interpreted those links as a discovery surface and created a crawler instead of the single-page extractor I intended.

The scraper was technically working.

It was just scraping the wrong thing.

The most important bug

A brand-new collector's first run entered Bright Data's slower batch-processing path and failed.

My engine recorded that as:

successful run
+
zero rows
=
empty baseline
Enter fullscreen mode Exit fullscreen mode

That was wrong.

A command crash and a successful scraper returning zero records are not the same state.

I separated those outcomes and added tests for them.

And that bug was almost the perfect demonstration of why Molt exists:

The monitoring system itself can silently misclassify failure as success.

Honest limitations

Molt isn't magic.

There are real constraints:

  • Scraper create descriptions are limited to 500 characters.
  • Heal prompts are limited to 1000 characters.
  • Create and heal are AI-flow jobs with a concurrent-job limit, so they need serialization.
  • Target pages should stay under roughly 200 KB for the intent analyser.
  • Bright Data's cloud scrapers cannot reach localhost.
  • Credit numbers shown by the product are estimates because Bright Data doesn't publish a fixed per-operation price list.

The numbers 

Test Files    29 passed
Tests         513 passed
Enter fullscreen mode Exit fullscreen mode

All offline.

No API key.

No network.

No credits.

Strict TypeScript throughout, including:

noUncheckedIndexedAccess
exactOptionalPropertyTypes
Enter fullscreen mode Exit fullscreen mode

Why I built this

A scraper failing loudly is annoying.

A scraper failing silently is dangerous.

The worst scraper failure isn't:

ERROR: scraper failed
Enter fullscreen mode Exit fullscreen mode

It's:

HTTP 200
Job completed
60 rows returned
Everything looks fine
Enter fullscreen mode Exit fullscreen mode

while the actual data has been wrong for three weeks.

That's the failure mode Molt is designed to catch.

And when possible, it doesn't just tell you that the scraper is broken.

It shows:

  • what changed
  • why it looks broken
  • what should be repaired
  • what the repair actually did
  • whether the data recovered

That's the difference between monitoring a scraper and monitoring the truth of its output.

Try Molt

🔴 Live Demo: https://web-pink-one-39.vercel.app/

🔗 GitHub: https://github.com/PrinceXDev/molt-scraperstudio

🎥 Demo Video: https://www.youtube.com/watch?v=-ba-uywfS3I

If you're building scrapers, data pipelines, or monitoring infrastructure, I'd genuinely love to hear how you detect silent data corruption today.

And if you have questions about the Scraper Studio integration, drift-detection math, or the self-healing workflow, drop them in the comments.

Top comments (0)