DEV Community: Vahid A.Nezhad

[Boost]

Vahid A.Nezhad — Wed, 06 Aug 2025 12:24:56 +0000

Vahid A.Nezhad

Aug 6 '25

🔁 LLMs Get Worse at Debugging Their Own Code

#llm #coding #ai

Comments 3

2 min read

🔁 LLMs Get Worse at Debugging Their Own Code

Vahid A.Nezhad — Wed, 06 Aug 2025 12:24:31 +0000

If you’re building tools that use large language models to write or fix code, there’s something you should know:

LLMs lose most of their debugging ability after just 1 or 2 attempts.

This isn’t intuition — it’s measurable.

A new paper from Oxford and McGill introduces something called the Debugging Decay Index.
It tracks how LLMs perform as they try to fix the same code, over and over.

What it shows is simple, but important:

📉 LLMs Plateau After 2 Fixes

The first fix attempt usually helps.

The second one adds marginal value.
By the third, things start breaking down — repeated edits, semantic drift, hallucinations.
Performance drops up to 80% by the 5th round.

So if your tool sends the same broken code back to the LLM again and again hoping it’ll “get it right” eventually — it probably won’t.

🧠 Why It Happens

LLMs don’t really “understand” what went wrong, they often repeat patterns they think look like a fix.
They get overconfident, even when wrong.
They lose track of the original intent.
The more they edit their own output, the worse it gets.

It’s like copying answers from your own wrong homework — again and again.

🤖 Why This Matters for LLM Builders

If you're working on:

Autonomous coding agents
LLM dev assistants
Auto-debug pipelines
Or anything where models fix their own mistakes…

This research is a warning.

You need to limit how many times the model “self-fixes” before restarting, re-rolling, or using an external signal (like a test suite).

More isn’t better.
More is worse.

🔄 Rethink the Loop

Instead of letting the model rewrite broken code over and over, try this:

1–2 fix attempts max
Then restart from the original prompt, or
Try a different approach entirely
Or use tests + heuristics to guide the fix

Debugging with LLMs isn’t a straight line. It’s a decision tree — and going in circles doesn’t help.

🧾 Source
Paper: The Debugging Decay Index (Oxford & McGill, 2024)

Python Prompts -Vibe Coding

Vahid A.Nezhad — Tue, 05 Aug 2025 16:25:39 +0000

In this short article, we’ll cover best practices for using coding LLMs to save time on everyday development tasks, so you can spend more effort building real value.

Here are 4 key stages of a typical Python backend project lifecycle, from initialization to feature expansion, showing how LLMs can help at each step.

1. Project Initialization

Start with a simple version, test thoroughly, and then scale your code step by step:

Set up a basic FastAPI app with `/healthcheck`, Pydantic validation, and Uvicorn entrypoint.

Generate a Django project with `users` and `projects` apps. Add SQLite DB and default URLs.

Write a `docker-compose.yml` for this Node.js + PostgreSQL + Redis stack.

Add CORS support to this FastAPI app, allowing requests from `localhost:3000`.

2. Git (Commit, Push, PR Review)

As long as LLMs understand your code changes, they can generate great commit messages.

Summarize changes in this file for a good commit message.

Write a commit message for this diff, following the Conventional Commits format.

Check this PR diff and point out risky changes or missing tests.

Fix merge conflict in this file. Keep changes from both branches where possible.

3. Debugging

Explain this traceback and suggest a fix. Python 3.11 + SQLAlchemy.

Why is this Prisma migration failing? Here's the schema and error.

This Django form isn't saving correctly — find the issue and fix it.

Logs show 500 on `/login`. Trace the root cause in the controller and model.

4. Adding Features to Existing Project

Adding new features can be tricky, so make sure the LLM reviews your code carefully. Keep the feature simple at first. Sometimes LLMs expect production-ready code and try to generate your feature at that level.

Add a `PUT /users/{id}` endpoint to this FastAPI router. Validate input and return updated user.

Create a background job in Django using Celery to send weekly emails.

Add file upload support to this Node.js Express route. Limit to 5MB images only.

Update this Prisma schema to support tags (many-to-many) for blog posts.

Refactor this 200-line function into smaller reusable helpers. Keep tests passing.

This article was originally published on:
https://medium.com/@vahida9i/python-prompts-vibe-coding-642ece6ff8f9