The rise of LLMs has been explosive. Tools like Claude Code and GitHub Copilot give both seasoned devs and juniors superpowers. But there's a catch.
Two deep‑dives (one from Anthropic, one from the Jellyfin project) lay out the uncomfortable trade‑offs we all need to see and keep in mind.
tl;dr
- AI is a tool, NOT a replacement.
- Be able to explain the architecture and code without the LLM.
- Test and iterate manually.
Anthropic Study
In a study ran by Anthropic with this setup:
- 52 junior developers were split into two groups.
- Both had to learn an unfamiliar Python library (Trio for async programming).
- One group used AI assistants; the other didn't.
- After completing a coding task, everyone took a quiz on the library.
The results were striking:
The AI group had only a small productivity gain on the task itself, but a significant negative impact on learning. The tool let them skip the deep understanding that comes from struggling through the code.
That's a sobering wake-up call for anyone who leans on AI for every
line!
Jellyfin Guideline
The Jellyfin project (a popular open‑source media platform) recently published a policy for AI‑generated code contributions. It’s refreshingly blunt.
They start by acknowledging the rise of LLMs, but then:
We are seeing a precipitous rise in contributors using AI … as well as criticism and concern about LLMs generally.
Here's the golden rule from their policy:
Do not just let an LLM loose on the codebase with a vague vibe prompt and then commit the results as is. This is lazy development and will always result in a poor quality contribution … we are not at all interested in such slop.
Their full guidelines (paraphrased):
| Rule | Why |
|---|---|
| No LLM output in communication (issues, PRs, forum posts) - except for translations | Avoids noise and extra work for other devs |
| PRs must be concise & focused - if it touches unrelated files, it’s rejected | Prevents shotgun‑blast changes |
| You must be able to explain every change without referencing the LLM | Ensures you actually understand the code |
| Test your code, follow formatting, and break PRs into small commits | Basic quality hygiene |
| Reviewers have final discretion - they don’t care if it’s AI‑generated; they care about quality | Quality over source |
And the kicker:
If you can't explain what the LLM did, we are not interested in the change.
Open Question
So I believe the fair question now would be YOLO mode vs. owning the Code.
Many of us love the YOLO vibe - just let the agent go and watch the magic happen. And that's fun, and often effective. But there's an asymmetry:
- It takes seconds to generate hundreds of lines of AI code.
- It takes hours/days for another devs or you to review, understand, and fix that code if something goes wrong.
The Jellyfin policy is a stand against that asymmetry. It says: "Make an effort, or please do not bother".
Top comments (1)
The 52-developer Trio study is the sharpest warning here: a small task-speed gain paired with significantly worse learning is a bad trade when the library is unfamiliar. Jellyfin's requirement that contributors explain every change, keep PRs focused, and test the result turns "owning the code" into observable behavior rather than a slogan. For teams, I'd treat review capacity as the real limit on agent output-generation is cheap, but every added line creates a verification and maintenance obligation that still belongs to a human.