The rise of LLMs has been explosive. Tools like Claude Code and GitHub Copilot give both seasoned devs and juniors superpowers. But there's a catch.
Two deep‑dives (one from Anthropic, one from the Jellyfin project) lay out the uncomfortable trade‑offs we all need to see and keep in mind.
tl;dr
- AI is a tool, NOT a replacement.
- Be able to explain the architecture and code without the LLM.
- Test and iterate manually.
Anthropic Study
In a study ran by Anthropic with this setup:
- 52 junior developers were split into two groups.
- Both had to learn an unfamiliar Python library (Trio for async programming).
- One group used AI assistants; the other didn't.
- After completing a coding task, everyone took a quiz on the library.
The results were striking:
The AI group had only a small productivity gain on the task itself, but a significant negative impact on learning. The tool let them skip the deep understanding that comes from struggling through the code.
That's a sobering wake-up call for anyone who leans on AI for every
line!
But I believe we can still mitigate this to some extend by prompting LLM with something like this for a ReactJS project:
Please now write me a comprehensive tutorial in Markdown that's suitable for a complete beginner in frontend coding to walk me through what you've done here. Include a summary of the technology, a high-level walkthrough, a detailed code review with code samples, and end with five suggestions for the ways the code could be improved based on a self-review.
In fact this would go nicely hand in hand with what "Jellyfin Guideline" ask devs to be able to do.
Jellyfin Guideline
The Jellyfin project (a popular open‑source media platform) recently published a policy for AI‑generated code contributions. It’s refreshingly blunt.
They start by acknowledging the rise of LLMs, but then:
We are seeing a precipitous rise in contributors using AI … as well as criticism and concern about LLMs generally.
Here's the golden rule from their policy:
Do not just let an LLM loose on the codebase with a vague vibe prompt and then commit the results as is. This is lazy development and will always result in a poor quality contribution … we are not at all interested in such slop.
Their full guidelines (paraphrased):
| Rule | Why |
|---|---|
| No LLM output in communication (issues, PRs, forum posts) - except for translations | Avoids noise and extra work for other devs |
| PRs must be concise & focused - if it touches unrelated files, it’s rejected | Prevents shotgun‑blast changes |
| You must be able to explain every change without referencing the LLM | Ensures you actually understand the code |
| Test your code, follow formatting, and break PRs into small commits | Basic quality hygiene |
| Reviewers have final discretion - they don’t care if it’s AI‑generated; they care about quality | Quality over source |
And the kicker:
If you can't explain what the LLM did, we are not interested in the change.
Open Question
So I believe the fair question now would be YOLO mode vs. owning the Code.
Many of us love the YOLO vibe - just let the agent go and watch the magic happen. And that's fun, and often effective. But there's an asymmetry:
- It takes seconds to generate hundreds of lines of AI code.
- It takes hours/days for another devs or you to review, understand, and fix that code if something goes wrong.
The Jellyfin policy is a stand against that asymmetry. It says: "Make an effort, or please do not bother".
Top comments (2)
The 52-developer Trio study is the sharpest warning here: a small task-speed gain paired with significantly worse learning is a bad trade when the library is unfamiliar. Jellyfin's requirement that contributors explain every change, keep PRs focused, and test the result turns "owning the code" into observable behavior rather than a slogan. For teams, I'd treat review capacity as the real limit on agent output-generation is cheap, but every added line creates a verification and maintenance obligation that still belongs to a human.
Exactly. Honestly, I have experienced this while working at my former employer. There, we had this situation where other developers who were by no means experienced backend developers were letting Opus 4.7 run wild on the code across multiple services, opening PRs, and sometimes I didn't even know who approved and merged their changes.
Long story short, I had to constantly ask myself why someone did something. I even tried letting the LLM just code, steering it as much as I could, yet the final decisions were still questionable.
That is why I really like what Jellyfin is requesting from devs.