DEV Community: Mohan Paramasivam

Karpathy's "Autoresearch" Just Went Viral — Here's How Software Engineers Can Actually Use the Pattern at Work

Mohan Paramasivam — Tue, 16 Jun 2026 03:36:12 +0000

Forget neural networks for a second. The real idea inside this repo is a blueprint for letting AI agents run unattended overnight — and it maps onto problems you already have on your team.

If you've been anywhere near tech Twitter or LinkedIn this week, you've probably seen people losing their minds over a small GitHub repo called autoresearch, published by Andrej Karpathy — former Tesla AI director and OpenAI founding member.

The framing is dramatic: an AI agent that runs machine learning experiments on its own, overnight, while you sleep. Tweak the code, train for five minutes, check if it got better, keep it or throw it away, repeat. Wake up to a log of a hundred experiments and a model that's quietly improved itself.

If you're not an ML researcher, your instinct might be to scroll past. "Cool, but I don't train neural networks. How does this apply to me?"

Here's the thing — the neural network part is almost incidental. What Karpathy actually open-sourced is a pattern for structuring AI-agent work: a specific way of dividing responsibility between human and AI that happens to generalize to a huge range of engineering problems. Once you see the pattern, you start noticing places in your own job where it fits.

What's Actually in This Repo

The repo itself is intentionally tiny — and that's the point. There are really only three files that matter:

The evaluator (untouchable). A file containing the fixed constants, data preparation, and the scoring logic. The agent is never allowed to modify this. It's the ruler everything else gets measured against.

The implementation (the agent's playground). A single file containing the actual model, training loop, and hyperparameters. This is the only file the agent is allowed to change. Architecture, batch size, optimizer — all fair game.

The instructions (the human's only job). A plain Markdown file describing what the agent should try, what the constraints are, how to interpret results, and what to do when something breaks. Karpathy calls this "programming the research org in Markdown."

The loop itself is almost embarrassingly simple: the agent reads the instructions, forms a hypothesis, edits the implementation file, runs a fixed time-boxed experiment (exactly 5 minutes), checks a single number, and decides — keep the change or revert it with git. Then it does it again. Roughly 12 times an hour. Around 100 times overnight.

One detail that's easy to miss but matters a lot: the philosophy baked into the instructions explicitly favors deleting code while maintaining performance as a win, and treats tiny improvements that add complexity as not worth keeping. That's a value judgment a human encoded once — and the agent now applies it autonomously, every cycle, for as long as it runs.

The Pattern Behind the Pattern

Strip away the machine learning context, and what you're left with is this:

Evaluator (fixed, automatic, trustworthy) + Implementation (the thing being improved) + Direction (human-authored intent and constraints) = a loop an AI agent can run unattended, with a built-in safety mechanism — revert if it didn't help.

This is exactly the shape of a huge number of problems that already exist on software teams — we just don't usually frame them this way. Anywhere you have something that can be objectively measured, something that can be changed, and a clear sense of what "better" means, this pattern applies.

The biggest shift this represents isn't technical. It's about where your effort goes. In the old model, you spend your time editing the implementation — writing the code, tweaking the config, trying the next idea yourself. In this model, you spend your time writing and refining the instructions — the evaluator and the direction file. The agent burns through the implementation iterations. You define what "good" means and let volume do the work.

Karpathy's own phrase for this is "programming the programmer." You're not writing the training script anymore. You're writing the thing that writes it, over and over, faster than you ever could by hand.

6 Ways to Apply This Pattern in Your Actual Job

1. Performance Optimization Loops

If you've ever had a slow function, query, or API endpoint that "could probably be faster" but nobody has time to dig into — this is the cleanest possible fit.

The evaluator: A benchmark script that runs the function with realistic inputs and reports a number — latency, throughput, memory.

The implementation: The function or module itself.

The direction file might say:

Your goal is to reduce p95 latency of the batch processing function without changing its public interface or breaking any existing tests. Run the benchmark after every change. If latency improves and all tests pass, commit the change. If not, revert. Prefer simpler implementations — if two versions perform similarly, keep the one with less code. Stop after 15 attempts or once you've had 3 consecutive attempts with less than 1% improvement.

Set this running against a non-critical service, come back in an hour, and review the diff and the experiment log — not just the final code.

2. Hunting Down Flaky Tests

Every team has that one test that fails 1 in 20 runs and nobody has gotten around to fixing. This is a perfect "run it 50 times and see" problem.

The evaluator: Run the flaky test repeatedly (say, 30 times) and report the pass rate.

The implementation: The test file and the code it's testing.

The direction file might say:

This test fails intermittently. Form a hypothesis about why — race condition, timing assumption, shared state, or something else — make a targeted fix, then run the test suite 30 times. If the pass rate improves and no other tests regress, keep the change. If not, revert and try a different hypothesis. Log your reasoning for each attempt, including failed ones, so a human can review the investigation afterward.

Even if the agent doesn't fully fix it, the log of hypotheses it tried and ruled out is often more valuable than starting from scratch yourself.

3. Refactoring Under a Safety Net

"This module needs cleanup" is a sentence every engineer has said and almost nobody has time for. The autoresearch pattern turns your existing test suite into the evaluator.

The evaluator: Your existing test suite, which must stay green, plus a complexity metric such as lines of code or cyclomatic complexity.

The implementation: The module being refactored.

The direction file might say:

Reduce the complexity of this module while keeping all tests passing. Favor deleting code over adding it — a change that removes 20 lines and keeps tests green is more valuable than one that adds a new abstraction. After each change, run the full test suite. If anything breaks, revert immediately. Stop after 10 attempts or when you can no longer find a simplification that doesn't break a test.

This directly borrows Karpathy's "deletion is a win" philosophy — which most engineering teams say they hold but rarely enforce systematically.

4. Tuning Configuration Against a Cost or Latency Metric

Thread pool sizes, cache TTLs, retry and backoff settings, connection pool limits, batch sizes for background jobs — most teams set these once, based on a guess, and never revisit them.

The evaluator: A load-testing script that reports your target metric — cost per request, p99 latency, error rate — under a fixed, repeatable load.

The implementation: The configuration file.

The direction file might say:

Your goal is to minimize average cost per request without increasing p99 latency above 400ms or error rate above 0.1%. Change one configuration value at a time. Run the load test after each change. Keep changes that improve cost without violating the constraints; revert everything else. Try at least 20 configurations before stopping.

This is the closest analog to the original repo — fixed time budget, single metric, lots of small experiments, git-based keep or revert.

5. Prompt and Eval Iteration for AI Features

If your product has any AI-powered features — and increasingly, most products do — this is arguably the most natural application of the entire pattern, because it maps almost one-to-one.

The evaluator: A fixed evaluation set of inputs with known-good expected outputs, scored automatically through exact match, similarity score, or a rubric-based judge.

The implementation: Your prompt template, system instructions, or few-shot examples.

The direction file might say:

Your goal is to improve the accuracy score on the evaluation set without increasing average token usage by more than 10%. You may modify the prompt template, instructions, and examples — but not the evaluation set itself. Run the evaluation after every change. Keep improvements, revert regressions. Prefer shorter prompts when scores are equivalent.

This is literally the same three-file contract — the eval set as the untouchable evaluator, the prompt as the implementation, and your instructions as the direction file.

6. Build and CI Pipeline Optimization

Slow CI pipelines are a tax every team pays daily, and "someone should really look at why the build takes 18 minutes" is a perennial backlog item that never gets prioritized.

The evaluator: Total CI run time, reported automatically by your pipeline.

The implementation: Your CI configuration, caching strategy, and parallelization setup.

The direction file might say:

Your goal is to reduce total CI run time while keeping the pipeline green and not removing any test coverage. Try caching strategies, parallelization, and dependency optimization one change at a time. Run the full pipeline after each change. Keep improvements, revert if the pipeline fails or coverage drops. Try at least 10 distinct approaches.

The Guardrails That Make This Safe — and Where It Breaks Down

The original setup works because the scope is extremely tight: one file can be changed, one machine is affected, one metric decides success, and every change is either committed or instantly reverted with git. Nothing the agent does is irreversible, and nothing it does affects anything outside a sandboxed environment.

The moment you widen that scope, the calculus changes. So when you apply this pattern at work, keep these non-negotiables:

Always run on a branch, never on main. Git revert is your entire safety net.
Always run in an isolated environment. A local machine, a sandbox, or a disposable container — never shared infrastructure.
Make the evaluator genuinely untouchable. If the agent can edit the thing that scores its own work, the loop becomes meaningless.
Put a hard cap on attempts. "Run until diminishing returns" needs an explicit number attached.
Review the log, not just the final diff. The sequence of hypotheses tried — including the failed ones — is often the most useful artifact.

Used within these boundaries, this isn't reckless automation. It's closer to setting up a very thorough, very patient junior engineer with a clear rubric and walking away for a few hours.

How to Try This This Week

You don't need a GPU or a research budget to test this pattern. Pick the smallest, lowest-stakes version of one of the ideas above — something on a branch, in a sandbox, with a test suite or benchmark you already trust.

A good starting point: take a function with an existing unit test, write a short direction file describing the goal and constraints, point your coding agent at it, and ask it to run a handful of iterations — committing improvements and reverting regressions as it goes.

Then do the part that actually matters: read the log of what it tried. Not the final code — the reasoning trail. That's where you'll find out whether this pattern earns a permanent spot in your workflow.

Want to go straight to the source? The original repo is here: github.com/karpathy/autoresearch

Part of the **AI for Everyone* series — practical AI guides for engineers and professionals. Follow for more.*

10 AI Habits I Wish I'd Built Sooner as a Software Engineer (And You Should Start Today)

Mohan Paramasivam — Mon, 15 Jun 2026 13:30:28 +0000

Looking back, these are the AI habits that would have saved me months of wasted effort — and the ones I had to learn the hard way.

Remember when the debate was whether AI would replace software engineers?

That debate is over. The answer is no — but with a significant asterisk.

Software engineers who use AI effectively will replace those who don't.

The data makes this undeniable. Job postings requiring experience with AI coding tools increased by 340% between January 2025 and January 2026. Meanwhile, pure implementation roles — jobs focused primarily on translating specifications into code — declined by 17%. The market has already decided what it values.

But here's what nobody tells you: using AI tools and using them well are two completely different things. Most engineers who "use AI" are getting a fraction of the value available to them, while a smaller group has figured out how to work with AI in a way that fundamentally changes what they can accomplish.

This article is about crossing that gap.

First: Where Things Actually Stand in 2026

🔧 95% of engineers now use AI tools at least weekly.

📊 75% of engineers use AI for at least half of their software engineering work.

🏆 Claude Code went from zero to the #1 most-used AI coding tool in just 8 months — overtaking GitHub Copilot and Cursor.

🤖 55% of engineers now regularly use AI agents. Staff+ engineers lead adoption at 63.5%.

💻 At Anthropic, 80%+ of code merged today was written by Claude.

The engineers at the top of the market are adopting agents fastest. The higher your judgment and experience, the more AI amplifies your output.

The Mental Model That Changes Everything

Stop treating AI as an autocomplete engine. Start treating it as a highly capable junior engineer.

Autocomplete thinks in lines. A junior engineer thinks in features, files, and systems. With autocomplete, you write code and it fills in the next token. With a junior engineer, you describe what you need, provide context, review their work critically, give feedback, and ask them to iterate. The second approach produces dramatically better results.

The Essential Toolkit

Tool	Best For	Tier
Claude Code	Complex reasoning, large codebase navigation, multi-file changes	#1 Most Used
GitHub Copilot	IDE-integrated suggestions, VS Code & JetBrains, enterprise teams	Default Choice
Cursor	Full IDE experience with AI, fast-growing	Rising Fast
Claude / ChatGPT	Architecture, debugging, documentation, reviews	Daily Companion
Copilot Workspace	Issue-to-PR agent workflows	Agentic Work

10 Habits Worth Building

1. Always Give Context — Not Just the Task

AI doesn't know your codebase, conventions, or constraints unless you tell it. Every prompt should include: language/framework/version, what the surrounding code does, constraints that matter, and what "good" looks like for your team.

❌ "Write a function to fetch user data"

✅ "I'm working in a Node.js 18 Express API. Write an async function to fetch user data by ID from PostgreSQL using the pg library. Follow our existing error handling pattern where we throw custom AppError objects, and handle the case where the user doesn't exist. Example pattern: [paste example]"

2. Review Every Line of AI-Generated Code — Without Exception

AI code is a first draft, not a finished product. Check for: logic correctness, edge cases (null values, empty arrays, network failures), security issues (SQL injection, XSS, input validation), performance (N+1 queries, leaks), and whether it matches your team's conventions.

⚠️ AI writes incorrect code with the same confidence and formatting as correct code. Clean formatting is not the same as correct code.

3. Use AI to Write Tests — Not Skip Them

Write the function, then immediately ask AI to write comprehensive tests covering the happy path, edge cases, and error conditions. Run them. Failures surface either a code bug or a test gap — both valuable. AI writes code → AI writes tests → gaps surface → AI fixes them, with you overseeing direction.

4. Use AI as Your First Debugging Partner

Before staring at a stack trace for 30 minutes, paste the full error, the relevant code, and your environment details into Claude or ChatGPT and ask for likely causes and fixes. Partial information produces partial diagnoses — give full context.

5. Let AI Handle Documentation — For Real This Time

After writing or reviewing any significant function, spend 60 seconds asking AI for clear documentation: a description, parameter and return annotations, error cases, and a usage example. Documentation becomes a byproduct of the workflow instead of a backlog item.

6. Use AI for Architecture and Design Thinking — Not Just Implementation

Most engineers use AI only for boilerplate. The highest-value use is architecture: describe a system you're designing and ask AI to walk through different approaches, tradeoffs, and the questions you should answer before deciding. AI surfaces options — your judgment still decides.

7. Never Paste Sensitive Data or Proprietary Code into Public AI Tools

Never paste API keys, credentials, customer PII, regulated data, proprietary business logic, real production schemas, or environment variables into public AI tools.

⚠️ Anonymize and abstract instead — placeholder names, generic variables, described patterns. You get most of the value with none of the risk. Push for enterprise tools (Copilot Enterprise, Claude for Enterprise, self-hosted) for sensitive work.

8. Build AI Into Your Code Review Process

Before submitting any PR, run your diff through an AI review for logic errors, security vulnerabilities, performance issues, missing error handling, and convention violations. Some teams now run AI-on-AI reviews in pipelines — one model generates, another reviews.

9. Understand What You Ship — Every Time

If AI writes code you don't fully understand, don't ship it until you do — ask AI to explain it line by line, including why this approach was chosen and what would break if changed. AI-generated code you don't understand is technical debt with an unknown interest rate.

10. Invest in Prompt Engineering for Your Specific Stack

Build a personal prompt library tuned to your stack and conventions: templates for code generation, debugging, code review, documentation, and PR descriptions. Save what works, refine what doesn't.

Do This, Not That

Do: give full context, review every line, write tests immediately, ask AI to explain unfamiliar code, use AI for architecture thinking, build a prompt library, run AI code review before every PR, use enterprise tools for sensitive work.

Don't: paste credentials or customer data, ship code you don't understand, trust output without review, use AI only for boilerplate, give vague prompts, skip tests, treat AI output as documentation, or reuse the same prompt for everything.

The Skill That Matters Most Right Now

Pick one thing this week: become an exceptional reviewer of AI-generated code. As AI handles more implementation, the bottleneck becomes human review. As one senior engineer put it: "The water level is rising. Everyone can ship something. Fewer people can ship sound systems." AI lowered the floor. Your job is to raise the ceiling.

What This Means for Your Career

The engineers who thrive won't be the ones typing the most code — they'll be the ones who understand systems deeply enough to direct AI, review critically enough to catch its mistakes, communicate clearly enough to give it context, and think architecturally enough to make the calls AI can't.

The best time to build these habits was six months ago. The second best time is today.

Are you a software engineer using AI in your workflow? What's working and what isn't? Drop it in the comments.