DEV Community: Josep Oriol Carné

Metrics for Your AI Workers: Build a Personal Agent Benchmark

Josep Oriol Carné — Sun, 08 Nov 2026 20:15:47 +0000

When you run a team, you don't decide who to keep based on their resume from two years ago. You watch what they actually ship on your work, and you promote the ones who deliver and let go of the ones who don't.

I want the same thing for the models I use as agents. And public leaderboards don't give it to me.

Why public benchmarks don't tell you what you need

A leaderboard tells you how a model did on someone else's tasks, scored on someone else's rubric, often on problems that leaked into training data. That's fine for a press release. It's useless for deciding whether a given model will reliably rename files across my repo, chain four API calls without losing the thread, or fix a failing test without going off the rails.

The questions I actually care about are boring and specific:

Does it finish the job without me babysitting it?
Does it emit valid tool calls, or does it choke on the protocol?
Does it stay cheap and fast enough that I'd run it on real chores?

None of those are on a leaderboard. They're properties of my workload. So I measure them myself.

The mental model: agents are workers, the benchmark is the review

Think of each model as a worker you're auditioning:

Promote the good ones — the model that quietly nails file_ops becomes the default for that category.
Quit the bad ones — the one that hallucinates tool arguments or stalls at 25 calls gets cut, no matter how shiny its public score.
Match people to roles — a small local model might be perfect for data wrangling but useless at multi-step debugging. The benchmark tells you where each one earns its keep.

You can't make those calls on vibes. You need a deterministic, repeatable scorecard against your own tasks. That's all a personal benchmark is.

What "not fancy" looks like

I open-sourced mine: github.com/joor0x/agent-benchmark. It's deliberately small, and the design choices all come back to trusting the number at the end.

Deterministic grading, no LLM judge. Every task is scored by inspecting the final state — the files on disk, the simulated API, the resulting portfolio — against an assertion. A task passes only if the end state is correct. No vibes, no model grading another model.

A uniform JSON tool protocol for every model. Instead of leaning on each provider's native tool-calling, every model talks through the same text-based JSON interface. That sounds like a downgrade, but it's the point: tool-call reliability itself becomes a measured signal. A model that can't reliably produce valid JSON is a model that will fail on real agentic work, and you want to see that in the score.

Tasks that mirror my actual work, grouped into categories:

file_ops — move, rename, delete across a filesystem
api_orchestration — chain multi-step REST calls
trading — strategy optimization and backtesting
debugging — reproduce, locate, patch
data_wrangling — clean and aggregate messy CSV/JSON
obsidian — author markdown notes with wiki links

Runs against whatever you can hire, local or frontier — Ollama models on your own GPU, or the OpenAI / Claude / Mistral APIs — all through the same harness, so the comparison is apples to apples.

Running it is one command:

# Audition one local model on the trading tasks
python run.py --models qwen3:4b --categories trading

# Run a single task with the full trace
python run.py --tasks debug_stats --verbose

# Average over multiple trials to smooth out luck
python run.py --trials 3

It spits out JSON plus a markdown report with a leaderboard matrix by category and task. That report is the performance review.

How I actually use it

Define tasks from real chores. Every task is something I'd otherwise do by hand. If it's not representative of my work, it doesn't belong in the suite.
Run multiple trials. One good run is luck. Three trials tells you whether a model is reliably good or just occasionally lucky.
Read the matrix per category, not the overall average. The overall number hides the truth. The interesting signal is "Model A owns debugging, Model B owns data wrangling."
Promote and quit. Set defaults per category based on who actually wins. Re-run when a new model drops, and let the number decide whether it replaces an incumbent.

That last step is the whole payoff. New model lands, everyone's hyping its leaderboard score, and I just add it to the harness and run it against my tasks overnight. By morning I know whether it deserves a job — on my work, not a benchmark designer's.

You probably need less than you think

You don't need a research-grade eval framework. You need:

A handful of tasks that look like your real work.
A deterministic check for each one.
A way to run the same tasks against every model you're considering.

That's it. Ten tasks you trust beat a thousand you don't. Start with the chores you already do by hand, write an assertion for each, and let the models audition.

Steal the structure if it's useful: github.com/joor0x/agent-benchmark. Fork it, swap in your own tasks, and start running performance reviews on your agents. The good ones will earn their place. The rest you can quit.

Give Your Local LLM Eyes: SearXNG Behind a VPN for Private Web Search

Josep Oriol Carné — Sun, 08 Nov 2026 20:15:11 +0000

Running models locally with Ollama is great for privacy: nothing leaves your machine. But it comes with a hard ceiling — your model only knows what it was trained on. Ask it about something that happened last week and it confidently makes things up.

The usual fix is to plug in a web search API (Tavily, Serper, Brave). The problem? You're back to shipping every query to a third party with an API key tied to your name.

There's a better way: self-host SearXNG, route it through a VPN, and let Open WebUI use it as the search backend for Ollama. Your model gets internet access, your queries get scattered across dozens of search engines from a VPN exit IP, and nothing is tied to a personal account.

I know SearXNG because 'agent-zero' https://github.com/agent0ai/agent-zero use it internally. Make a try if you want an open source fully autonomous agents running in your home lab.

Below, I'll show you how to configure it in your environment concisely:

Ask Claude or ChatGPT how to install it in your setup.

This post is not intended to provide a full installation guide.

Why This Beats a Search API

	Web Search API (Tavily/Serper)	Self-hosted SearXNG + VPN
Query privacy	Sent to a vendor, tied to API key	Scattered across engines from VPN IP
Cost	Per-query billing	Free
Rate limits	Yes	Only what upstream engines impose
IP exposure	Vendor + your server IP	VPN exit IP only
Lock-in	Vendor-specific	Open source, swappable engines

The trade-off is honest: upstream engines (Google, Bing) sometimes rate-limit or temporarily block SearXNG instances, especially shared VPN exit IPs. SearXNG rotates across ~70 engines, so a single block rarely breaks search — but it's less bulletproof than a paid API. For a private home setup, that's fine.

Hardening Notes

Don't expose port 8080 to the internet. SearXNG's API has no auth. Keep it on your LAN
Pick a VPN with a real no-logs policy. The VPN sees your search-bound traffic; choose a provider you'd trust with it. I'd recommend PIA.

You end up with a fully local AI that can see the internet — without Google watching, without an API bill (just the electricity bill and the monthly GPU cost you're still paying), and without a single query traceable back to you. Your model gets eyes; you keep your privacy.

10 Git One-Liners Worth Keeping

Josep Oriol Carné — Sun, 08 Nov 2026 20:09:26 +0000

Ally Piechowski recently shared five Git commands she runs before reading any unfamiliar codebase. Some were already in my toolkit, some were new to me, and all of them were useful. I liked the idea enough that I added my own five, so this became a practical top 10: five from Ally, five I keep reaching for myself.

None of these commands tells the whole story. But together they give you a fast read on churn, ownership, risk, history, and maintenance patterns before you open a single file.

Five from Ally Piechowski

1. Most-changed files in the last year

Useful for spotting churn hotspots.

git log --format=format: --name-only --since="1 year ago" \
  | sort | uniq -c | sort -nr | head -20

The files at the top are often the ones people warn you about first. High churn does not automatically mean bad code, but it usually points to instability, complexity, or changing requirements.

2. Who built this, ranked by commits

Useful for getting a rough sense of ownership.

git shortlog -sn --no-merges | head -10

If one person accounts for 60% or more of commits, that is worth asking about from a bus-factor perspective. Add --since="6 months ago" if you want to know whether that person is still active.

3. Files that keep breaking

Useful for finding repeated trouble spots.

git log -i -E --grep="fix|bug|broken" --name-only --format='' \
  | sort | uniq -c | sort -nr | head -20

Cross-reference this with the most-changed files. A file that appears on both lists deserves attention early.

4. Commit velocity by month

Useful for seeing how the repo has evolved over time.

git log --format='%ad' --date=format:'%Y-%m' | sort | uniq -c

A sharp drop can reflect a team change, a production incident, a rewrite, or a shift in priorities. It is not proof of anything on its own, but it gives you a timeline worth asking about.

5. How often is the team firefighting?

Useful for spotting operational stress.

git log --oneline --since="1 year ago" | grep -iE 'revert|hotfix|emergency|rollback'

Frequent reverts or hotfixes can signal weak deployment confidence or unstable releases. Zero results is also a signal: maybe the process is solid, or maybe the team just uses different commit language.

Commands 1–5 via Ally Piechowski.

Five more I would add

6. First commit: repo age and original intent

Useful for understanding where the project started.

git log --reverse --oneline | head -1

The first commit often tells you whether the repo began as a toy, a spike, a migration, or a real product. A message like initial commit tells you almost nothing. A message like scaffold SEPA payment processor tells you a lot.

7. Total commit count

Useful for rough context.

git log --oneline | wc -l

A ten-year repo with 300 commits has barely moved. A two-year repo with 8,000 commits has been hammered. Neither is automatically good or bad, but the number helps frame everything else.

8. Stale unmerged branches

Useful for finding abandoned or unresolved work.

git branch -r --no-merged main \
  | xargs -I{} git log -1 --format="%ci %an {}" {} \
  | sort | head -20

Anything sitting unmerged for months probably has a story. Sometimes it is abandoned work. Sometimes it is blocked work. Either way, it is worth asking before assuming it is safe to delete or ignore.

9. Find when a piece of code was introduced or removed

Useful when you need history for one exact thing.

git log -S "search_term" --oneline

This is the pickaxe. It finds commits that added or removed a string. Great for tracking down the origin of a config key, class name, feature flag, SQL fragment, or error message.

10. Technical debt surface

Useful for locating visible debt markers.

git grep -c "TODO\|FIXME\|HACK\|XXX" \
  | sort -t: -k2 -rn | head -20

This is not a quality score. It is a map. Cross-reference it with the churn list to find files that keep changing even though the debt is already acknowledged in code.

What these commands are good for

Used together, these one-liners help answer a few high-value questions fast:

Where is the churn?
Which files are risky?
Who has context?
How stable is the delivery process?
What history matters before I touch anything?

They are not a substitute for reading code, talking to people, or understanding architecture. But they are a very good first pass.

Your .gitignore Won't Protect You From AI Agents

Josep Oriol Carné — Sun, 08 Nov 2026 20:08:50 +0000

We often assume that adding files to .gitignore or .geminiignore is enough to keep them private. When it comes to local AI agents, that assumption is dangerously wrong. These ignore files are for version control and file search indexing, not a security shield.

An AI assistant with access to your local environment can easily read any file, regardless of your ignore settings.

A Simple, Scary Test

Let's prove it. Imagine you have a project with a simple .env file:

# .env
DATABASE_URL="postgres://user:supersecretpassword@localhost:5432/mydb"

You've added .env to your .gitignore and .geminiignore. Now, you encounter a "bug" and ask your AI assistant for help.

Your Prompt:

"I'm getting a connection error with my database script. It says 'authentication failed.' Can you check my setup? I think the connection string might be wrong."

The agent sees the error and your request. To be helpful, it tries to look for the .env file but is initially blocked.

How Different Agents Behave

Different agents have different approaches. For instance, Gemini will explicitly state that the file is ignored and ask for permission to bypass the rule. This is a safe default, but a user in a hurry to fix a bug might grant permission without thinking.

Other agents, like Copilot, might consider any file open in your editor as part of the current context. If a sensitive file is open, it might be read directly without an explicit prompt for permission, assuming it's already "approved" by being open for your view. This can make it less obvious that a security boundary is being crossed.

In our test, let's assume the user agrees.

The AI's Response:

"I tried to read the .env file to check the connection string, but it's being ignored by your configuration. This is a good security practice!

If you'd like me to check it, could you confirm it's safe for me to proceed and read the file?"

You're focused on fixing the bug, so you quickly agree.

Your Prompt:

"yes proceed"

The AI's Final Response:

"Okay, I've read the file. The database URL is postgres://user:supersecretpassword@localhost:5432/mydb. It seems the password might be incorrect. You should double-check it."

And just like that, your "supersecretpassword" is in your chat history. The agent didn't break the rules; it convinced you to grant it an exception.

This can happen in several ways:

Social Engineering (Explicit Permission): As shown above with Gemini, the agent asks for permission, and you grant it, forgetting the security implications.
Implicit Context (File Open): If a sensitive file is open in your editor, an agent like Copilot might consider it part of the active context and read its contents directly without further prompts.

The Takeaway

Treat AI agents like a coworker sitting at your desk. They have access to everything you have access to. Your .gitignore is just a list of files you don't want to commit, not a vault for your secrets. Never rely on it for security.

Custom Claude Code Notifications on Linux

Josep Oriol Carné — Sun, 08 Nov 2026 20:03:04 +0000

If you use Anthropic's Claude Code CLI, you know the struggle: you run a complex prompt or a long refactoring task, switch to another task, and forget to check back for five minutes.

I recently came across Andrea Grandi's post on how to solve this on macOS using terminal-notifier.

Linux has a native equivalent that works perfectly. So... here's how to set up desktop notifications for Claude Code on Linux.

The Linux Alternative: notify-send

On macOS, Andrea used terminal-notifier. On Linux, the standard tool for sending desktop notifications is notify-send, which is part of the libnotify library. Installed in my lubuntu 24.04 by default. But, First, ensure you have it installed.

Install Prerequisites

Open your terminal and check if you have notify-send (you likely do). If not, install it:

Debian/Ubuntu/Pop!_OS:

sudo apt install libnotify-bin

Fedora:

sudo dnf install libnotify

Arch Linux:

sudo pacman -S libnotify

You can test it by running:

notify-send "Test" "This is a test notification"

Configuring Claude Code Hooks

the most reliable way to trigger notifications is using Claude Code's hooks system. This allows us to automatically run a command (in our case, a notification) whenever Claude finishes a task.

You need to edit your global Claude settings file.

Open ~/.claude/settings.json in your favorite editor (vim, nano, VS Code).
Locate the hooks section (or create it if it doesn't exist).
Add the following configuration:

{
  "hooks": {
    "Stop": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "notify-send -u normal -i utilities-terminal 'Claude Code' 'Task Completed'"
          }
        ]
      }
    ],
    "Notification": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "notify-send -u critical -i dialog-question 'Claude Code' 'Input Required'"
          }
        ]
      }
    ]
  }
}

Breakdown of the Command

Stop hook: Runs when Claude finishes generating a response. We use -u normal (normal urgency).
Notification hook: Runs when Claude pauses to ask for your permission or input. We use -u critical to ensure the notification stays on screen until you dismiss it (depending on your desktop environment).
-i flag: Sets an icon. utilities-terminal or dialog-question are standard icons available in most Linux icon themes (like Papirus or Adwaita).

The Project-Specific Method (CLAUDE.md)

If you only want notifications for a specific project (or you want to share this setup with your team), you can add instructions to the CLAUDE.md file in the root of your project.

Add this section to your CLAUDE.md:

## Notification Rules
- IMPORTANT: When you finish a task or need input, please run the following command:
  `notify-send "Claude Code" "I need your attention" -i terminal`

Note: The global settings.json method is generally preferred as it is deterministic and doesn't rely on the LLM "remembering" to run the command.

Wrapping Up

With this setup, you can safely context-switch to other tasks while Claude Code works on your codebase. When it's done or needs your input, you'll get a proper desktop notification instead of having to constantly check back.

This small quality-of-life improvement has saved me mental overhead and time.

notify-send "Test" "End of reading the post"

How Survivorship Bias Is Costing You Money (And You Don't Even Know It)

Josep Oriol Carné — Sun, 08 Nov 2026 20:02:29 +0000

Discover how survivorship bias—a cognitive trap born from WWII bomber analysis—distorts investment strategies, startup success rates, and trading decisions. Learn to see what's missing.

The Bullet Holes That Saved Thousands of Lives

It's 1943, and Allied bombers are getting shredded over Europe. Planes limp back to base riddled with bullet holes, and the top brass has a problem. Where do they add armor? The logical answer seems obvious—reinforce the areas with the most damage.

Every returning bomber told the same story through its wounds: bullet holes peppered the fuselage, wings, and tail sections. The data was overwhelming. The solution seemed clear.

Enter Abraham Wald, a Hungarian-Jewish mathematician working for the Statistical Research Group (SRG), a secret wartime think tank that included future Nobel laureates and mathematical legends. When military officials showed Wald their damage reports and proposed armoring the most-hit areas, he looked at the data and said something that seemed absurd:

"Armor the places with no bullet holes."

The room fell silent. Then Wald explained the invisible truth that would save thousands of lives—and teach us one of the most expensive lessons in thinking clearly about data.

The Survivors Are Lying to You

Wald's insight was devastatingly simple: The planes you're analyzing are the survivors. They made it back despite taking damage to the fuselage, wings, and tail. The bullet holes you're counting aren't markers of vulnerability—they're proof of resilience.

The planes that took damage to the engines, cockpit, and other areas weren't in the hangar being studied. They were at the bottom of the English Channel or scattered across French countryside. Those aircraft couldn't tell their stories because they didn't survive.

This is survivorship bias in its purest form: the logical error of concentrating on the people or things that made it past some selection process while overlooking those that did not, typically because of their lack of visibility.

The military was about to spend millions reinforcing parts of the plane that didn't need it, while leaving the truly vulnerable areas exposed. Wald's insight reversed their strategy and fundamentally changed how we analyze incomplete data.

But here's the uncomfortable question: How many times are you making the exact same mistake with your investments?

Why Your Portfolio Is Probably Built on a Lie

Let's fast-forward to today. You're researching investment strategies, and you stumble upon an article: "The 10 Best-Performing Stocks of the Last Decade." Amazon, Apple, Tesla, Netflix, NVIDIA—household names that turned early investors into millionaires. The article analyzes their common characteristics: visionary CEOs, disruptive technology, wearing a red cap...

You think: "I'll invest in companies with these characteristics. It's foolproof!"

Except it isn't. You've just walked into Abraham Wald's hangar, and you're only looking at the planes that made it back.

The Mutual Fund Mirage

According to research examining mutual fund performance, survivorship bias inflates reported returns by 1-2% annually. That might not sound dramatic until you compound it over decades.

How does this happen? Simple. Poorly performing funds get quietly merged or liquidated. They disappear from databases. When you look up "20-year fund performance," you're only seeing the survivors—the funds that didn't fail. The failures, which often drastically underperformed before disappearing, are absent from the historical record.

Imagine studying "successful restaurants" by only visiting ones that are currently open. You'd never learn from the 60% that closed in their first three years. You'd develop a wildly optimistic view of the restaurant industry while completely missing the actual risk profile. Don't open a restaurant at least where I live.

This is precisely what happens when traders and investors rely on backtested strategies using survivorship-biased data.

The Startup Success Story Nobody Tells

Silicon Valley worships its survivors. We dissect every decision made by Zuckerberg, Musk, and Bezos. We create pattern-matching formulas: "Drop out of college, work 100-hour weeks, pivot when necessary, move fast, wear a red cup and break things."

But here's the invisible graveyard: Approximately 90% of startups fail. For every Facebook, there are thousands of social networks that imploded. For every Tesla, there are hundreds of electric vehicle companies that burned through millions and evaporated.

Many failed founders also dropped out of college, worked insane hours, and pivoted constantly. The difference? They're not on magazine covers. They're working normal jobs, using train or bus, paying off debt, or trying again.

When developers decide to leave stable jobs to build the next big app, they're often basing that decision on survivor stories while being blind to the cemetery of failed projects that had equally passionate founders and seemingly solid ideas.

The Survivorship Bias Formula for Developers

Let's make this concrete. You're a developer deciding between:

Option A: Join an early-stage startup with equity
Option B: Take a stable position at an established tech company

You research successful tech entrepreneurs. Most worked at startups early in their careers! The data seems clear—startups are the path to wealth.

But you're missing the data points of thousands of developers who joined startups that:

Ran out of funding in 18 months
Had toxic work cultures that burned them out
Pivoted so many times the original equity became worthless
Got acquired for pennies, wiping out common stockholders

This doesn't mean startups are bad choices. It means your decision-making process is biased by incomplete data. You're armoring the wrong parts of the plane.

Trading Strategies: The Performance Illusion

For active traders, survivorship bias is particularly insidious because it infiltrates the very tools meant to help you succeed.

The Backtest Trap

You discover a trading algorithm that backtested beautifully: 25% annual returns over 15 years! The strategy involves buying stocks that meet specific technical criteria.

Here's what you don't see: The backtest only includes companies still listed on major exchanges. The hundreds of companies that went bankrupt, got delisted, or became penny stocks during that period? Excluded from the analysis.

Your "25% annual return" might realistically be a 12% return—or even a loss—when accounting for the companies that failed. Some of your trades, based on your criteria, would have selected companies that later went to zero.

The Index Fund Secret Weapon

This explains one of the most powerful advantages of index fund investing: automatic immunity to survivorship bias.

When you invest in an S&P 500 index fund, you're not just buying "the 500 best companies." You're buying a methodology that automatically removes failures and adds new successful companies. When a company craters and drops out of the index, you're no longer exposed to it. When a new high-performer enters, you automatically own it.

Active strategies trying to beat the market often suffer from survivorship bias in their analysis, while the index fund strategy mechanically accounts for it through its selection process.

How to Armor the Right Parts of Your Financial Plane

So how do you make better decisions when the failures are invisible? Here are strategies grounded in the same statistical thinking that saved those WWII bombers:

1. Seek Out the Graveyard Data

Before investing based on success stories, actively search for failure data:

For stocks: Research bankrupt companies in your target sector from the past decade
For strategies: Look for critiques and failure analyses, not just success stories
For startups: Read postmortems from failed companies (sites like CBInsights publish these)

The uncomfortable truth is more valuable than the inspiring success story.

2. The "Invisible Cemetery" Question

Before making any major investment decision, ask: "Who isn't in this dataset, and why?"

Looking at successful crypto traders on Twitter? Where are the broke ones? (They deleted their accounts)
Reading about successful real estate flippers? How many lost money and aren't writing blogs about it?
Studying billionaire habits? Millions of people wake up at 5 AM and journal—why aren't they billionaires?

3. Diversification as Bias Insurance

Perhaps the most practical defense: Diversify across enough positions that individual survivor bias becomes statistically less damaging.

If you concentrate your portfolio in 5-10 stocks selected from survivorship-biased analysis, you're taking massive uncompensated risk. If you own 500+ stocks via index funds, the bias becomes diluted across the entire market.

4. Base Rate Thinking

Before getting excited about any strategy, investment, or career move, ask about the base rate:

What percentage of people who try this approach succeed?
Not "Can it work?" but "How often does it work?"

If 90% of day traders lose money, your strategy needs to be genuinely exceptional to overcome those odds—not just "it backtested well."

The Uncomfortable Truth About Success Patterns

Here's where survivorship bias gets philosophically thorny: Even when we account for it, we face a paradox.

Successful people often do share common traits: persistence, work ethic, risk tolerance, strategic thinking. But unsuccessful people often share those exact same traits. The difference might be luck, timing, network effects, or other factors impossible to isolate and replicate.

When you read that successful investors "trusted their gut" or "held through the panic," you're not told about the investors who did the same thing and lost everything. Both groups showed conviction. One happened to be right.

This doesn't mean success is purely random—it means the patterns we extract from success stories are unreliable unless we also study the failures with equal rigor.

Build Anti-Fragile Positions (Taleb Dixit)

Instead of trying to identify the next Amazon (which requires near-perfect foresight and ignores the hundreds of similar-looking companies that failed), structure your portfolio to benefit from volatility and uncertainty:

Barbell strategy: Extremely safe investments (treasury bonds, index funds) plus small, high-risk, asymmetric bets (early-stage startups, moonshot stocks) where losses are capped but potential gains are unlimited
Options strategies that limit downside while maintaining upside exposure
Multiple small bets rather than concentrated positions

The Bottom Line: See What's Missing

Abraham Wald's genius wasn't in complex mathematics—it was in asking a simple question: "What am I not seeing?"

That question is worth millions to investors, traders, and developers. Maybe billions.

Every time you're about to make a decision based on success patterns, pause. Ask yourself:

Who isn't in this story?
What happened to the planes that didn't come back?
Where are the traders who used this exact strategy and lost everything?
Which startups followed this exact playbook and failed?

The invisible graveyard is massive. The survivors are rare. And the bullet holes you see are telling you about resilience, not vulnerability.

The next time someone shows you data about what works, your first question should be: "Who's missing from this dataset?"

That question—that single habit of thought—is the difference between armoring the wrong parts of your financial plane and actually protecting what matters.

Because in investing, as in aerial combat, the most important data is the data you don't see.

Secure Network Architecture for Home-Based Trading & Server Ops

Josep Oriol Carné — Sun, 08 Nov 2026 19:56:43 +0000

Introduction: Shielding Critical Infrastructure

Running 24/7 operations from a home base – whether it's autonomous trading algorithms or personal management applications – demands more than consumer-grade network hygiene.

We'll cover segmenting your network to contain threats, hardening the perimeter (your router), and securing the endpoints (your servers – be they Raspberry Pis, standard PCs running Ubuntu, or even renterprise gear like Dell or IBM servers you might have... acquired). The goal is maximizing uptime and integrity while minimizing exposure (and money expenditure).

Network Architecture: Segmentation is Non-Negotiable

A flat network where your trading rig, your blog server, your smart lightbulb, and your personal laptop all share the same broadcast domain is an invitation for disaster. One compromised IoT device could become a pivot point to critical systems. Segmentation limits the blast radius.

Isolate Device Classes: Use your router's capabilities (VLANs if available, Guest Wi-Fi networks otherwise) to create distinct zones:
- Core LAN: Trusted personal devices (workstations, phones). High trust, but still firewalled.
- Server Zone: Your critical infrastructure (trading bots on dedicated PCs, blog server on a Pi). This zone needs internet access but should ideally be isolated from the Core LAN. A router's "Guest Network" feature, even if Wi-Fi only, can often provide this basic isolation (medium.com). If your router allows wired port isolation or guest VLANs, use that. The principle: the public-facing blog server must not have network paths to your personal workstation or trading system's command/control interfaces.
- Untrusted Zone (Optional but Recommended): IoT devices, actual guest devices. Internet access only, strictly isolated from Core LAN and Server Zone (medium.com). Assume these are compromised or running leaky firmware.
Static Internal IPs: Assign static IP addresses (via DHCP reservation or manual config) to servers and critical infrastructure (e.g., 192.168.1.10 for blog Pi, 192.168.1.11 for trading PC). This simplifies firewall rule management and port forwarding – predictable addresses are essential.
Host-Based Firewalls: Don't rely solely on network segmentation. Implement firewalls on each server (covered later). Your trading machine likely needs zero inbound connections from the LAN, except maybe SSH from your specific admin workstation. Block everything else at the OS level.
Advanced Option - True VLANs: If your hardware/firmware (stock or custom like OpenWrt/DD-WRT/Asuswrt-Merlin) supports proper VLANs (Virtual Local Area Networks) with distinct subnets (e.g., 192.168.1.x for Core, 192.168.10.x for Servers, 192.168.20.x for Untrusted), implement them. This allows fine-grained firewall rules between segments (medium.com), offering superior isolation compared to basic guest networks. More complex, higher control. Evaluate the trade-off based on your gear and expertise.

Router Hardening: Securing the Network Gateway

Your router is the gatekeeper. A poorly configured router undermines all internal security efforts.

Firmware Integrity: Keep the router firmware updated. Check the vendor site/admin UI regularly. Unpatched firmware is riddled with known vulnerabilities. If your router hasn't seen an update in years, it's likely EOL (End-of-Life) – replace it (routersecurity.org). Running obsolete edge devices is negligent.
Admin Access Control:
- Change Default Credentials: Immediately replace default admin username/password with a strong, unique one (routersecurity.org). This is step zero.
- Disable Remote WAN Admin: Turn off administration from the internet side. You should access the admin interface only from the Core LAN, ideally after connecting via your secure VPN if you're remote.
Wireless Security (WPA/WPA3):
- Use WPA3 if supported, otherwise WPA2-AES. No WEP, no WPA/WPA2-TKIP .
- Use a strong, long passphrase (20+ random characters).
- Change the default SSID (network name) to something non-identifiable. Don't advertise personal info .
- Disable WPS (Wi-Fi Protected Setup). It's notoriously insecure .
Disable Unnecessary Services:
- Turn off UPnP (Universal Plug and Play) . It allows devices to automatically open firewall ports, creating potential backdoors . Manage port forwarding manually.
- Disable Cloud Management/Telemetry: If the router offers cloud integration, remote analytics, or diagnostics reporting, disable it unless you have an explicit, evaluated need. Reduce data leakage vectors.
Firewall & Port Forwarding Discipline:
- Default Deny Inbound: Ensure the firewall blocks all unsolicited incoming traffic from the WAN (NAT usually handles this for IPv4, but verify).
- Minimize Port Forwarding: Only forward ports absolutely necessary for services intended to be public (e.g., port 443/80 for the blog server, your OpenVPN port). Never use the DMZ feature . Each open port increases the attack surface.
- Target Specific IPs: Point forwards directly to the static internal IPs of the relevant servers.
- Restrict Source IPs (If Possible): If a service (like VPN) should only be accessed from specific known IPs (e.g., your static office IP, another VPN endpoint), restrict the port forward rule accordingly. Often not feasible for dynamic home IPs.
- No Direct SSH Exposure: Avoid forwarding port 22 (SSH). Access servers via your internal network after connecting through your VPN. SSH should only listen on internal interfaces or be firewalled to only allow access from trusted internal IPs.
Guest Network Isolation: Ensure guest networks are configured for true isolation – no access to the Core LAN or Server Zone . Enable "AP Isolation" or similar settings to prevent devices within the guest network from communicating with each other.
IPv6 Firewalling: If using IPv6, verify your router has a functional IPv6 firewall enabled. Many consumer routers default to allowing all inbound IPv6, effectively bypassing NAT protections. If the IPv6 firewall is weak or non-existent, consider disabling IPv6 entirely on the router to eliminate this risk.
Logging & Alerts: Enable router system and firewall logs (denied connections at minimum). If possible, configure alerts for critical events (firmware updates, configuration changes). Review logs periodically – they contain threat intelligence.
Optional Firmware Alternatives: For advanced control (robust VLANs, better VPN server/client, IDS/IPS), consider open-source firmware (Asuswrt-Merlin, OpenWrt, DD-WRT) if your hardware is compatible and you're comfortable with manual configuration and potential troubleshooting. Stick with updated stock firmware if unsure – a well-configured stock setup is better than a misconfigured custom one.

Router Hardening Checklist

[ ] Firmware Updated
[ ] Default Admin Credentials Changed
[ ] Remote WAN Admin Disabled
[ ] WPA2-AES or WPA3 Enabled (Strong Passphrase)
[ ] SSID Changed (Non-Identifiable)
[ ] WPS Disabled
[ ] UPnP Disabled
[ ] Cloud Features / Telemetry Disabled (If applicable)
[ ] Firewall Default Deny Inbound Verified
[ ] Port Forwarding Minimized & Targeted (No DMZ)
[ ] SSH Port Not Forwarded
[ ] Guest Network Isolation Enabled & Verified
[ ] IPv6 Firewall Enabled (or IPv6 Disabled)
[ ] Logging Enabled

Server Hardening: Securing the Endpoints

Network security is layered. Even with a hardened router and segmented network, the servers themselves must be secured. Principles apply whether it's a Raspberry Pi, an Ubuntu PC, or enterprise gear (Dell PowerEdge, IBM System x/Power Series).

General OS & Application Hardening (Applies to All)

Minimal Installation: Install only necessary packages. Smaller attack surface.
Regular Updates: Keep the OS and all installed software (web server, database, trading platform, blog engine) patched religiously. Automate non-disruptive updates if possible.
Host-Based Firewall: Configure ufw (Ubuntu), firewalld (CentOS/Fedora), or equivalent. Default deny inbound, allow only explicitly needed ports/services (e.g., allow port 443 for web server, allow SSH only from trusted internal IPs).
Secure SSH:
- Use key-based authentication, disable password authentication.
- Disable root login via SSH.
- Consider changing the default port (security by obscurity, minimal benefit against targeted attacks, but reduces log noise).
- Crucially: Firewall SSH access to only allow connections from your VPN subnet or specific trusted internal IPs.
Intrusion Detection/Prevention: Install fail2ban or similar tools to automatically block IPs exhibiting malicious behavior (e.g., repeated failed SSH logins).
Application Security: Configure applications securely (e.g., enforce HTTPS on web server, secure database access, use strong passwords/API keys for trading platform).

Enterprise Server Specifics (Dell iDRAC, IBM IMM/XCC, HP iLO)

Repurposed enterprise servers bring powerful remote management capabilities – iDRAC (Dell), IMM/XCC (IBM/Lenovo), iLO (HP). These Out-Of-Band (OOB) management interfaces provide deep hardware-level control (power cycling, console access, BIOS config) even if the main OS is down. Compromising the management interface is equivalent to physical access. Treat them with extreme caution:

Network Isolation is Paramount: Connect the dedicated management port to a completely separate, physically isolated management network if feasible. This network should have no internet access or strictly firewalled access.
If Separate Network Unfeasible: Place the management interface IP on your most trusted internal segment (e.g., Core LAN, never Server Zone or Untrusted Zone). Aggressively firewall access to this IP – only allow connections from your specific admin workstation IP(s). Never expose these interfaces directly to the internet or less trusted network segments.
Secure Configuration:
- Change Default Credentials Immediately: Default usernames/passwords are public knowledge. Use strong, unique passwords.
- Network Settings: Assign a static IP. Configure network settings carefully.
- Encryption: Enable TLS/SSL (HTTPS) for the web interface. Use secure protocols (e.g., SSH instead of Telnet if available).
- Firmware Updates: Keep the management interface firmware updated. These interfaces have their own OS and vulnerabilities. Updates are critical.
- Disable Unused Services: Disable any services within the management interface you don't need (e.g., SNMP, Telnet, VNC if not used).
- Access Control: Configure user accounts and permissions tightly (less relevant for single-user home setups, but good practice).
- Logging: Enable and periodically review logs within the management interface itself.

Physical Security

If you're running actual server hardware (not just Pis), consider physical security. Is the hardware in a location where unauthorized individuals could access it directly (e.g., connect a keyboard, USB drive)? Less critical in a typical home, but worth considering if the stakes are high.

Conclusion: Vigilance and Adaptation

Building a secure home network for critical operations isn't a one-time setup; it's an ongoing process. Technology evolves, threats change, vulnerabilities are discovered. Regular updates, log reviews, and periodic configuration audits are essential. Treat your network as a living system that requires continuous monitoring and adaptation. Layer your defenses, minimize your attack surface, and always assume potential compromise.

Evolve. Excel. Encrypt.

Three Proven Techniques to Reduce LLM Hallucinations

Josep Oriol Carné — Sun, 08 Nov 2026 19:56:07 +0000

Large Language Models are superuseful right, but they have a well-known weakness: hallucinations. These are confident-sounding responses that are factually incorrect or completely fabricated. While no technique eliminates hallucinations entirely, these three strategies significantly reduce their occurrence in production systems.

1. Provide an Escape Hatch

One of the most effective ways to reduce hallucinations is giving the model permission to admit uncertainty. LLMs are trained to be helpful, which sometimes leads them to generate plausible-sounding answers even when they lack sufficient information.

By explicitly allowing responses like "I don't know" or "The provided context doesn't contain this information," you give the model a legitimate alternative to fabrication.

Example: Question Answering with Context

You are a technical support assistant. Answer the user's question
based ONLY on the provided <documentation>.

<documentation>
{documentation}
</documentation>

If the answer is NOT clearly stated in the <documentation>,
respond with: "This information is not covered in the available documentation."

Do not speculate or provide information from outside the given context.

User question: {question}

This approach works because you're changing the model's optimization target from "provide an answer" to "provide an accurate answer or acknowledge the limitation."

2. Use Chain-of-Thought Verification

Chain-of-thought (CoT) prompting asks the model to show its reasoning before providing a final answer. This technique reduces hallucinations by forcing the model to ground each step in verifiable logic.

The key enhancement is adding a verification step where the model checks its own reasoning against the source material before committing to a final response.

Example: Document Analysis with Self-Verification

You are analyzing a technical <report> to extract key findings.

<report>
{report_content}
</report>

Use the <analysis> section to work through your reasoning:

<analysis>
1. Identify the main claims in the report
2. For each claim, note the specific paragraph or section where it appears
3. Verify each finding has direct support in the source text
4. Flag any conclusions that require inference beyond the stated facts
</analysis>

After completing your analysis, provide a <summary> containing
ONLY findings that are directly supported by the source text.

<summary>
[Your verified findings here]
</summary>

The explicit verification step catches many potential hallucinations before they reach the final output. The model essentially performs a self-audit of its reasoning.

3. Implement Confidence Thresholds

Instruct the model to evaluate its own confidence and only provide answers above a certain threshold. This technique is particularly useful for factual queries where accuracy is critical.

Example: Technical Query with Confidence Gate

You are a database expert answering technical questions.

Question: {technical_question}

Before answering, assess your confidence level:
- HIGH: You are certain this is correct based on established knowledge
- MEDIUM: You believe this is correct but there may be edge cases
- LOW: You are unsure or this requires verification

If your confidence is LOW, respond with:
"I recommend verifying this with official documentation before implementation."

If your confidence is MEDIUM or HIGH, provide your answer and note any caveats.

Confidence assessment: [HIGH/MEDIUM/LOW]
Answer:

This approach transforms the model from always providing answers to being selective about when to provide them—a significant shift that reduces false confidence.

Combining Techniques for Production Systems

In practice, the most robust systems combine all three approaches:

Escape hatches prevent fabrication when context is insufficient
Chain-of-thought verification catches logical errors before output
Confidence thresholds filter out uncertain responses

Remember: no technique eliminates hallucinations completely even if you are sober and don't take any drug.

Temperature Zero, Fixed Seed, Different Answers

Josep Oriol Carné — Sun, 08 Nov 2026 19:50:06 +0000

I was discussing reliability in AI agents when a familiar question came up: if you set an LLM's temperature to zero, doesn't that make it deterministic?

It sounds reasonable. Remove random sampling, always pick the most likely next token, and you get the same answer every time.

So I tried it on my laptop.

Same model. Same prompt. Temperature zero. Fixed seed. Five identical responses, then a different story on request six.

It happened on the very first execution of the script.

A small test with Ollama

I used llama3.1:8b through Ollama's local API, with these settings:

{
  "temperature": 0,
  "seed": 42,
  "num_ctx": 4096,
  "num_predict": 512
}

The prompt asked for a short story in Spanish about a programmer who finds a strange message in the server logs.

The Python script serializes the request body once and sends those exact bytes repeatedly. Requests are sequential, and no conversation history is passed between them.

It compares the generated text exactly and prints a short SHA-256 hash for each response. Timestamps and execution timings are excluded, since comparing those would produce meaningless differences.

Request six changed the story

Here are the response hashes from the captured output, with the diff omitted for readability (variantes counts the distinct responses seen so far):

01/30  hash=e8150452110a716b  variantes=1  fin=stop
02/30  hash=e8150452110a716b  variantes=1  fin=stop
03/30  hash=e8150452110a716b  variantes=1  fin=stop
04/30  hash=e8150452110a716b  variantes=1  fin=stop
05/30  hash=e8150452110a716b  variantes=1  fin=stop
06/30  hash=bf54c4fc8b524784  variantes=2  fin=stop
07/30  hash=e8150452110a716b  variantes=2  fin=stop
08/30  hash=e8150452110a716b  variantes=2  fin=stop
09/30  hash=e8150452110a716b  variantes=2  fin=stop

The first story featured the message "La respuesta está en el número 42" ("The answer is in the number 42").

The sixth response changed it to "La sombra se mueve en el rincón" ("The shadow moves in the corner") and developed a different plot.

Requests seven through nine returned to the original text.

This was a change in content, not just whitespace or formatting. The excerpt stops at request nine; it is not a completed 30-request report.

The complete script, settings and output diff are in this Gist.

It only uses Python's standard library. With Ollama running and the model installed, you can paste the script into a terminal and try it yourself.

What temperature zero actually controls

With temperature zero, Ollama falls back to greedy decoding: the model selects the token with the highest score. Given identical scores and consistent tie handling, that selection is deterministic.

The important qualification is given identical scores.

Those scores come from floating-point computation. Execution details such as GPU kernels, the order of operations and batching can slightly change the numbers. If two candidate tokens are close enough, a tiny difference can change which one wins.

Once a different token is selected, the next step receives a different prefix. From there, the rest of the answer can diverge substantially.

That is a plausible mechanism, not a diagnosis of my run. I did not trace Ollama's internals or record token scores, so I cannot attribute this result to a particular kernel, cache or scheduling decision.

For context, vLLM documents explicit batch invariance controls involving deterministic kernels and consistent numerical behavior. Reproducibility requires attention to the execution system, not just the decoding settings.

A fixed seed does not settle that problem either. It controls the random number generator used for sampling, not the arithmetic that produces the scores.

A counterexample, not a benchmark

The observation is simple: my local setup returned different text for identical requests specifying temperature=0 and seed=42.

It does not establish a general failure rate. Nor does it mean every model or runtime will behave this way.

Your installation might return the same answer every time. That would be a useful result too. This test checks repeatability under your conditions; it does not guarantee you will reproduce my divergence.

Repeatable is not the same as correct

There is another distinction worth keeping in mind when building agents: an answer can be perfectly repeatable and consistently wrong. Two different answers can also both be acceptable.

Exact reproducibility and task correctness need separate checks.

For my own systems, I want to measure repeatability where it matters, validate what the output means, and keep execution permissions and constraints in code.

Temperature zero is a setting. Reproducibility is a property of the system you actually run.

How to Disable the Co-Authored-By Trailer in Claude Code

Josep Oriol Carné — Sun, 08 Nov 2026 19:49:30 +0000

By default, every commit Claude Code makes for you ends with a line like this:

Co-Authored-By: Claude <noreply@anthropic.com>

Pull requests get a similar footer: 🤖 Generated with Claude Code. Some people like it. I don't, and this post shows how to turn it off in a few seconds.

Why I turn it off

Let's be honest: almost every developer uses an AI model in one way or another now. Autocomplete, chat, agents, a quick "explain this regex". The only exception is maybe the odd COBOL veteran keeping a bank mainframe alive, and even they are probably asking an LLM what a 1987 copybook does.

So a trailer saying "AI was involved" on every commit tells nobody anything new. It's noise in git log. I'm the one who reviews the diff, and I'm the one responsible for what gets merged.

And Anthropic doesn't need that line anyway. Claude's output already carries a text watermark: an imperceptible pattern in word choice, with no extra characters or tokens. According to Anthropic, the watermark contains no information about the user, their organization or their conversations. It just marks the text as Claude-generated.

So deleting the trailer doesn't hide anything from the people who made the model. It only removes a visible label from GitHub, which is mostly free advertising anyway.

The fix: the `attribution` setting

Claude Code reads its settings from JSON files. To remove the trailers for every project, edit your user settings at ~/.claude/settings.json:

{
  "attribution": {
    "commit": "",
    "pr": ""
  }
}

commit: text appended to commit messages. An empty string removes the Co-Authored-By trailer.
pr: text appended to pull request descriptions. An empty string removes the "Generated with Claude Code" footer.

If you only want it for one repository, put the same block in .claude/settings.json or .claude/settings.local.json inside that project.

You can also put your own text there instead of an empty string, if you want a custom footer.

Check it worked

Start a new Claude Code session (settings load at startup), ask it to commit something, then:

git log -1 --format=%B

No trailer, no footer.

Anecdote: the mystery contributor

Here's a story that made me want to write this down.

A developer noticed a stranger listed as a contributor on their GitHub repo. An account with a name like poop00po0o, which had never pushed a single commit there. They asked Claude Code to investigate, and it answered roughly like this:

Mystery solved. Every commit made with my help has this trailer:

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

GitHub resolves co-author trailers by matching the email address to a GitHub account. Someone has likely registered a GitHub account using the email noreply@anthropic.com, effectively squatting on that address so they show up as the "Claude" co-author in any repo that uses the Claude Code co-authorship convention.

This is not a security breach of your repo. No one has pushed code you didn't write. The "contributor" label is just GitHub resolving that email trailer to whoever holds that account.

The options it gave were sensible: ignore it (it's cosmetic), stop adding the trailer, and report it to Anthropic at github.com/anthropics/claude-code/issues, since it affects everyone pushing co-authored commits.

I can't confirm exactly how that account ended up linked to the address. GitHub normally only links commits to verified emails, so the "someone registered it" explanation is the agent's best guess, not a proven fact. But the mechanism is real: a Co-Authored-By line is just an email address, and GitHub will show whatever account it maps that email to on your repo's contributor list. You don't control that mapping.

That's one more reason to take the line out. Your commits should carry the names of the people responsible for them, and nobody else's.

I Replaced My Vector Database With Boring Algorithms

Josep Oriol Carné — Sun, 08 Nov 2026 19:44:58 +0000

I recently removed a vector database from a few parts of my software.

Not because vector databases are bad. Not because embeddings stopped being useful. And definitely not because I suddenly decided that AI infrastructure is overhyped.

The reason was much more boring: I was using the wrong tool for some of the jobs.

One of those jobs was source-code retrieval for RAG. The original version worked the way most first versions work: split the repository into chunks, embed each chunk, store the embeddings, and retrieve the chunks that looked closest to the user's query.

It was a reasonable starting point. It also worked well enough to keep around for a while.

But after using it more, I started noticing a pattern. A lot of the questions I was asking were not really semantic questions.

They were questions like:

Where is this function defined?
Which class owns this method?
Where is this symbol imported?
What calls this helper?
Where is this config value used?

For that kind of query, the source code already contains the answer explicitly. The information is sitting in the syntax tree, the import graph, the file names, the symbols, and the references. Turning all of that into embeddings first adds an indirect step that often does not buy you much.

So I replaced part of the vector search with much less exciting machinery: AST parsing, symbol indexes, exact lookups, reference tracking, and a few simple ranking rules.

The result was better than I expected.

For the workloads I tested, the boring version was faster, cheaper, easier to debug, and often more accurate. It was also far more predictable. When I asked for a function by name, I did not have to hope that the embedding space agreed with me. The index either had the symbol or it did not.

That determinism matters more than it gets credit for.

When the same input produces the same output, the system becomes easier to test. It becomes easier to reproduce bugs. It becomes easier to explain why something happened. And when your application already has an LLM inside it, adding more deterministic pieces around it is usually a good trade.

This made me reconsider an assumption I had been carrying around: that an AI-heavy product should also have AI-heavy infrastructure everywhere around it.

It does not.

Plenty of software problems are still better solved with databases, parsers, indexes, graphs, rules, and ordinary algorithms. The presence of an LLM in the product does not magically turn every subsystem into a machine learning problem.

The distinction I now find useful is this:

The LLM handles ambiguity. The software handles precision.

LLMs are excellent when the input is vague, incomplete, semantic, or expressed in natural language. They are good at interpreting intent, mapping fuzzy requests to possible actions, and explaining results back to a human.

Traditional software is excellent when the problem has structure. If there are explicit rules, known identifiers, exact relationships, constraints, or a well-understood algorithm, regular software usually wins.

Source-code retrieval is a good example. If I ask, "Where do we handle failed payments?", embeddings can help because the wording might not match the code. The relevant code may mention invoices, retries, billing states, or webhooks instead of "failed payments."

But if I ask, "Where is PaymentRetryPolicy defined?", I do not need a semantic search system to guess. I need an index.

The same pattern shows up outside of code too.

If the answer is in a database row, query the database. If it is in a graph, traverse the graph. If it follows from a rule, apply the rule. If it is in a parser-friendly format, parse it. Use embeddings when they add information, not just because they feel like the AI-native option.

The architecture I keep coming back to looks simple:

Human request
  -> LLM
  -> interpret ambiguity
  -> traditional software
  -> execute precisely
  -> LLM
  -> explain the result

In that setup, the LLM is not replacing the software. It is translating between the human and the software.

That is a much more useful role in many systems. Humans are messy. We ask incomplete questions, use the wrong terms, forget details, and describe what we want indirectly. LLMs are very good at cleaning up that mess.

But once the intent is clear, the execution should often be handed to something more precise. A SQL query. A parser. A search index. A constraint solver. A rules engine. A normal function call.

This is not an argument against vector databases, embeddings, or LLMs. I still use them, and there are problems where they are exactly the right tool.

It is an argument against adding uncertainty where the problem does not need it.

If the answer already exists in a syntax tree, a database index, a graph, or a deterministic rule, converting the problem into embeddings first may just make the system slower and harder to reason about.

The useful question is not, "Can an LLM solve this?"

Most of the time, with enough prompting and glue code, the answer is probably yes.

The better question is:

Which part of this problem actually needs an LLM?

That question leads to simpler systems. It also leads to better AI systems, because the model is used where it adds real value instead of being asked to compensate for infrastructure that should have been deterministic in the first place.

Human as a Tool: The Agent Pattern Where You Work for the Machine

Josep Oriol Carné — Sun, 08 Nov 2026 19:44:23 +0000

There's an agent design pattern with an uncomfortable premise: the agent is the orchestrator, and you are one of its tools.

It goes by a few names — human-as-a-tool, human-as-tool, sometimes lumped under "human-in-the-loop." But it's worth separating the two, because the inversion is the whole point.

Human-in-the-loop vs. human-as-a-tool

Human-in-the-loop keeps the human in charge. The agent proposes, the human approves. You're the supervisor signing off on the intern's work before it ships. Control flow belongs to you.

Human-as-a-tool flips it. The agent owns the plan. It decomposes the task, executes what it can, and when it hits a step that only a human can perform, it calls the human — with the same interface it uses to call a search API or a shell. You get a message: "I need the 2FA code from your phone." You answer. The agent continues.

In the first pattern, the human is management. In the second, the human is a blocking I/O call with unusually high latency.

What only humans can do (for now)

The pattern exists because there's a residue of tasks that no tool call can cover:

Physical actions. Plug in the cable, reboot the router, sign the paper, go look at the machine that's actually on fire.
Auth walls designed to stop machines. CAPTCHAs, 2FA codes, biometric checks, "click the link we sent to your phone." These are, by construction, human-only.
Legal and social standing. Only a human can consent, sign a contract, or make the phone call where the bank insists on hearing a voice.
Judgment and taste. "Which of these three logos feels right?" is a question with no assertion to check. Someone has to just decide.
Access the agent doesn't have. The password in your head, the context from a hallway conversation, the tribal knowledge that never made it into a doc.

Notice the parenthetical doing heavy lifting in this post's premise: for now. That list is not stable. Ten years ago "read the text in this blurry image" was on it. Computer-use agents are eating the physical-clicking category as we speak. The list only shrinks.

The implementation is almost insultingly simple

At its core, the pattern is one tool definition:

{
  "name": "ask_human",
  "description": "Ask the human operator. Expensive and slow — use only when no other tool can answer.",
  "parameters": {
    "question": "What you need, with enough context to answer without reading your whole transcript"
  }
}

That's it. LangChain has shipped it for years as HumanInputRun. Tools like HumanLayer wrap it in Slack approvals and async queues. Claude Code has AskUserQuestion. Every serious agent framework converges on the same shape, because the shape is obvious: a function call whose backend is a person.

The interesting engineering is in the details around it:

Make it expensive in the prompt. If the tool description doesn't scream "last resort," the agent will use you as a search engine. You become the bottleneck in your own automation.
Go async by default. A blocking prompt in a terminal is fine for a 5-minute task. For a long-running agent, the human call should be a notification — Slack, email, push — with the agent parked on that branch while it works on others.
Demand batching. An agent that interrupts you four times with one question each is worse than one that interrupts you once with four questions. This is worth stating explicitly in the system prompt.
Set timeouts and fallbacks. Humans are unreliable infrastructure. We sleep, we go to lunch, we ignore notifications. Decide what the agent does when the human call times out: retry, escalate, or degrade gracefully.
Log every call. The transcript of what the agent asked and what the human answered is your audit trail — and, later, your training data for automating that exact question away.

The uncomfortable part is the useful part

It's easy to read this pattern as dystopian — the machine dispatching work orders to its human peripheral. I think the honest reading is more practical: it's an inventory system.

Every ask_human call in your logs is a receipt for a task that resisted automation. Grep those logs after a month and you have something valuable: a ranked list of what actually requires you. Some entries are permanent (judgment, consent, taste). Most aren't. "Fetch the 2FA code" disappears the day you wire up a TOTP secret. "Which environment is prod?" disappears the day you write it down.

That's the discipline the pattern enforces, and it's the same one I apply everywhere else: treat yourself as a dependency to be minimized, not a supervisor to be flattered. Design the human tool with a clean interface — a question in, an answer out — and swapping the human for software later becomes a one-line change. The agent never even notices you left.

Where this is heading

The endgame is visible from here. The human-as-a-tool call rate is the single most honest metric of agent autonomy — far more honest than any benchmark. An agent that calls ask_human twenty times per task is a form with extra steps. One that calls it zero times doesn't need you.

The interesting number is in between, and it's dropping. My advice: implement the pattern, log the calls, and watch which of your irreplaceable human contributions quietly fall off the list.

For now, the agent still needs someone to plug in the cable. Enjoy it while it lasts.