DEV Community

HumanLayer Service
HumanLayer Service

Posted on

Stop Your AI Agents from Going Rogue: A Practical Guide to Human-in-the-Loop

#ai #automation #python #opensource

Stop Your AI Agents from Going Rogue: A Practical Guide to Human-in-the-Loop

Your AI agent just sent an email to your entire customer list.

The email was supposed to go to one person. The agent was confident. The agent was wrong.

Sound familiar? If you're building AI agents, you've probably had (or narrowly avoided) a moment like this.

The Autonomy Paradox

Here's the uncomfortable truth about AI agents in 2025:

The more capable they become, the more dangerous their mistakes.

An agent that can only answer questions is low-risk. An agent that can send emails, execute code, transfer money, and modify databases? That's a different story.

We want agents to be autonomous because that's the whole point — freeing humans from repetitive tasks. But full autonomy without oversight is a recipe for disaster.

The Solution: Selective Human-in-the-Loop

The answer isn't "approve everything" or "approve nothing." It's knowing which operations need human judgment.

Consider this spectrum:

Operation Risk Level Human Needed?
Fetch weather data 🟢 Low No
Search knowledge base 🟢 Low No
Draft email 🟡 Medium Maybe
Send email 🔴 High Yes
Transfer $100 🔴 High Yes
Transfer $10,000 🔴 Critical Definitely
Delete database records 🔴 Critical Definitely

The pattern: read operations are usually safe, write operations often aren't.

Introducing HumanLayer

We built HumanLayer to make human-in-the-loop trivially easy to implement.

Installation 

pip install humanlayer
Enter fullscreen mode Exit fullscreen mode

Basic Usage 

from humanlayer import HumanLayer

hl = HumanLayer()

@hl.require_approval()
def send_email(to: str, subject: str, body: str):
    """This function now requires human approval before executing."""
    return email_client.send(to, subject, body)
Enter fullscreen mode Exit fullscreen mode

That's it. One decorator.

When your agent calls send_email(), HumanLayer:

  1. ⏸️ Pauses execution
  2. 📱 Sends an approval request to your configured channel
  3. ⏳ Waits for human response
  4. ✅ Either executes or blocks based on the decision

Conditional Approval

Not everything needs approval. Use conditions:

@hl.require_approval(
    condition=lambda amount, **kwargs: amount > 1000
)
def transfer_money(amount: float, recipient: str):
    return bank.transfer(amount, recipient)
Enter fullscreen mode Exit fullscreen mode

Now transfers under $1000 execute immediately, while larger ones require approval.

Multiple Approval Channels

Meet your team where they work:

from humanlayer import HumanLayer
from humanlayer.channels import SlackChannel, EmailChannel

hl = HumanLayer(
    approval_channels=[
        SlackChannel(webhook_url="https://hooks.slack.com/..."),
        EmailChannel(to="approvals@company.com"),
    ]
)
Enter fullscreen mode Exit fullscreen mode

Real-World Use Cases

1. Sales Automation 

@hl.require_approval()
def send_proposal(client_email: str, proposal_pdf: bytes):
    """Sales team reviews before sending proposals."""
    return email_service.send_with_attachment(
        client_email, 
        "Your Custom Proposal", 
        proposal_pdf
    )
Enter fullscreen mode Exit fullscreen mode

2. Financial Operations 

@hl.require_approval(
    condition=lambda invoice, **kwargs: invoice.amount > 5000
)
def pay_invoice(invoice: Invoice):
    """Large invoices need CFO approval."""
    return accounting.process_payment(invoice)
Enter fullscreen mode Exit fullscreen mode

3. DevOps Automation 

@hl.require_approval()
def scale_infrastructure(service: str, replicas: int):
    """Ops team confirms scaling decisions."""
    return kubernetes.scale(service, replicas)

@hl.require_approval()
def delete_resources(resource_ids: list[str]):
    """Destructive operations always need approval."""
    return cloud.delete(resource_ids)
Enter fullscreen mode Exit fullscreen mode

Best Practices

1. Start Conservative

Begin by requiring approval for everything, then relax restrictions as you build confidence.

2. Provide Context

Give approvers enough information to make decisions.

3. Set Timeouts

Don't let pending approvals hang forever.

4. Log Everything

HumanLayer logs all decisions for audit trails.

Conclusion

Human-in-the-loop isn't about distrusting AI. It's about building AI systems that know their limits.

The best AI agents are the ones that:

  • ✅ Execute routine tasks autonomously
  • ✅ Recognize when they're uncertain
  • ✅ Ask for help on critical decisions
  • ✅ Learn from human feedback

HumanLayer makes this pattern easy to implement.

Links

What approval workflows would you add to your AI agents? Let me know in the comments!

Top comments (0)