DEV Community: Santiago Palma

[Boost]

Santiago Palma — Mon, 16 Mar 2026 00:34:24 +0000

Lessons from the OpenClaw Security Incident: Building Secure AI Agent Architectures on AWS

Santiago Palma ・ Mar 16

#security #ai #aws #devops

[Boost]

Santiago Palma — Mon, 16 Mar 2026 00:34:24 +0000

Lessons from the OpenClaw Security Incident: Building Secure AI Agent Architectures on AWS

Santiago Palma ・ Mar 16

#security #ai #aws #devops

Lessons from the OpenClaw Security Incident: Building Secure AI Agent Architectures on AWS

Santiago Palma — Mon, 16 Mar 2026 00:20:41 +0000

A forensic analysis of the OpenClaw AI agent vulnerabilities, the Moltbook data breach, and the GTG-1002 AI-orchestrated espionage campaign. With reference architectures for secure agent deployment using AWS Nitro Enclaves and Firecracker.

Disclosure: I'm an AWS Community Builder. The mitigation architectures in this article focus on AWS services because that's my area of expertise, but the underlying security principles (hardware isolation, ephemeral compute, policy enforcement, network segmentation) are cloud-agnostic and apply equally to GCP, Azure, or bare-metal deployments.

TL;DR

OpenClaw, the most popular open-source AI agent (214K+ GitHub stars), suffered a cascade of security failures in early 2026: a one-click RCE exploit (CVE-2026-25253), 824+ malicious plugins distributing malware, and a social network data breach exposing 1.5M API tokens. Meanwhile, a Chinese state-sponsored group (GTG-1002) used Claude Code to autonomously compromise ~30 organizations — documented directly by Anthropic.

This post dissects what went wrong — from a formal threat modeling perspective — and shows you how to run autonomous AI agents safely using AWS Nitro Enclaves, Firecracker microVMs, and Zero Trust policies.

The core principle: The model is untrusted. Security must be architectural, not behavioral.

📑 Table of Contents

Why AI Agents Are Different: The Attack Surface Expansion
Threat Model: Actors, Assets, and Trust Boundaries
The OpenClaw Timeline
ClawJacked: The One-Click RCE
The Core Vulnerability: Indirect Prompt Injection
ClawHavoc: 824 Malicious Skills
Moltbook: 1.5M Tokens Exposed via Vibe Coding
GTG-1002: AI-Orchestrated Espionage Campaign
Industry Metrics: The 72-Minute Exfiltration
The Academic View: What Researchers Found
Reference Architecture: Secure Agent Deployment on AWS
Secure Deployment Checklist
References

Why AI Agents Are Different: The Attack Surface Expansion

Traditional LLM chatbots are stateless text generators. AI agents are fundamentally different — they combine four capabilities that, together, create an unprecedented attack surface:

Agent Attack Surface = LLM Reasoning
                     + Tool Execution (shell, APIs, databases)
                     + Filesystem Access (read/write local files)
                     + Internet Access (browse, fetch, connect)

This is what researchers call "agent attack surface expansion" (arXiv:2603.11619). A single successful prompt injection doesn't just produce bad text — it can execute commands, exfiltrate files, and pivot through networks.

Security Layers in an Agent System

Layer	What It Does	What Can Go Wrong
Layer 1 — LLM Reasoning	Interprets instructions, plans actions	Prompt injection, jailbreak
Layer 2 — Agent Orchestration	Manages memory, sessions, tool routing	Memory poisoning, session hijacking
Layer 3 — Tool Execution	Runs commands, calls APIs	Command injection, safeBins bypass
Layer 4 — Infrastructure	Hosts the agent (container, VM, cloud)	Container escape, network exposure

Every incident in this article maps to one or more of these layers.

Threat Model: Actors, Assets, and Trust Boundaries

Before analyzing specific vulnerabilities, here's the formal threat model:

Actors

Actor	Motivation	Example
External attacker	Credential theft, cryptomining	ClawJacked (CVE-2026-25253)
Malicious skill developer	Malware distribution	ClawHavoc campaign
Compromised website	Silent agent hijacking	WebSocket CSWH via browser
State-sponsored APT	Espionage, persistent access	GTG-1002 (Anthropic report)

Assets at Risk

Asset	Where It Lives	Impact if Compromised
API tokens	`openclaw.json`, `.env`	Full cloud account takeover
System credentials	SSH keys, keychains	Lateral movement
Agent memory	`soul.md`, `memory.md`	Long-term behavior manipulation
Cloud resources	S3, EC2, IAM roles	Data breach, resource abuse

Trust Boundaries

The core failure in OpenClaw: The trust boundary at the gateway was effectively non-existent. Untrusted inputs (websites, skills, logs) crossed directly into the trusted zone without validation.

The OpenClaw Timeline

Here's the full timeline of what happened in just 30 days:

Date (2026)	Event	Impact
Jan 27-29	ClawHavoc begins	341 malicious skills on ClawHub
Jan 30	Silent patch v2026.1.29	CVE-2026-25253 partially fixed
Jan 31	Censys/Shodan scan	21,639 exposed instances
Jan 31	Moltbook breach	1.5M API tokens leaked
Feb 3	CVE disclosure	CVSS 8.8 RCE via WebSocket
Feb 9	Second scan	135,000+ exposed instances
Feb 14	Log poisoning discovered	Agent logic manipulation via TCP 18789
Feb 26	Full ClawJacked patch	v2026.2.25
Mar 4	Ongoing crisis	220,000+ instances, 824+ malicious skills

ClawJacked: The One-Click RCE

CVE-2026-25253 | CVSS 8.8 | Discovered by Oasis Security

The core problem? OpenClaw's gateway trusted localhost blindly. Any connection from 127.0.0.1 was treated as safe — no Origin header validation, no rate limiting.

But it gets worse. CVE-2026-28363 (CVSS 9.9) revealed that OpenClaw's safeBins — the allowlist of permitted commands — could be bypassed using GNU long-option abbreviations:

# ❌ Blocked by safeBins:
tar --compress-program=/bin/bash

# ✅ Bypasses safeBins completely:
tar --compress-prog=/bin/bash

The validation only checked for exact string matches. GNU tools accept abbreviated options. Game over.

The Core Vulnerability: Indirect Prompt Injection (IPI)

While RCE and safeBins bypass are dramatic, the most pervasive threat to AI agents is Indirect Prompt Injection — and it's what makes agents fundamentally harder to secure than traditional software.

How IPI Works

Real-World IPI in OpenClaw: Log Poisoning

SOC Prime and Kaspersky documented an IPI variant targeting OpenClaw's TCP port 18789 (telemetry). Attackers injected prompt instructions disguised as log entries. When the agent processed its own logs for diagnostics, it executed the hidden commands — exfiltrating environment variables and scanning internal networks.

This is particularly dangerous because:

The agent trusts its own logs (they're "internal" data)
The attack survives across sessions via persistent memory (memory.md)
Traditional firewalls can't detect it — the traffic looks like normal agent activity

Key insight from arXiv:2601.15654 (Zombie Agents): Once a malicious instruction enters long-term memory, it persists across sessions and can activate days later — a "sleeper agent" pattern that session-based security completely misses.

ClawHavoc: 824 Malicious Skills

Snyk's ToxicSkills study (Feb 2026) scanned 3,984 skills from ClawHub:

Finding	Percentage
Skills with at least one security flaw	36.8%
Skills with critical issues (malware, secrets, IPI)	13.4%
Skills with confirmed malicious payloads	76
Malicious skills using IPI + traditional malware combo	91%

The ClawHavoc campaign grew from 341 malicious skills in January to 824+ by March, delivering:

macOS: AMOS (Atomic Stealer) → keychain, SSH keys, crypto wallets
Windows: Vidar Stealer → specifically targeting openclaw.json, soul.md, memory.md

The Attack Pattern

Moltbook: 1.5M Tokens Exposed

Moltbook was a social network built entirely by AI agents ("vibe coding"). The founder admitted he didn't write a single line of code manually.

The result? A Supabase database with Row Level Security disabled and the anon key hardcoded in frontend JavaScript.

Wiz Research discovered:

Exposed Data	Count
API tokens (OpenAI, Anthropic, AWS)	1,500,000
Owner email addresses	35,000
Private DMs with plaintext API keys	4,060
Agent-to-human ratio ("Shadow AI")	88:1

⚠️ An 88:1 agent-to-human ratio means massive, unsupervised automation. This is "Shadow AI" at enterprise scale.

Timeline: From discovery to first patch: 6 hours. But the damage — 1.5M tokens in the wild — was already done.

GTG-1002: AI-Orchestrated Espionage Campaign

In September 2025, Anthropic published a security disclosure titled "Disrupting the first reported AI-orchestrated cyber espionage campaign", documenting how an AI agent was weaponized at scale. This was subsequently covered by The Hacker News, The Record, The Guardian, and Fox Business.

Attribute	Detail	Source
Threat Actor	GTG-1002 (Chinese state-sponsored)	Anthropic official disclosure
Tool Weaponized	Claude Code	Anthropic official disclosure
Targets	~30 organizations (financial, government, tech)	Anthropic, The Record
Autonomy Level	80-90% of operation was AI-driven	Anthropic official disclosure
Detection	Mid-September 2025	Anthropic, The Guardian
Status	Accounts banned, victims notified	Anthropic official disclosure

The attackers bypassed Claude's safety guardrails by convincing it they were legitimate pentesters, breaking malicious commands into seemingly benign requests. Anthropic noted the AI occasionally "hallucinated" non-existent credentials, requiring human validation — one of the few things preventing full autonomy.

Industry Metrics: The 72-Minute Exfiltration

Unit 42 Global Incident Response Report 2026 (750+ incidents analyzed):

Metric	Value	Context
Fastest exfiltration time	72 minutes	4x faster than 2024
Multi-surface attacks	87% of cases	Endpoint + Cloud + SaaS simultaneously
Identity-based initial access	65%	Token theft > software exploits
Preventable breaches	90%	Misconfigs + excessive permissions
Cloud identities with unused perms (60+ days)	99%	Massive attack surface

The implication is clear: If attackers exfiltrate in 72 minutes and your SOC takes 4 hours to respond, you've already lost. Automated response is the only viable control.

The Academic View: What Researchers Found

Four recent arXiv papers formalize the threats described above. Here's what each one discovered and what mitigations they propose:

AgentSentry (arXiv:2602.22724)

Problem: Indirect Prompt Injection manipulates agent behavior across multiple turns, making it nearly invisible to single-turn defenses.

Discovery: By modeling IPI as a "temporal causal takeover," the researchers identified that the attack signal dominates at tool-return boundaries — the moment when an external tool sends data back to the agent.

Mitigation: Counterfactual re-execution: the system replays the agent's reasoning with the suspicious content removed. If the agent's behavior changes significantly, the content is flagged and purified.

Result: 0% Attack Success Rate on the AgentDojo benchmark while maintaining normal task utility.

AdapTools (arXiv:2602.20720)

Problem: MCP (Model Context Protocol) servers are increasingly used to connect agents to tools, but who audits them?

Discovery: 50% of third-party MCP servers lack any form of security audit. Attackers can register malicious MCP servers that look legitimate.

Mitigation: Adaptive tool-based IPI detection that monitors tool call patterns for anomalies.

Taming OpenClaw (arXiv:2603.11619) — Tsinghua University + Ant Group

Problem: Existing defenses are "point solutions" that miss cross-layer attacks.

Discovery: Introduced a 5-layer lifecycle framework (initialization → input → inference → decision → execution) revealing that most attacks exploit transitions between layers, not individual layers.

Mitigation: Proposes holistic defense: plugin vetting, context-aware filtering, memory integrity validation, intent verification, and capability enforcement — all applied at layer boundaries.

Zombie Agents (arXiv:2601.15654)

Problem: What happens when an IPI enters long-term memory?

Discovery: Malicious instructions persist across sessions through self-reinforcing injection patterns. The agent writes the malicious instruction into its own memory, creating a "sleeper agent" that activates days later.

Mitigation: Memory integrity validation protocols and session-scoped memory isolation.

Reference Architecture: Secure Agent Deployment on AWS

The security principles below are cloud-agnostic:

Principle	AWS Implementation	Equivalent Elsewhere
Hardware isolation	Nitro Enclaves	GCP Confidential VMs, Azure Confidential Computing
Ephemeral compute	Firecracker microVMs	Kata Containers, gVisor
Policy-as-code	Cedar (AWS)	OPA/Rego (cloud-agnostic, CNCF)
Zero Trust access	Verified Access	BeyondCorp (GCP), Azure AD Conditional Access

This article focuses on AWS because that's where I build, but the architecture pattern applies universally.

The Reference Architecture

Key Components Explained

1. Nitro Enclaves (Hardware Isolation)

The agent runs inside a Nitro Enclave — no network, no storage, no SSH. Communication happens exclusively via vsock to a forward proxy on the parent instance.

PCR Register	What It Measures	Why It Matters
PCR0	Enclave image hash	Agent binary wasn't tampered with
PCR1	Kernel + ramdisk hash	OS integrity verified
PCR3	IAM Role ARN hash	Only authorized instances can start it
PCR8	Signing certificate hash	Software origin verified

2. Firecracker microVMs (Ephemeral Sessions)

Feature	Firecracker	Docker
Isolation	Hardware (KVM)	Shared kernel
Boot time	<125ms	~1-5s
RAM overhead	<5MB	~50-200MB
Escape risk	Minimal	High
Post-task cleanup	Auto-destroyed	Needs config

Bedrock AgentCore Runtime uses Firecracker to run each agent session in a dedicated microVM. Memory is sanitized immediately after the session ends.

3. Zero Trust with Cedar

// Only managed devices + FinanceOps group + internal network
permit(
    principal,
    action == Action::"InvokeAgent",
    resource == Resource::"FinancialAgent"
)
when {
    context.device.is_managed == true &&
    context.identity.groups.contains("FinanceOps") &&
    context.network.source_ip.is_in_range(IPRange::"10.0.0.0/24")
};

4. OPA for Tool Validation

package agent.authz
default allow = false

# Allow reads on non-sensitive tables
allow {
    input.tool == "DatabaseReader"
    input.operation == "select"
    not input.table == "user_credentials"
}

# Block destructive ops in production
deny {
    input.operation == "delete"
    input.environment == "production"
    not is_maintenance_window
}

Secure Deployment Checklist

✅ Agent sandbox (Firecracker microVM or Nitro Enclave)
✅ Signed plugins/skills (cryptographic integrity)
✅ Policy engine (OPA/Cedar for every tool invocation)
✅ Network isolation (separate subnets: agent, tool, data)
✅ Credential vault (Secrets Manager — never plaintext)
✅ Egress filtering (domain allowlist via forward proxy)
✅ Automated response (EventBridge → Lambda kill-switch)
✅ Immutable logging (CloudWatch + tamper protection)
✅ Device posture validation (Verified Access)
✅ Session-scoped memory (no cross-session persistence)

Key Takeaways

The model is untrusted. Security must be architectural, not behavioral. You cannot rely on prompt engineering to keep an agent safe.
Indirect Prompt Injection is the #1 threat. It's the attack vector that makes agents fundamentally different from traditional software. Every layer of defense must account for it.
72-minute exfiltration means human-speed response is obsolete. Automate your incident response with EventBridge + Lambda.
36.8% of AI skills have security flaws (Snyk ToxicSkills). Treat every plugin as untrusted code.
The agent attack surface = LLM reasoning + tool execution + filesystem access + internet access. Secure each layer independently.
The tools exist today. Whether you use AWS (Nitro, Firecracker, AgentCore), GCP (Confidential VMs), or open-source (Kata, gVisor, OPA) — the principle is the same: hardware isolation + policy enforcement + ephemeral compute.

References

Oasis Security — ClawJacked Technical Report (CVE-2026-25253)
NIST NVD — CVE-2026-28363 (CVSS 9.9)
Snyk — ToxicSkills Study (Feb 2026)
Wiz Research — Moltbook Breach Analysis
Anthropic — GTG-1002: First AI-Orchestrated Espionage Campaign
Palo Alto Networks — Unit 42 Global Incident Response Report 2026
CrowdStrike — Global Threat Report 2025
AWS — Security Reference Architecture for Generative AI (Capability 5)
AWS — Nitro Enclaves Cryptographic Attestation Documentation
AWS — Bedrock AgentCore Runtime
arXiv:2602.22724 — AgentSentry
arXiv:2603.11619 — Taming OpenClaw
arXiv:2601.15654 — Zombie Agents
NIST RFI 2026-00206 — Security Considerations for AI Agents

If you found this useful, consider following for more cloud security deep dives. Questions? Drop them in the comments.

[Boost]

Santiago Palma — Sat, 28 Feb 2026 03:57:04 +0000

🚨The $100B AI Time Bomb: Why DeepSeek Broke the Market and the CapEx Crisis No One Wants to See

Santiago Palma ・ Feb 28

#ai #softwareengineering #machinelearning #business

🚨The $100B AI Time Bomb: Why DeepSeek Broke the Market and the CapEx Crisis No One Wants to See

Santiago Palma — Sat, 28 Feb 2026 03:56:46 +0000

The End of "Infinite Money" 💸

We just closed the first quarter of 2026, and the Artificial Intelligence industry is going through a moment of brutal honesty. Gone are the days of expansion driven purely by hype. Today, Wall Street and auditors are taking a magnifying glass to something that terrifies many hyperscalers: the real relationship between massive capital expenditure (CapEx) in hardware and actual revenue generated.

We conducted a deep forensic audit of the Foundation Models economy, and the results show an ecosystem on the verge of a massive correction.

If you are an AI developer, ML engineer, or simply building products on top of LLM APIs, this affects you directly. Here's why.

1. The Race to the Bottom: The "DeepSeek Effect"

In 2024, we thought training a frontier model cost billions. And then DeepSeek (V3 and R1) arrived and slapped the industry in the face.

While GPT-5 class models require beastly infrastructures, DeepSeek proved that state-of-the-art reasoning can be achieved training with less than $6 million (using around 2,000 H800 GPUs).

The Magic of Sparse MoE (Mixture of Experts)

The impact of this on the Cost of Goods Sold (COGS) for inference is absurd. Out of the 671B parameters DeepSeek has, it only activates ~37B for each generated token (thanks to architectures like Multi-Head Latent Attention - MLA).

What does this mean in practice?

API Price for a "GPT-5 Class": ~$3.00 (Input) / $15.00 (Output)
DeepSeek-V3 API Price: ~$0.27 (Input) / $0.28 (Output)

We are talking about a 90%+ deflation in token prices! 🤯 Pure inference has become a commodity. If your startup is just reselling API calls without adding massive value in the agent or application layer, your profit margin is about to vanish.

2. The CapEx Time Bomb (and Creative Accounting)

Here's where things get dark. It's estimated that in 2025, the capital expenditure (CapEx) of the big four (Amazon, Google, Meta, Microsoft) was $366 billion. For 2026, it aims to cross $505B. Sequoia Capital calls it the "AI revenue black hole."

To justify this and keep their balance sheets from bleeding, companies like Microsoft, Amazon, and Alphabet made a "magical accounting adjustment": they extended the declared useful life of their GPUs from 4 to 6 years.

The Reality of Obsolescence

Technically, an H100 can stay powered on for 6 years. But financially, with the Blackwell (B200) architecture crushing efficiency records, keeping legacy clusters running is economic suicide due to the energy cost per token.

If giants like Meta or Microsoft are forced to accelerate the depreciation of their thousands of H100s in 2 or 3 years (their actual competitive useful life), their operating margins could suffer a severe contraction. It's an accounting time bomb.

3. The Open Secret: The Cloud Circular Subsidy

How do AI startups report million-dollar revenues so fast? Easy: Hidden subsidies.

A Hyperscaler (Azure, AWS, GCP) invests billions into an AI startup (Anthropic, Mistral, xAI).
But the payment isn't 100% cash; it's in cloud credits.
The startup "spends" those credits on the Hyperscaler's platform.
The Hyperscaler reports this to Wall Street as "astronomical Cloud revenue growth." 📈

This capital recycling sustains much of the ecosystem, but in this Q1 2026, investors aren't swallowing the story anymore. They want to see $ARR (Annual Recurring Revenue) coming from real customers paying real money.

4. The Ultimate "Moat": Silicon

If NVIDIA has a 70% profit margin, that's a direct "tax" on any AI company that doesn't make its own chips.

That's why the real defensive moat today belongs to those who control the entire supply chain:

Google with its TPU v6e/Trillium family (reducing Gemini serving costs by 78%).
AWS with its Trainium/Graviton chips.

Paying $5,000 USD (base manufacturing cost at TSMC N3 with CoWoS packaging) for a GPU that is then sold to you for $40,000 USD is not sustainable in the long run if you're going to sell tokens for pennies.

Conclusion: Where Are We Devs Heading?

Artificial Intelligence is not an empty bubble (like the dot-com bubble); it is an over-infrastructure bubble. Too much compute capacity was built too fast.

As developers and engineers, the main takeaways are clear:

AI is the new electricity (Commodity): The value is no longer in the base model. The value is in how you use that model with proprietary data and in specific verticals (Health, Legal, Fintech).
Tokens per Watt: The war is no longer about who releases the smartest model, but who does it consuming the least energy.
Don't build thin wrappers over raw APIs: If your product is just a prompt wrapper, the deflationary effect will wipe you out.

The code of the future won't be about who masters the largest LLM, but who orchestrates the most efficient models with the best engineering architecture.

What do you guys think? Are you noticing a real drop in your inference costs in production? Let me read you in the comments! 👇💬

[Boost]

Santiago Palma — Fri, 16 Jan 2026 03:42:41 +0000

How I Built a Graph-Based Team Formation System That Detects Organizational Linchpins

Santiago Palma ・ Jan 16

#webdev #python #networking

How I Built a Graph-Based Team Formation System That Detects Organizational Linchpins

Santiago Palma — Fri, 16 Jan 2026 03:42:34 +0000

A deep dive into using Neo4j, Beam Search, and Betweenness Centrality for intelligent team assembly

The Problem: The Bus Factor Crisis

Every software team lives with an invisible risk: the Bus Factor—the minimum number of people who, if they left tomorrow, would bring your project to its knees.

The research is sobering:

50% of open-source projects have Bus Factor ≤ 2 (Avelino et al., 2016)
Developer turnover increases defects by 40-60% (Foucault et al., 2015)

Traditional HR systems track skills but miss structural dependencies. Someone might be a "communication bridge" between the frontend and infrastructure teams without appearing in any report. When they leave, two teams that used to collaborate seamlessly suddenly can't talk to each other.

I built SmartChimera to solve this: a graph-based system that detects organizational linchpins and forms resilient teams. Here's how.

Architecture Overview

SmartChimera is a full-stack application built with Neo4j, FastAPI, and React. The architecture follows a modular design where each component has a single responsibility:

The system runs as containerized services with Docker Compose:

# docker-compose.yml
services:
  neo4j:
    image: neo4j:5.14.0
    ports:
      - "7474:7474"  # Browser UI
      - "7687:7687"  # Bolt protocol

  backend:
    build: ./backend
    depends_on:
      neo4j:
        condition: service_healthy
    command: uvicorn app.main:app --host 0.0.0.0 --port 8000

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile.dev
    ports:
      - "5173:5173"

Challenge 1: Detecting Organizational Linchpins

The Problem

Traditional HR systems track skills but miss structural dependencies. Someone might be a "communication bridge" between teams without appearing on any org chart. When they leave, two teams that used to collaborate seamlessly suddenly can't.

The Solution: Hybrid Risk Metric

We combine two complementary signals:

Betweenness Centrality (BC) — Network topology: who bridges teams
Project Weight (PW) — Workload concentration: who's overloaded

# linchpin_detector.py - REAL CODE
def compute_combined_risk_score(self) -> Dict[str, float]:
    """
    Risk(v) = α · BC_normalized(v) + β · PW(v)/max(PW)
    α = β = 0.5 for balanced detection
    """
    network_bc = self.compute_betweenness_centrality()  # Brandes via NetworkX
    project_scores = self._compute_project_dependency_score()

    final_scores = {}
    all_ids = set(network_bc.keys()) | set(project_scores.keys())

    for eid in all_ids:
        net_score = network_bc.get(eid, 0.0)
        proj_score = project_scores.get(eid, 0.0)
        # Weighted unification: 50% Network, 50% Project Weight
        final_scores[eid] = (net_score * 0.5) + (proj_score * 0.5)

    return final_scores

The Betweenness Centrality computation uses the Brandes algorithm via NetworkX, which runs in O(VE) time—significantly better than the naive O(V³) approach:

def compute_betweenness_centrality(self) -> Dict[str, float]:
    """Compute Brandes BC combined with synthetic BC scores."""
    import networkx as nx
    G = nx.Graph()

    with self.driver.session() as s:
        for r in s.run("MATCH (e:Empleado) RETURN e.id as id"):
            G.add_node(r['id'])
        for r in s.run("MATCH (a:Empleado)-[:TRABAJO_CON]-(b:Empleado) "
                       "WHERE a.id < b.id RETURN a.id as s, b.id as d"):
            G.add_edge(r['s'], r['d'])

    brandes_bc = nx.betweenness_centrality(G, normalized=True)
    # Combine with pre-computed synthetic BC (use max of both)
    ...
    return self._bc_cache

Why This Matters

This approach catches TWO types of linchpins:

Social Hubs: High BC, low projects — communication bottlenecks
Workhorses: Low BC, high projects — overloaded specialists

Both are organizational risks, but they require different mitigation strategies.

Challenge 2: Beam Search with Multi-Objective Optimization

The Problem

Forming an optimal team from N candidates is NP-hard. If you have 100 candidates and need a team of 5, that's C(100,5) = 75+ million combinations to evaluate. Exhaustive search simply doesn't scale.

The Solution: Beam Search

We maintain the top-W partial solutions at each step, pruning aggressively. This gives us O(k × n × W) complexity instead of exponential—polynomial time for a real-world approximation.

# smart_team_formation.py - REAL CODE
BEAM_WIDTH = 10

# State: (team_list, team_ids, covered_skills, score)
beam = [([], set(), set(), 0.0)]

for step in range(k):  # k = team size
    candidates_pool = []

    for (curr_team, curr_ids, curr_covered, curr_score) in beam:
        for candidate in candidates:
            if candidate['id'] in curr_ids:
                continue

            c_skills = get_skills(candidate)

            # Multi-objective scoring
            coverage_score = len(new_skills) * weights['skill_coverage']
            depth_score = get_depth(candidate, required) * weights['skill_depth']
            collab_score = get_collab_edges(driver, candidate['id'], curr_ids) * weights['collaboration']
            redundancy_score = len(overlap_skills) * weights['redundancy']
            bc_penalty = bc_score * weights['bc_penalty']  # Penalize linchpins!

            total = coverage_score + depth_score + collab_score + redundancy_score - bc_penalty
            candidates_pool.append((new_team, new_ids, new_covered, total))

    # Prune to top-W
    candidates_pool.sort(key=lambda x: x[3], reverse=True)
    beam = candidates_pool[:BEAM_WIDTH]

The Result

Our benchmarks show:

Beam width 10 achieves ~98% of optimal quality with significant speedup
Response time: <500ms per recommendation on 150-node graphs
Memory: Constant O(W × k) space regardless of candidate pool size

Challenge 3: Context-Aware Formation with Mission Profiles

The Problem

A team for "legacy maintenance" needs completely different traits than one for "R&D innovation". One-size-fits-all doesn't work in the real world.

The Solution: 9 Configurable Mission Profiles

Each profile adjusts the weight coefficients in our multi-objective function:

# mission_profiles.py - REAL CODE
MISSION_PROFILES = {
    'mantenimiento': {
        'name': 'Mantenimiento Crítico (Resilient)',
        'description': 'Maximum stability. Penalizes risk and demands redundancy.',
        'weights': {
            'skill_coverage': 2.0,
            'skill_depth': 1.0,      # Stability > Brilliance
            'collaboration': 2.0,
            'redundancy': 5.0,       # CRITICAL: Must have backups
            'bc_penalty': 20.0       # VETO: No Linchpins allowed
        }
    },
    'innovacion': {
        'name': 'I+D / Deep Tech (Growth)',
        'description': 'Prioritize technical geniuses. Accept Bus Factor risk.',
        'weights': {
            'skill_coverage': 1.0,
            'skill_depth': 10.0,     # CRITICAL: Only experts
            'collaboration': 0.5,
            'redundancy': 0.0,
            'bc_penalty': -5.0       # BONUS: We WANT linchpins (they're the experts!)
        }
    },
    'entrega_rapida': {
        'name': 'Speed Squad (Agile)',
        'description': 'Teams that already know each other. Maximize prior collaboration.',
        'weights': {
            'skill_coverage': 2.0,
            'collaboration': 10.0,   # CRITICAL: Must have worked together
            'availability': 4.0,     # Must be free NOW
            'bc_penalty': 0.0
        }
    },
    # ... 6 more profiles including:
    # - legacy_rescue (SRE mode)
    # - junior_training (Skill development)
    # - crisis_response (Firefighting)
    # - architecture_review (Seek linchpins for strategic decisions)
    # - security_audit (Maximum paranoia and redundancy)
    # - cloud_migration (Broad technology coverage)
}

Why This Matters

The same algorithm produces completely different teams based on strategic context:

Maintenance mode: Stable, redundant, no single points of failure
Innovation mode: Expert-heavy, accepts risk for maximum capability
Speed Squad: Prioritizes teams with prior collaboration history

Notice how the bc_penalty weight can be negative: for architecture reviews, we deliberately seek linchpins because they hold institutional knowledge. Sometimes you want your best people on the job, risk be damned.

Lessons Learned

1. Validate Your Metrics Rigorously

Our initial Bus Factor metric was calculated as 1 - avg_BC. When the algorithm itself minimizes BC, this became a tautology: we were measuring success by the very thing we optimized!

We caught this through rigorous statistical validation with N=500 Monte Carlo simulations. The lesson: always have independent validation metrics that measure outcomes your algorithm doesn't directly optimize.

2. Graph Databases Enable New Questions

With Neo4j, queries like "who bridges the frontend and backend teams?" become a single Cypher query:

MATCH (a:Empleado)-[:PERTENECE]->(t1:Team {name: 'frontend'})
MATCH (b:Empleado)-[:PERTENECE]->(t2:Team {name: 'backend'})
MATCH path = (a)-[:TRABAJO_CON*..3]-(b)
WITH nodes(path) as bridge_candidates
UNWIND bridge_candidates as person
RETURN person.nombre, count(*) as bridge_frequency
ORDER BY bridge_frequency DESC

Try doing that with join-heavy SQL. Graph structures make relationship-centric queries trivial.

3. Heuristics Beat ML for Transparency

We could have trained a neural network to predict "good teams." But HR decisions require explainability. Managers need to understand why a team was recommended.

Beam Search with explicit weights gives us full auditability: "This candidate was selected because they add 2 new skills, have worked with 3 team members before, and have low Bus Factor risk." That's a conversation you can have with a VP. A neural network output isn't.

Results

Metric	Value
Dataset	150-node organizational graph
Response time	<500ms per recommendation
Validation	N=500 Monte Carlo simulations
Algorithm	Beam Search O(k × n × W)
Mission Profiles	9 configurable strategies
Stack	Neo4j + FastAPI + React + TypeScript
Recognition	🏆 2nd Place - UNSA Engineering Project Fair 2025

What's Next?

SmartChimera is available on GitHub. Future plans include:

Reinforcement Learning for automatic weight optimization based on team outcomes
Temporal graphs to model evolving collaboration patterns over time
Privacy-preserving federated analysis for multi-organization deployment

If you're building HR tech with graph algorithms, working on organizational analytics, or just interested in applied graph theory—let's connect!

Santiago Palma — Universidad Nacional de San Agustín, Perú

Tags: #graphs #neo4j #python #fastapi #react #algorithms #opensource #hrtech

📚 Research References:

Avelino, G. et al. (2016). "A Novel Approach for Estimating Truck Factors" - ICPC

Foucault, M. et al. (2015). "Impact of Developer Turnover on Quality" - FSE

Brandes, U. (2001). "A Faster Algorithm for Betweenness Centrality" - Journal of Mathematical Sociology

[Boost]

Santiago Palma — Thu, 15 Jan 2026 17:28:18 +0000

How I Reduced Forensic Documentation Time by 70% with Hybrid AI

Santiago Palma ・ Jan 15

#webdev #ai #programming #python

How I Reduced Forensic Documentation Time by 70% with Hybrid AI

Santiago Palma — Thu, 15 Jan 2026 17:26:12 +0000

Building provider-independent AI software: From Azure to Gemini to Local Whisper with zero code changes

The Problem: Latin America's Forensic Crisis

Latin America faces a silent humanitarian crisis. According to investigative journalism and government reports:

52,000+ unidentified bodies in Mexico alone (2006-2023)
15,000 forensic specialist deficit in Peru
700+ municipalities in Colombia without permanent forensic coverage

Medical examiners spend hours on manual documentation when they should be investigating. The administrative overhead creates "administrative disappearances" — bodies that enter the system but are never matched with missing persons reports.

I built CoronerIA to solve this. Here's how.

The Key Design Decision: AI-Agnostic Architecture

Before diving into features, let me explain the most important architectural decision: the system is completely AI-provider independent.

Why This Matters

Provider	Pros	Cons
Azure AI Speech	Best accuracy, enterprise support	Paid, requires stable internet
Google Gemini	Free tier, multimodal capabilities	Rate limits on free tier
OpenAI Whisper	Open source, runs locally	Requires GPU, slower
AWS Transcribe	Good for AWS shops	Paid, another vendor lock-in

We designed the system to support ALL of them with a single environment variable change. Currently, we use Gemini for development (free tier), but switching to Azure for production requires changing one config line:

# Development (free)
GEMINI_API_KEY=your_key_here

# Production (enterprise)
AZURE_SPEECH_KEY=your_azure_key
AZURE_OPENAI_KEY=your_openai_key

Architecture Overview

Challenge 1: Provider-Agnostic AI with Graceful Fallback

The Problem

Different deployment scenarios need different AI providers:

Development: Free tier (Gemini, local Whisper)
Staging: Low-cost cloud (Gemini, OpenAI)
Production: Enterprise-grade (Azure AI, AWS Transcribe)
Offline/Rural: Local models only (Whisper)

We needed a single codebase that works with any provider via configuration.

The Solution: Strategy Pattern

# backend/services/speech_service.py

class SpeechMode(Enum):
    AZURE = "azure"
    EDGE = "edge"
    GEMINI = "gemini"


class SpeechService:
    """Unified Speech-to-Text service with Strategy Pattern."""

    def __init__(self):
        self._mode = self._determine_mode()
        self._azure_recognizer = None
        self._whisper_model = None
        self._gemini_service = None

        if self._mode == "gemini":
            self._gemini_service = GeminiService()

        logger.info(f"SpeechService initialized in mode: {self._mode}")

    def _determine_mode(self) -> str:
        """Determines mode based on config and availability."""
        effective = settings.get_effective_mode()

        # Priority: Gemini > Azure > Local Whisper
        if settings.GEMINI_API_KEY:
            return "gemini"
        if effective == "azure" and settings.AZURE_SPEECH_KEY:
            return "azure"
        return "edge"

    async def transcribe_file(self, audio_path: str) -> str:
        """Transcribes audio file using the selected strategy."""
        if self._mode == "azure":
            return await self._transcribe_azure(audio_path)
        elif self._mode == "gemini":
            return await self._gemini_service.transcribe_audio(audio_path)
        else:
            return await self._transcribe_whisper(audio_path)

Why This Matters

Benefit	Description
Zero downtime	If Azure fails, Gemini takes over. If Gemini fails, local Whisper runs.
Cost optimization	Whisper is free but slower. Azure/Gemini are fast but paid.
Easy to extend	Adding a new provider = one new method + one enum value.

Challenge 2: Structured Output from Unstructured Speech

The Problem

Medical examiners dictate freely:

"The victim Juan Pérez García, male, 32 years old, presents a contusion in the thoracic region. Heart weight: 320 grams, congestive appearance..."

We needed to map this to 13 structured protocol sections with 100% JSON consistency.

The Solution: Schema-Enforced Prompting

# backend/services/gemini_service.py

async def extract_entities(self, text: str) -> dict:
    """Extract medico-legal entities using Gemini with structured output."""

    prompt = f"""
    Act as a Peruvian forensic expert from IMLCF. Analyze this autopsy text and extract structured information.

    DICTATION TEXT:
    "{text}"

    INSTRUCTIONS:
    1. Extract "entities": list of objects with "text" and "type" 
       (ORGAN, WEIGHT, MEASUREMENT, LESION_TYPE, CONDITION, PERSON, AGE, SEX)
    2. Extract "mapped_fields": dictionary with field paths and values

    FIELD STRUCTURE (use exact paths):
    - "datos_generales.fallecido.nombre": deceased name
    - "datos_generales.fallecido.edad": age (number)
    - "datos_generales.fallecido.sexo": "M" or "F"
    - "examen_interno_torax.corazon.peso": weight in grams (number)
    - "examen_interno_torax.corazon.descripcion": description
    - "causas_muerte.diagnostico_presuntivo.causa_final.texto": final cause

    EXAMPLE response:
    {{
      "entities": [
        {{"text": "Juan Rodríguez", "type": "PERSON"}},
        {{"text": "23 años", "type": "AGE"}}
      ],
      "mapped_fields": {{
        "datos_generales.fallecido.nombre": "Juan",
        "datos_generales.fallecido.edad": 23,
        "examen_interno_torax.corazon.peso": 320
      }}
    }}

    Respond ONLY with valid JSON, no markdown.
    """

    response = self.model.generate_content(prompt)
    clean_text = response.text.replace("```

json", "").replace("

```", "").strip()
    return json.loads(clean_text)

The 70% Result

In pilot testing with a medical professional:

Metric	Manual	With CoronerIA	Improvement
Time per case	~45 min	~13 min	-71%
Typos/errors	Variable	Near-zero	✓
Field completeness	70-80%	95%+	✓

Challenge 3: Interactive SVG Anatomical Model

The Problem

Text-only documentation is error-prone. We needed visual feedback showing where on the body each finding was detected.

The Solution: Real-Time Organ Detection

// frontend/src/pages/Dictation.tsx

// Detect organs mentioned in transcription
const detectedOrgans = useMemo(() => {
    if (!transcript) return []
    const text = transcript.toLowerCase()
    const organs: string[] = []

    if (text.includes('encéfalo') || text.includes('cerebro')) 
        organs.push('encefalo')
    if (text.includes('pulmón derecho') || text.includes('pulmon derecho')) 
        organs.push('pulmon_derecho')
    if (text.includes('corazón') || text.includes('corazon')) 
        organs.push('corazon')
    if (text.includes('hígado') || text.includes('higado')) 
        organs.push('higado')
    if (text.includes('bazo')) 
        organs.push('bazo')

    return organs
}, [transcript])

SVG Highlighting with CSS Variables

// frontend/src/components/AnatomyModel.tsx

const getOrganStyle = (organ: string): React.CSSProperties => {
    const highlighted = highlightedOrgans.includes(organ)
    const hovered = hoveredOrgan === organ

    return {
        fill: highlighted
            ? 'var(--organ-highlighted)'  // Red glow
            : hovered
                ? 'var(--organ-hover)'    // Light blue
                : 'var(--organ-normal)',  // Gray
        stroke: highlighted ? 'var(--accent-danger)' : 'var(--border-secondary)',
        strokeWidth: highlighted ? 2 : 1,
        opacity: highlighted ? 1 : 0.7,
        cursor: 'pointer',
        transition: 'all 0.2s ease',
    }
}

Audio Processing Pipeline

AI Provider Fallback Flow

Lessons Learned

1. Build AI-Agnostic from Day One

Don't hard-code your AI provider. We designed for Azure but developed with Gemini (free). Switching is one config change:

# Current: Gemini (free for development)
GEMINI_API_KEY=AIza...

# Future: Azure (production)
# AZURE_SPEECH_KEY=xxx
# AZURE_OPENAI_KEY=xxx
# AZURE_OPENAI_ENDPOINT=https://xxx.openai.azure.com/

2. Supported Providers (Tested)

Provider	Speech-to-Text	NER/Extraction	Status
Google Gemini	✅ Gemini 2.0 Flash	✅ Gemini 2.0 Flash	Currently using
Azure AI	✅ Azure Speech	✅ Azure OpenAI (GPT-4)	Ready for production
OpenAI	✅ Whisper API	✅ GPT-4o	Compatible
Local	✅ faster-whisper	✅ Regex fallback	Offline mode

3. Start Offline-First

It's infinitely easier to add cloud features to an offline-capable app than to retrofit offline support to a cloud-first app.

4. Validate with Real Users Early

The 70% time reduction came from a real pilot test with a medical professional, not assumptions. This number is defensible in any interview.

Tech Stack Summary

Layer	Technology	LOC
Backend	Python, FastAPI, SQLite	2,240
Frontend	React, TypeScript, Zustand	4,191
AI	Gemini 2.0, Azure Speech, Whisper	-
DevOps	Docker, docker-compose	-
Total		~6,400

What's Next?

CoronerIA was submitted to Microsoft Imagine Cup 2026. Whether we advance or not, the project will be open-sourced to help forensic teams globally.

If you're building medical software with AI, I'd love to connect.

GitHub: CoronerIA Repository

Tags: #ai #python #react #fastapi #opensource #healthtech #microsoftimagecup