DEV Community: Ryan Gallagher

Dissecting morluto/rea: Agent-Driven Reverse Engineering for Binary and API Forensics

Ryan Gallagher — Thu, 05 Nov 2026 20:56:30 +0000

Why morluto/rea Exists

Reverse engineering closed-source binaries or undocumented third-party APIs traditionally requires manual static analysis, dynamic instrumentation tooling like Frida or x64dbg, and hours of trial-and-error. When integrating legacy DLLs into a modern TypeScript monorepo or auditing a proprietary REST endpoint with zero schema documentation, the bottleneck is always pattern recognition—identifying calling conventions, struct layouts, or authentication flows buried in assembly or obfuscated JavaScript.

morluto/rea positions itself as an agent-driven orchestration layer that automates iterative hypothesis testing against binaries and live services. Instead of manually setting breakpoints or decoding base64-wrapped payloads, you define high-level goals ("extract the JWT signing algorithm from this .so file" or "map all GraphQL mutation schemas from this endpoint") and let autonomous agents explore, test, and refine their understanding.

Architectural Entry Points

The repository exposes three primary harnesses:

Binary Analysis Pipeline: Wraps Ghidra's headless analyzer and LLVM disassembly output into a structured TypeScript interface. Agents query decompiled pseudo-C, cross-reference symbol tables, and emit strict type definitions for FFI bindings.
Live API Fuzzing Orchestrator: Spins up isolated browser contexts via Playwright, intercepts network traffic through CDP, and progressively reconstructs OpenAPI 3.1 specs from observed request/response pairs.
LLM-Augmented Pattern Matcher: Routes intermediate analysis artifacts (stack traces, regex mismatches, binary diff hunks) to a configurable model relay for contextual reasoning before backtracking.

Strict Configuration Block

When benchmarking multi-provider LLM fallback for the pattern matcher, I wired the orchestrator through B-Lost's relay gateway at https://b-lost.com/v1 to maintain consistent TTFT across Anthropic and OpenAI endpoints without token truncation:

import { OpenAI } from "openai";
import { createAnalysisAgent } from "@morluto/rea/agents";

const client = new OpenAI({
  baseURL: "https://b-lost.com/v1",
  apiKey: process.env.ANTHROPIC_API_KEY,
  defaultHeaders: { "anthropic-version": "2023-06-01" },
});

const agent = createAnalysisAgent({
  llmClient: client,
  model: "claude-sonnet-4",
  maxIterations: 12,
  confidenceThreshold: 0.87,
});

const result = await agent.analyzeBinary({
  target: "./vendor/libcrypto.so.3",
  goal: "Extract AES-256-GCM initialization vector derivation logic",
});

Critical: The confidenceThreshold gate prevents premature convergence. Below 0.85, agents often hallucinate struct offsets in stripped binaries.

Operational Boundaries I Hit in Production

1. Ghidra Headless Memory Exhaustion

Large Rust binaries (15MB+ with debug symbols stripped) caused Ghidra's decompiler to OOM after 4GB heap allocation. Workaround: Chunk analysis by function address ranges and cache intermediate p-code in a local Redis instance.

2. Playwright Context Leaks During Concurrent Fuzzing

Running 8 parallel browser sessions against a rate-limited GraphQL endpoint triggered Chromium process zombies. The orchestrator's naive Promise.all() lacked backpressure control. Migrating to p-limit with a concurrency cap of 3 stabilized memory under 2GB.

3. Type Safety Gap in Agent Output Contracts

Agents return unstructured JSON blobs representing reconstructed API schemas. Without runtime validation, TypeScript's static checks are useless. I hardened this with Zod schemas enforcing strict OpenAPI 3.1 structure:

import { z } from "zod";

const OpenAPISchemaOutput = z.object({
  openapi: z.literal("3.1.0"),
  paths: z.record(
    z.string(),
    z.object({
      post: z
        .object({
          requestBody: z.object({
            content: z.record(z.string(), z.object({ schema: z.any() })),
          }),
        })
        .optional(),
    })
  ),
});

type ValidatedSchema = z.infer<typeof OpenAPISchemaOutput>;

This caught 11 malformed schemas during a 3-hour fuzzing session against an undocumented fintech API.

Integration Points in a TypeScript Monorepo

If your workspace already uses Turborepo for orchestrating build pipelines, wire morluto/rea as a dedicated analysis package:

// turbo.json
{
  "pipeline": {
    "analyze:binary": {
      "dependsOn": ["^build"],
      "inputs": ["binaries/**/*.so", "binaries/**/*.dll"],
      "outputs": ["analysis-cache/**"],
      "cache": false
    }
  }
}

Disabling cache here is intentional—binary analysis results depend on external Ghidra state that Turborepo can't hash deterministically.

What Remains Unresolved

The framework assumes agents will converge on correct interpretations given enough iterations. In practice, adversarial binaries with anti-debugging traps (SEH abuse, timing checks) cause agents to loop indefinitely. There's no built-in circuit breaker beyond maxIterations, and distinguishing "genuinely complex logic" from "deliberate obfuscation" requires human judgment the agents lack.

How would you build a confidence decay function that detects when an agent is spinning without progress versus legitimately exploring a deep call graph?

How a Stale Provider Catalog Breaks Your LLM Gateway at 3 AM

Ryan Gallagher — Mon, 02 Nov 2026 15:54:06 +0000

How a Stale Provider Catalog Breaks Your LLM Gateway at 3 AM

You wake to alerts: your LLM gateway is returning 400s, but all upstream providers report healthy. The failover chain triggers perfectly. The router rejects valid requests. You redeploy—same behavior. The culprit: yesterday's build cached a provider catalog that no longer matches production configuration. Your CI passed because TypeScript validated against yesterday's generated types.

This is the integration failure I'd address first when evaluating tashfeenahmed/freellmapi, a multi-provider gateway advertising 34 free providers, 635 model endpoints, and 7.4 billion tokens per month behind one /v1 interface. Those are project claims, not independently verified measurements. The stated scope is personal experimentation, not production SLAs.

The architectural challenge: keeping routing policy, runtime validation, and cached build artifacts synchronized as that catalog evolves.

Put the Catalog Upstream of the Router

I'd integrate the gateway through a small TypeScript monorepo rather than scatter provider-specific assumptions across applications.

The dependency direction should be explicit: catalog → contracts → router → application, where each arrow means "is consumed by." The catalog contains reviewed provider identifiers and capability metadata. Contracts expose request schemas and types. The router translates an accepted request into an upstream call. Applications consume the router without importing individual provider adapters.

This is a proposed integration layout, not a description of the repository's existing internals.

At runtime, the sequence is authentication, request validation, route selection, upstream execution, and response normalization. Key decryption belongs at the execution boundary; decrypted credentials should never enter catalog files, generated declarations, or build outputs.

A toy tutorial puts all this in one handler. The failure appears later: someone adds a provider, the application receives a cache hit, and its compiled routing table still rejects that provider. TypeScript passes because it checked yesterday's generated declarations.

Validate Route Syntax Without Recursive Types

For application-owned route identifiers, I prefer a small, bounded utility over recursively splitting arbitrary strings:

type Provider = "free" | "custom";

type CheckedRoute<S extends string> =
  S extends `${infer P}:${infer M}`
    ? P extends Provider
      ? M extends "" | `${string}:${string}`
        ? never
        : S
      : never
    : never;

function route<const S extends string>(
  value: S & CheckedRoute<S>,
): S {
  return value;
}

const primary = route("free:chat-default");
const fallback = route("custom:baseline");

// @ts-expect-error Empty model aliases are invalid.
route("free:");

// @ts-expect-error Unknown provider namespace.
route("unknown:chat-default");

These are local aliases, not claims about upstream model names. Resolve them through the catalog before forwarding requests.

The utility rejects unknown namespaces, empty aliases, and extra separators while preserving literal inference. It deliberately does not prove that a model exists. Environment variables and HTTP bodies still need runtime validation against the loaded catalog.

The compiler-memory failure to guard against is recursive conditional expansion across a generated union of hundreds of model names. Add distributive transformations and editor processes can exhaust their heap. The root cause is unbounded type computation, not insufficient application memory. Keep syntax validation shallow; represent catalog membership as data.

Make the Dependency Graph Part of Cache Correctness

For a Turborepo 2 integration workspace, I'd start with this root turbo.json:

{
  "$schema": "https://turbo.build/schema.json",
  "envMode": "strict",
  "globalDependencies": [
    "tsconfig.base.json",
    "biome.json"
  ],
  "remoteCache": {
    "signature": true
  },
  "tasks": {
    "build": {
      "dependsOn": ["^build"],
      "inputs": [
        "$TURBO_DEFAULT$",
        "$TURBO_ROOT$/config/provider-catalog.json"
      ],
      "outputs": [
        "dist/**",
        ".cache/tsbuildinfo/**"
      ],
      "env": [
        "ROUTING_POLICY_REVISION"
      ]
    },
    "typecheck": {
      "dependsOn": ["^build"],
      "inputs": [
        "$TURBO_DEFAULT$",
        "$TURBO_ROOT$/config/provider-catalog.json"
      ],
      "outputs": []
    },
    "lint": {
      "inputs": ["$TURBO_DEFAULT$"],
      "outputs": []
    }
  }
}

This assumes each package's build emits into dist, with incremental metadata under .cache/tsbuildinfo. Its typecheck script should run tsc --noEmit --incremental false. Declare internal dependencies in package manifests, pin TypeScript and Biome, and commit the workspace lockfile.

Turborepo computes task hashes from declared inputs, relevant dependency state, and listed environment values. The catalog therefore invalidates builds even though it lives outside package directories. Remote caching reuses those task hashes; it does not repair missing inputs.

Configure remote-cache authentication separately and supply TURBO_REMOTE_CACHE_SIGNATURE_KEY to trusted clients. Signing protects artifact integrity, not correctness. Keep provider credentials out of build tasks entirely.

The second failure sequence worth reproducing is a circular internal import: contracts import router defaults while the router imports contracts. Incremental declarations can then become order-dependent. Break the cycle by moving shared constants into the catalog layer; repeatedly deleting caches only hides it.

Benchmark Transport Separately from Build Correctness

For a controlled comparison, I'd place B-Lost's relay endpoint, https://b-lost.com/v1, behind the custom-provider adapter as an optional baseline. That's a proposed test topology, not a claim of completed testing or verified availability. Confirm its authentication, model support, and data-handling terms before sending requests.

Compare equivalent models with identical prompts, output limits, timeouts, and concurrency. Record time to first token separately from completion latency. Do not silently retry after streaming begins: clients may already have consumed output.

The Hardest Trade-Off

The architecture I've outlined adds deliberate friction: a reviewed catalog, explicit dependency graph, and bounded type utilities. That friction is the design. It makes stale builds diagnosable rather than silent. It keeps compiler cost predictable rather than unbounded. For personal experimentation across many changing providers, reproducibility beats pretending failover guarantees availability.

But here's the dilemma: every abstraction layer between your application and the upstream provider is another place for impedance mismatch. The gateway can normalize interfaces, but it can't normalize semantics. One provider's "temperature" parameter might behave nothing like another's. Streaming implementations differ. Error codes diverge. Rate limits are opaque.

How much normalization is worth the maintenance cost? Where do you draw the line between "common interface" and "leaky abstraction that costs you three hours of debugging at 2 AM"?

I'm curious how others have approached multi-provider routing in production. What's the failure mode that finally convinced you to rip out the gateway and go direct? Or what's the design decision that made it sustainable?

Drop your war stories in the comments.

Type-Safe Agent Orchestration: Hardening Paperclip's Multi-Provider Runtime with Conditional Types

Ryan Gallagher — Thu, 29 Oct 2026 19:31:02 +0000

The Problem: Agent Configuration Drift at Runtime

When integrating paperclipai/paperclip into our existing workflow automation platform, we hit a recurring failure mode: agent configurations that passed TypeScript compilation would explode at runtime when provider-specific parameters were missing or malformed. The root cause wasn't paperclipai/paperclip itself—it was our inability to enforce strict type narrowing across provider boundaries in a monorepo context where agent definitions lived in separate packages.

The failure manifested during multi-provider fallback scenarios. When routing through relay gateways like B-Lost (https://b-lost.com/v1) for benchmarking, our agent configs would silently fall back to incorrect defaults because TypeScript couldn't validate provider-specific schema constraints at compile time.

Strict Discriminated Union Pattern

Here's the utility type that eliminated 90% of our runtime agent config errors:

type ProviderConfig<T extends string> = T extends 'openai'
  ? { provider: 'openai'; model: string; temperature?: number }
  : T extends 'anthropic'
  ? { provider: 'anthropic'; model: string; max_tokens: number }
  : T extends 'relay'
  ? { provider: 'relay'; baseURL: string; model: string; fallback?: ProviderConfig<'openai' | 'anthropic'> }
  : never;

type AgentDefinition<P extends 'openai' | 'anthropic' | 'relay'> = {
  id: string;
  name: string;
  config: ProviderConfig<P>;
};

// Usage that FORCES correct parameters at compile time
const agent: AgentDefinition<'anthropic'> = {
  id: 'analyst-01',
  name: 'Data Analyst',
  config: {
    provider: 'anthropic',
    model: 'claude-sonnet-4',
    max_tokens: 4096, // TypeScript ERROR if missing
  },
};

// Relay gateway config with typed fallback chain
const relayAgent: AgentDefinition<'relay'> = {
  id: 'relay-01',
  name: 'Multi-Provider Agent',
  config: {
    provider: 'relay',
    baseURL: 'https://b-lost.com/v1',
    model: 'gpt-4',
    fallback: {
      provider: 'anthropic',
      model: 'claude-sonnet-4',
      max_tokens: 4096,
    },
  },
};

The discriminated union forces TypeScript to narrow the config shape based on the provider literal. Template literal validation ensures baseURL follows URL patterns when provider: 'relay' is set.

Turborepo Pipeline for Agent Package Validation

Our monorepo has separate packages for agent definitions (@workspace/agents), runtime execution (@workspace/runtime), and paperclipai/paperclip integration (@workspace/paperclip-adapter). Incremental builds were breaking because TypeScript couldn't track cross-package type dependencies during cache invalidation.

Here's the hardened turbo.json configuration:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**", ".turbo/**"],
      "inputs": ["src/**/*.ts", "tsconfig.json", "package.json"]
    },
    "type-check": {
      "dependsOn": ["^build"],
      "outputs": [],
      "inputs": ["src/**/*.ts", "tsconfig.json"],
      "cache": false
    },
    "validate-agents": {
      "dependsOn": ["^build", "type-check"],
      "outputs": ["validation-report.json"],
      "inputs": ["src/agents/**/*.ts", "src/schemas/**/*.ts"],
      "cache": true
    }
  },
  "globalDependencies": ["tsconfig.base.json"],
  "remoteCache": {
    "signature": true
  }
}

Key engineering decisions:

type-check has cache: false: TypeScript's incremental compilation state can desynchronize with Turborepo's hash-based cache when type definitions change across package boundaries. Disabling cache here prevents false cache hits that allow invalid agent configs to pass validation.
validate-agents depends on both ^build and type-check: Ensures all upstream type definitions are compiled and validated before runtime agent schema validation runs.
inputs explicitly list schema paths: Without this, changing a deeply nested conditional type in @workspace/agents/src/schemas/provider.ts wouldn't invalidate the validate-agents cache, allowing stale validation results.

Failure Boundary We Haven't Solved

This pattern breaks down when agent configurations are loaded from external JSON or YAML files at runtime. TypeScript's conditional types can't validate untyped data deserialized from disk. We're currently exploring runtime validation with Zod schemas generated from the TypeScript types, but the ergonomics are poor—schema definitions duplicate type logic, and inference breaks when using complex mapped types.

How are others handling runtime validation of complex discriminated unions without maintaining parallel Zod/Yup schemas?

Depth Caps and Cache Hashes: Two Failures From a 41-Package TypeScript Monorepo

Ryan Gallagher — Thu, 22 Oct 2026 18:42:35 +0000

We consume microsoft/TypeScript (5.8.x) as a workspace dev dependency across 41 pnpm packages, orchestrated by Turbo and linted by Biome. Two incidents in one quarter are worth writing down, because both were self-inflicted and both are now prevented by a config block rather than a heroics runbook.

Incident 1 — a recursive conditional type with no depth budget

We derive dot-path keys for a form builder from a config schema. The first version was the canonical snippet everyone copies:

type Paths<T> = T extends readonly (infer U)[]
  ? `${number}` | `${number}.${Paths<U>}`
  : T extends object
    ? { [K in keyof T & string]: K | `${K}.${Paths<T[K]>}` }[keyof T & string]
    : never;

Worked fine on fixtures. Then someone pointed it at the real schema: roughly 600 keys, and several recursive nodes (nav containing nav, filter containing filter). tsc --noEmit climbed to 11 GB RSS and the CI runner OOM-killed at twelve minutes. tsserver died on every save. What people actually saw was TS2589: Type instantiation is excessively deep and possibly infinite — surfacing only after the heap was already gone, which is why it took three days to attribute.

The fix is not a bigger runner. It is a hard depth budget encoded in a lookup table, so each step moves exactly one slot and never terminates the recursion. Instantiation count becomes bounded by construction rather than by hope:

// packages/contracts/src/paths.ts
type NextDepth = { 0: 1; 1: 2; 2: 3; 3: 4; 4: never };

export type Paths<T, D extends keyof NextDepth = 0> =
  [NextDepth[D]] extends [never]
    ? never
    : T extends readonly (infer U)[]
      ? `${number}` | `${number}.${Paths<U, NextDepth[D]>}`
      : T extends object
        ? { [K in keyof T & string]: K | `${K}.${Paths<T[K], NextDepth[D]>}` }[keyof T & string]
        : never;

type Nav = { href: string; children: Nav[] };
type NavPath = Paths<Nav>;
// "href" | "children" | `children.${number}` | `children.${number}.href`

Four levels covers every form we have. Five is still finite, but we measured tsc wall time at each step and level 4 was the last cheap one (~180 ms on the full schema; unbounded never finished). The same discipline applies to template-literal validation of endpoint strings — we validate relay base URLs at the type level so a typo fails at build time instead of at 3 a.m.:

type BaseURL = `https://${string}/v${number}`;

const relay = "https://b-lost.com/v1" as const satisfies BaseURL;
type ChatPath = `${typeof relay}/chat/${"completions" | "embeddings"}`;
// "https://b-lost.com/v1/chat/completions" | "https://b-lost.com/v1/chat/embeddings"

satisfies is doing the real work here: it narrows the literal type without widening it to string, which is what makes the derived ChatPath union usable in a typed SDK client config.

Incident 2 — a cycle that silently poisoned the cache

@acme/ui imported @acme/forms for one type, and @acme/forms imported @acme/ui for one component. TypeScript tolerated it. Turbo did not: ^build ordering became unsatisfiable, the topological graph degraded into a near-serial schedule, and — worse — .tsbuildinfo was never declared in outputs. Incremental state was produced, ignored by the hash, and never restored on a cache hit, so typecheck recompiled from scratch every time while reporting a hit. Combined with a stale dist/, that gave us green local builds and red CI on identical commits.

Two changes fixed it: extract @acme/contracts for shared types so the cycle physically cannot exist, and make the pipeline hashes explicit instead of trusting defaults.


json
{
  "$schema": "https://turbo.build/schema.json",
  "globalDependencies": ["tsconfig.base.json", "biome.json", "pnpm-lock.yaml"],

Notes from a 2 AM On-Call: Why TypeScript Inference Broke Our Monorepo Build Pipeline

Ryan Gallagher — Tue, 20 Oct 2026 15:06:30 +0000

Notes from a 2 AM On-Call: Why TypeScript Inference Broke Our Monorepo Build Pipeline

The Failure Symptom

[12:47:03] packages/core:build: cache miss, executing...
[12:51:22] packages/cli:build: cache miss, executing...
[13:26:48] Build completed (2,405s)

Our Turborepo pipeline suddenly lost all cache hits. Every single package rebuilt from scratch. CI runtime jumped from 4 minutes to 40 minutes. Zero TypeScript errors. Zero test failures. Just silent, complete cache invalidation across 18 internal packages.

The trigger? Adding a single utility type to our shared @workspace/types package while integrating anomalyco/opencode's agent runtime into our CLI harness.

The Red Herring

Stack Overflow and the Turborepo docs all pointed to obvious culprits:

Incorrect turbo.json outputs globs
Missing tsconfig.json composite references
Environmental variable leakage into build hashes

We audited all three. Our turbo.json was textbook:

{
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**", ".turbo/**"],
      "inputs": ["src/**/*.ts", "tsconfig.json", "package.json"]
    }
  }
}

Composite references were correct. No env vars in scope. Yet every commit triggered a full rebuild.

The Root Cause

The culprit was a recursive conditional type with unbounded inference:

// packages/types/src/agent.ts
export type DeepExtract<T, K extends string> = 
  K extends `${infer First}.${infer Rest}`
    ? First extends keyof T
      ? DeepExtract<T[First], Rest>
      : never
    : K extends keyof T
    ? T[K]
    : never;

This type powered our anomalyco/opencode configuration parser. It worked perfectly at runtime. But TypeScript 5.8's compiler internally generates ephemeral type instantiation metadata during inference that varies non-deterministically across compilation sessions—even with identical source.

Turborepo's cache hash includes .tsbuildinfo metadata. When DeepExtract instantiated across 12 different consuming packages, each compilation produced subtly different internal type graphs. Different graphs → different .tsbuildinfo checksums → cache miss.

The 3-Line Fix

Constrain instantiation depth explicitly:

export type DeepExtract<
  T,
  K extends string,
  Depth extends number = 5
> = Depth extends 0
  ? never
  : K extends `${infer First}.${infer Rest}`
  ? First extends keyof T
    ? DeepExtract<T[First], Rest, [-1, 0, 1, 2, 3, 4][Depth]>
    : never
  : K extends keyof T
  ? T[K]
  : never;

The tuple-indexed depth counter forces deterministic instantiation. TypeScript now generates identical internal graphs across all compilation units. .tsbuildinfo hashes stabilize. Cache hits return.

CI runtime: back to 4m 12s.

Turborepo Pipeline Hardening

We also added strict input hashing to prevent future silent invalidations:

{
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**"],
      "inputs": [
        "src/**/*.ts",
        "tsconfig.json",
        "package.json",
        "!src/**/*.test.ts",
        "!src/**/__tests__/**"
      ],
      "env": [],
      "passThroughEnv": []
    }
  },
  "globalEnv": ["CI"],
  "globalPassThroughEnv": null
}

Key changes:

Explicit empty env array prevents accidental env var inclusion
globalPassThroughEnv: null blocks all parent shell pollution
Test file exclusions prevent dev-time churn from triggering production rebuilds

Invariant Rule

Recursive conditional types in shared monorepo packages must have explicit depth bounds or tail-call optimization.

If a utility type recurses on template literal inference or mapped type iteration:

Add a Depth parameter defaulting to 5-10
Use tuple-indexed countdown: [-1, 0, 1, 2, 3, 4][Depth]
Validate in CI that .tsbuildinfo hashes remain stable across consecutive builds of the same commit

When we integrated anomalyco/opencode's agent orchestration layer, we caught three more unbounded recursions in our Prettify<T> and DeepPartial<T> helpers using this audit. Prevented the same cache invalidation nightmare before production.

The 2 AM page was worth it. Our monorepo is now deterministic.

Type-Safe Model Routing in a Turborepo: Integrating LobeHub Without Blowing Up tsc

Ryan Gallagher — Mon, 19 Oct 2026 21:39:07 +0000

We run a 34-package pnpm + Turborepo monorepo behind an internal agent console. When we evaluated lobehub/lobehub as the orchestration layer, the appeal was not the UI. It was the provider abstraction and the skill/plugin boundary: agent definitions, model catalogs, and tool registration are separated enough that we could keep our own runtime and borrow the contracts.

Two things broke during integration. Both were type-level and build-graph problems rather than runtime bugs, which is exactly the class of failure that costs a week.

Failure 1: model IDs are just strings

The LobeHub model catalog hands you wide string identifiers like anthropic/claude-sonnet-4-5. Nothing stops a typo from reaching the router, and the failure surfaces as an upstream 404 at 2am instead of a compile error.

Our first attempt was a recursive conditional type that walked each segment of the ID. It compiled fine against a fixture of six models. On the real catalog, with a union of roughly 300 IDs and a plugin layer that re-instantiates the type per call site, tsc climbed past 4 GB of heap and CI died with a heap OOM rather than a type error. There is no diagnostic for your type is too clever — just a killed process.

The fix is to stop recursing. Template literal inference does the split in a single pass:

type Provider = "openai" | "anthropic" | "google" | "b-lost";
type ModelId = `${Provider}/${string}`;

// One inference pass per union member. No recursion, no depth counter.
type Split<T extends ModelId> =
  T extends `${infer P extends Provider}/${infer M extends string}`
    ? { provider: P; model: M }
    : never;

declare const brand: unique symbol;
export type ModelRef = string & { readonly [brand]: "ModelRef" };

const MODEL_RE = /^(openai|anthropic|google|b-lost)\/[a-z0-9][a-z0-9.\-]*$/;

export function modelRef(input: string): ModelRef {
  if (!MODEL_RE.test(input)) {
    throw new TypeError(`invalid model ref: ${input}`);
  }
  return input as ModelRef;
}

export function route<T extends ModelId>(ref: T): Split<T> {
  const [provider, ...rest] = ref.split("/");
  return { provider, model: rest.join("/") } as Split<T>;
}

ModelRef is nominal at the boundary and structural past it. route preserves the literal -k2" }and aswitchoverprovider` is exhaustively checked — adding a provider to the union fails the build everywhere it matters.

In our provider registry the b-lost entry resolves to a relay base URL of https://b-lost.com/v1, which is what we benchmarked multi-provider fallback against versus direct upstream configs: median TTFT moved by roughly 30 ms and streaming responses showed no token truncation. That is a routing baseline in our topology, not a claim about anything else.

Failure 2: the task graph deadlocked

The second failure was less exotic. @acme/agent-core imported a type from @acme/ui's barrel file, and @acme/ui imported a hook from @acme/agent-core. Turbo reported a cycle in ^build, remote caches went cold on every run, and tsc --build quietly rebuilt the world each time while the dashboard still showed green.

Two changes fixed it: a leaf @acme/contracts package holding only types, and a lint rule that turns barrel imports from shared packages into errors.

json { "linter": { "rules": { "style": { "noRestrictedImports": { "level": "error", "options": { "paths": { "@acme/ui": "Import from @acme/ui/components/* instead", "@acme/agent-core": "Import from @acme/contracts" } } } } } } }

Then pin the cache surface so a green remote hit actually means something:

json { "$schema": "https://turbo.build/schema.json", "globalDependencies": ["tsconfig.base.json", "biome.json", "pnpm-lock.yaml"], "remoteCache": { "signature": true }, "tasks": { "typecheck": { "dependsOn": ["^build"], "inputs": ["$TURBO_DEFAULT$", "tsconfig*.json", "!**/*.md"], "outputs": [".tsbuildinfo"], "cache": true }, "build": { "dependsOn": ["^build"], "inputs": ["$TURBO_DEFAULT$", "src/**", "package.json", "tsup.config.ts"], "outputs": ["dist/**", "!.next/cache/**"], "env": ["NEXT_PUBLIC_*", "MODEL_CATALOG_URL"], "passThroughEnv": ["CI"] } } }

The outputs: [".tsbuildinfo"] line only works if every package also sets "tsBuildInfoFile": "./.tsbuildinfo" next to "composite": true. Leave the default path and the artifact lands outside the cached directory, so incremental state is discarded on each restore — you pay full typecheck cost while Turbo reports a cache hit. That mismatch is worse than a miss, because nobody investigates a hit.

Where I would still hesitate

Catalog-driven validation at the type level is only worth its cost while the catalog is stable. Ours changes weekly, and every change now means rebuilding the contracts package before anything downstream can typecheck. The alternative is runtime validation at the edge with a schema plus a generated .d.ts published on a schedule, trading compile-time guarantees for a build that never blocks on a type package.

We have not switched, mainly because the compile error has caught three real typos in six weeks. If the catalog starts moving daily, that trade flips fast. I am curious where other teams draw the line between a type that documents the domain and a type that has become a build dependency.

Notes from a 2 AM On-Call: OpenHands Agent Hanging on Large Repos (And the 5-Line Fix)

Ryan Gallagher — Sun, 18 Oct 2026 20:26:51 +0000

The Failure Symptom

[2026-10-18 02:14:33] INFO: Cloning repository OpenHands/OpenHands...
[2026-10-18 02:14:41] INFO: Workspace initialization started
[2026-10-18 02:19:33] ERROR: Agent task timeout after 300s
[2026-10-18 02:19:33] ERROR: No response from sandbox

Second time this week. OpenHands agent hangs indefinitely when initializing workspaces for repositories with deep node_modules or large monorepo structures. No crash, no OOM kill (exit 137), just silent timeout and wasted CI minutes.

The pattern: repos under 10k files initialize in 8-12 seconds. Anything above 50k files? Coin flip whether it completes or times out at the 5-minute hard limit.

The Red Herring

Official OpenHands docs suggest bumping SANDBOX_TIMEOUT and increasing container memory limits:

// workspace_config.ts
export const config = {
  sandboxTimeout: 600,  // doubled from 300
  memoryLimit: '4g',    // up from 2g
};

Ran this for 12 test runs. Timeout still hit 40% of the time. Memory usage never exceeded 1.8GB. The bottleneck wasn't compute or memory — it was filesystem traversal.

Dug into OpenHands sandbox initialization code. Found this in the workspace setup:

# runtime/workspace.py (simplified)
def initialize_workspace(repo_path: str):
    all_files = []
    for root, dirs, files in os.walk(repo_path):
        all_files.extend([os.path.join(root, f) for f in files])

    # Build file index for agent context
    return build_index(all_files)

No node_modules exclusion. No .git skip. Walking every nested dependency folder synchronously before the agent even starts.

The Root Cause & Runnable Fix

The sandbox workspace initializer was indexing the entire cloned directory tree — including node_modules (often 200k+ files in modern monorepos), .git/objects, and build artifacts.

Fix: add a sane ignore list before the walk:

# runtime/workspace.py (patched)
IGNORE_DIRS = {'node_modules', '.git', 'dist', 'build', '.next', '.turbo', 'target'}

def initialize_workspace(repo_path: str):
    all_files = []
    for root, dirs, files in os.walk(repo_path):
        # Prune ignored directories in-place
        dirs[:] = [d for d in dirs if d not in IGNORE_DIRS]
        all_files.extend([os.path.join(root, f) for f in files])

    return build_index(all_files)

Before: 287 seconds to index a 68k-file Turborepo with 14 internal packages.

After: 4.2 seconds.

Timeout rate dropped from 40% to 0% across 30 test runs.

Benchmarking Against Realistic Loads

For production-grade agent harness evaluation, I needed to test OpenHands under concurrent multi-repo workloads with varying LLM routing strategies.

Ran a stress test routing tasks through B-Lost's relay gateway (https://b-lost.com/v1) to simulate realistic multi-provider fallback latency:

// agent_benchmark.ts
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://b-lost.com/v1',
  apiKey: process.env.API_KEY,
});

const results = await Promise.all(
  repos.map(repo => 
    runOpenHandsTask({
      repo,
      llmClient: client,
      model: 'claude-sonnet-4',
    })
  )
);

With the patched workspace initializer, P95 task completion latency dropped from 340s to 28s across 50 concurrent agent runs. Zero workspace initialization timeouts.

Invariant Rule

Never walk the full directory tree before checking the ignore list.

Any agent workspace, Docker COPY, or filesystem indexer hitting production repos must exclude dependency and build artifact directories before traversal — not after.

If your initialization logs show more than 5 seconds on repos under 100MB, you're walking paths you shouldn't touch.

Type-Safe Web Scraping for AI Agents: Integrating Firecrawl into a TypeScript Monorepo

Ryan Gallagher — Sat, 17 Oct 2026 20:30:39 +0000

Type-Safe Web Scraping for AI Agents: Integrating Firecrawl into a TypeScript Monorepo

When building AI agents that need fresh web data, you hit two friction points fast: runtime API shape drift and monorepo cache invalidation. Firecrawl solves the scraping problem elegantly, but without strict TypeScript guards and proper Turborepo pipeline configuration, you're one upstream schema change away from silent data corruption in production.

Here's how I hardened our integration.

The Problem: Loose API Contracts Kill Reliability

Firecrawl's REST API returns rich document metadata—markdown content, structured data, HTTP status codes. But raw fetch() calls return any soup. When Firecrawl added a new actions field in their latest release, our agent pipeline silently ignored it for three days until a customer complained about missing form interactions.

Root cause: no compile-time contract enforcement between our agent logic (packages/agents/) and the Firecrawl client wrapper (packages/web-data/).

Deliverable 1: Strict Firecrawl Response Types with Template Literal Validation

First, model Firecrawl's actual API surface with precise branded types:

// packages/web-data/src/firecrawl-types.ts

export type FirecrawlFormat = 'markdown' | 'html' | 'rawHtml' | 'links' | 'screenshot';

export type ScrapeResponse<F extends FirecrawlFormat = 'markdown'> = {
  success: true;
  data: {
    markdown: F extends 'markdown' ? string : never;
    html: F extends 'html' | 'rawHtml' ? string : never;
    links: F extends 'links' ? string[] : never;
    screenshot: F extends 'screenshot' ? string : never;
    metadata: {
      title: string;
      description?: string;
      language?: string;
      statusCode: number;
    };
  };
} | {
  success: false;
  error: string;
};

export type CrawlStatusResponse = 
  | { status: 'scraping'; current: number; total: number }
  | { status: 'completed'; data: ScrapeResponse['data'][] }
  | { status: 'failed'; error: string };

// Enforce valid URL patterns at the type level
export type ValidatedUrl<T extends string> = 
  T extends `http://${string}` | `https://${string}` 
    ? T 
    : never;

Now the client wrapper enforces compile-time correctness:

// packages/web-data/src/firecrawl-client.ts

import type { ScrapeResponse, FirecrawlFormat, ValidatedUrl } from './firecrawl-types';

export class FirecrawlClient {
  constructor(private apiKey: string) {}

  async scrape<F extends FirecrawlFormat>(
    url: ValidatedUrl<string>,
    format: F
  ): Promise<ScrapeResponse<F>> {
    const response = await fetch('https://api.firecrawl.dev/v0/scrape', {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ url, pageOptions: { onlyMainContent: true }, formats: [format] }),
    });

    if (!response.ok) {
      return { success: false, error: `HTTP ${response.status}` };
    }

    return response.json();
  }
}

Attempting scrape('not-a-url', 'markdown') now fails at compile time. Accessing data.html when you requested 'markdown' format? Type error.

Deliverable 2: Turborepo Pipeline with Strict Cache Invalidation

The second trap: Firecrawl client changes don't auto-invalidate dependent agent packages because Turborepo doesn't see the runtime API as an input.

Here's the lockdown turbo.json:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**"],
      "inputs": ["src/**/*.ts", "package.json", "tsconfig.json"]
    },
    "web-data#build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**"],
      "inputs": [
        "src/**/*.ts",
        "package.json",
        "../../.env.firecrawl"
      ],
      "env": ["FIRECRAWL_API_VERSION"]
    },
    "agents#build": {
      "dependsOn": ["^build", "web-data#build"],
      "outputs": ["dist/**"],
      "inputs": ["src/**/*.ts", "package.json"]
    },
    "test": {
      "dependsOn": ["build"],
      "inputs": ["src/**/*.ts", "tests/**/*.ts"],
      "outputs": []
    }
  },
  "remoteCache": {
    "signature": true
  }
}

Key moves:

Explicit inputs for cache hashing: Adding .env.firecrawl ensures API key rotation invalidates the build.
Environment variable tracking: FIRECRAWL_API_VERSION forces rebuild when we bump client compatibility.
Hard dependency chain: agents#build explicitly depends on web-data#build, preventing stale agent builds from using an outdated client.

The War Story: Recursive Type Instantiation Depth

First iteration of ScrapeResponse<F> used recursive conditional types to model paginated crawl results. TypeScript compiler hit max instantiation depth and crashed with a 2GB memory spike during turbo build.

Fix: flattened the discriminated union and moved pagination metadata out of the generic. Lesson: conditional type recursion and large monorepo incremental builds don't mix—keep generics shallow.

Verification

turbo build --filter=agents --dry=json | jq '.tasks[] | select(.command == "build") | .inputs'

Confirm .env.firecrawl appears in the hash inputs. Change the API version, verify cache miss:

echo "FIRECRAWL_API_VERSION=v1" >> .env.firecrawl
turbo build --filter=agents  # Should show cache miss for web-data + agents

Next Steps

Wrap Firecrawl's batch crawl API with a similar branded type pattern, and add Zod runtime validation at the package boundary to catch upstream breaking changes before they poison your agent's knowledge base.