DEV Community: Heath Mcintyre

AI security monitoring at scale: one LLM call, every dashboard

Heath Mcintyre — Sat, 25 Apr 2026 18:03:29 +0000

How CoinHawk runs a continuous AI security scan for every connected user using a single shared LLM call every 5 minutes.

The dumb version doesn't scale

Imagine you want every user's dashboard to display a live "security score" produced by an LLM. The first instinct is:

GET /api/security/scan  →  call OpenAI  →  return result

If 1,000 users hit the dashboard, you make 1,000 LLM calls. At ~$0.02 per call you've burned $20 in 30 seconds, OpenAI rate-limits you, and your p95 latency is now whatever GPT feels like today.

The second instinct is to cache the response per-user. That's a little better, but you still scale costs linearly with users, and the cache invalidation logic gets ugly fast.

The real answer for monitoring-style features is dead simple: the scan is global, so make exactly one global scan and serve it to everyone. Below is the production pattern I shipped in CoinHawk's "Sentinel" feature.

The mental model

The data is shared, not user-specific. A "system security posture" doesn't care who's logged in.
Compute on a timer, not on demand. Refresh every N minutes regardless of traffic.
Reads are O(1) and instant. Every API request just returns the cached object.
One in-flight refresh at a time. No matter how many things ask, only one LLM call runs.

This pattern works for any "expensive global signal you want to display everywhere": status pages, market summaries, anomaly detectors, AI-curated newsfeeds, etc.

The full implementation (~50 lines)

import { openai } from "@workspace/integrations-openai-ai-server";

export type SecurityScanResult = {
  riskScore: number;
  status: "secure" | "watch" | "elevated" | "critical";
  summary: string;
  risks: Array<{
    severity: "low" | "medium" | "high" | "critical";
    category: string;
    title: "string;"
    description: "string;"
    recommendation: string;
  }>;
  scannedAt: string;
};

let cachedScan: SecurityScanResult = fallbackScan();
let inFlight: Promise<SecurityScanResult> | null = null;
let schedulerStarted = false;

export function getCachedScan(): SecurityScanResult {
  return cachedScan; // O(1), no I/O, every request hits this
}

export async function refreshScan(): Promise<SecurityScanResult> {
  // Coalesce concurrent refresh requests into the same in-flight call.
  if (inFlight) return inFlight;

  inFlight = (async () => {
    try {
      const result = await runSecurityScan(); // the actual OpenAI call
      cachedScan = result;
      return result;
    } catch (err) {
      logger.error({ err }, "Sentinel scan failed");
      return cachedScan; // serve stale on failure, never break the dashboard
    } finally {
      inFlight = null;
    }
  })();
  return inFlight;
}

export function startSecurityScanScheduler(intervalMs = 5 * 60 * 1000) {
  if (schedulerStarted) return;
  schedulerStarted = true;
  setTimeout(() => refreshScan().catch(() => {}), 3000); // warm-up
  setInterval(() => refreshScan().catch(() => {}), intervalMs);
}

And the route:

router.get("/security/scan", (req, res) => {
  res.json(getCachedScan());
});

That's it. 1 user or 100,000 users, the cost is the same: one LLM call every 5 minutes.

The four design decisions that matter

1. Coalesce in-flight calls

The inFlight promise is the most important line in the file. Without it, two concurrent calls to refreshScan() (e.g. scheduler + manual admin trigger) become two LLM calls. With it, the second call just awaits the first. This single guard collapses thundering-herd risk to zero.

2. Serve stale on failure

The catch block returns cachedScan instead of throwing. If OpenAI has an outage, your dashboard keeps showing the last known scan instead of a red error banner. Users stay calm. You buy yourself time to fix things.

3. Warm up before the first user hits

setTimeout(() => refreshScan(), 3000);

Three seconds after boot, kick off the first scan in the background. Otherwise the very first dashboard request after deploy waits on an OpenAI cold start (3–10s).

4. Force structured output

const completion = await openai.chat.completions.create({
  model: "gpt-5.4",
  response_format: { type: "json_object" },  // <-- strict JSON, no markdown
  messages: [
    { role: "system", content: SECURITY_SYSTEM_PROMPT },
    { role: "user", content: "Run a fresh security scan." },
  ],
});

response_format: { type: "json_object" } plus a system prompt that includes the exact schema means you can JSON.parse() without regret. Then defensively coerce every field — never trust the model's types.

The system prompt that produces useful output

The system prompt is doing real work. It defines the schema, enumerates allowed values, sets distribution expectations ("most scans should mostly be low/medium with at most 1-2 high or critical items"), and forbids prose around the JSON.

The interesting trick: include the portfolio context (holdings, win rate, exchanges connected) directly in the system prompt rather than the user message. The model produces more grounded, specific risks ("watch your AVAX exposure on bridge X") instead of generic ones ("be careful of phishing").

What about user-specific scans?

For per-user signals (e.g. "scan THIS wallet"), the same pattern applies — just key the cache:

const cache = new Map<string, { result: SecurityScanResult; ts: number }>();

export async function getScanForWallet(addr: string) {
  const hit = cache.get(addr);
  if (hit && Date.now() - hit.ts < 60_000) return hit.result;
  const result = await runSecurityScan(addr);
  cache.set(addr, { result, ts: Date.now() });
  return result;
}

But for the global header pill that says "Sentinel: secure" on every dashboard? One scan. Forever.

Try it

Watch the Sentinel pill update on the live CoinHawk dashboard:

→ https://71554e3f-e544-4c13-9297-83c480d696c1-00-3dqa8py4myltw.worf.replit.dev/

Connect a wallet, watch the header. The pill is reading directly from the cached global scan that refreshes every 5 minutes — same code as above, in production right now.

If you liked this pattern, the launch story is here: How I built an AI crypto trading dashboard in a weekend with Replit + Base.

Built with OpenAI, Express, and a stubborn refusal to scale costs linearly with users.

Verifying real wallet ownership without gas: a signed-nonce pattern with viem + Express

Heath Mcintyre — Sat, 25 Apr 2026 17:58:48 +0000

A production walkthrough of the auth pattern powering CoinHawk's admin layer — and why "the client says they're 0xABC" is a security bug.

The naive way (and why it's broken)

When a user connects MetaMask to your dapp, the browser hands you their wallet address through window.ethereum.request({ method: "eth_requestAccounts" }). Tempting flow:

Frontend asks MetaMask for the address
Frontend POSTs { address: "0xABC..." } to your backend
Backend stores it on the session
Backend uses that address to gate admin features, route fees, log trades, etc.

This is completely insecure. Anyone with curl can POST any address they want. There is zero proof the requester controls the private key for 0xABC. They could claim to be Vitalik. They could claim to be your admin wallet.

You need a proof of ownership step. The standard solution is a signed-nonce challenge — and it's surprisingly easy to get wrong if you trust the client for any of the inputs.

Here's the pattern I shipped in CoinHawk, with the real production code.

The pattern in three steps

Server issues a fresh nonce with a short TTL, stores it on the user's session
Client signs a message containing that nonce via personal_sign
Server reconstructs the exact message from server-stored values, then verifies the signature against the address

The key word is reconstructs. The client never sends the message back. Otherwise we'd be re-introducing a trust gap — a malicious client could send a different message than the one MetaMask actually signed.

Step 1 — Issue a nonce

// routes/wallet.ts
import { Router, type Request, type Response } from "express";
import { randomBytes } from "node:crypto";
import { getSession, updateSession } from "../lib/auth";

const router = Router();
const NONCE_TTL_MS = 5 * 60 * 1000; // 5 minutes

function buildSigningMessage(nonce: string, issuedAt: Date, expiresAt: Date) {
  return [
    "Welcome to CoinHawk.",
    "",
    "Sign this message to verify ownership of your wallet.",
    "This is free and will not submit a transaction.",
    "",
    `Nonce: ${nonce}`,
    `Issued: ${issuedAt.toISOString()}`,
    `Expires: ${expiresAt.toISOString()}`,
  ].join("\n");
}

router.get("/wallet/nonce", async (req: Request, res: Response) => {
  if (!req.isAuthenticated() || !req.sessionId) {
    return res.status(401).json({ error: "Authentication required" });
  }
  const session = await getSession(req.sessionId);
  if (!session) {
    return res.status(401).json({ error: "Authentication required" });
  }

  const nonce = randomBytes(16).toString("hex");
  const issuedAt = new Date();
  const expiresAt = new Date(issuedAt.getTime() + NONCE_TTL_MS);
  const message = buildSigningMessage(nonce, issuedAt, expiresAt);

  await updateSession(req.sessionId, {
    ...session,
    pendingWalletNonce: nonce,
    pendingWalletNonceExpiresAt: expiresAt.getTime(),
  });

  res.setHeader("Cache-Control", "no-store");
  res.json({ nonce, message, issuedAt, expiresAt });
});

Three things to notice:

The user must already be authenticated (logged in via your normal auth, in our case Replit OIDC). Wallet verification binds a wallet to an existing user — it's not a replacement for login.
The nonce lives on the server session, not in a cookie or localStorage. Storing it on the client would defeat the entire point.
Cache-Control: no-store because nonce responses should never be cached by intermediaries.

Step 2 — Client signs

// hooks/useWallet.ts (frontend)
async function connectWallet() {
  const [address] = await window.ethereum.request({
    method: "eth_requestAccounts",
  });
  const chainId = await window.ethereum.request({ method: "eth_chainId" });

  // Ask the server for a fresh challenge
  const { message } = await fetch("/api/wallet/nonce").then((r) => r.json());

  // Sign it via MetaMask. No gas, no transaction.
  const signature = await window.ethereum.request({
    method: "personal_sign",
    params: [message, address],
  });

  // Send the signature + claimed address back
  await fetch("/api/wallet/register", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ address, signature, chainId }),
  });
}

Note we never send the message text back. That's deliberate.

Step 3 — Server reconstructs and verifies

import { verifyMessage, isAddress } from "viem";

router.post("/wallet/register", async (req, res) => {
  if (!req.isAuthenticated() || !req.sessionId) {
    return res.status(401).json({ error: "Authentication required" });
  }

  const { address, signature, chainId } = req.body;
  const session = await getSession(req.sessionId);

  const nonce = session?.pendingWalletNonce;
  const expiresAt = session?.pendingWalletNonceExpiresAt;
  if (!nonce || !expiresAt || Date.now() > expiresAt) {
    return res.status(400).json({
      error: "No active signing challenge. Request a new nonce and try again.",
    });
  }
  if (!isAddress(address)) {
    return res.status(400).json({ error: "Invalid wallet address" });
  }

  // CRITICAL: reconstruct the exact message from SERVER-stored values.
  // Never accept the message text from the client.
  const issuedAt = new Date(expiresAt - NONCE_TTL_MS);
  const message = buildSigningMessage(nonce, issuedAt, new Date(expiresAt));

  let valid = false;
  try {
    valid = await verifyMessage({ address, message, signature });
  } catch {
    valid = false;
  }
  if (!valid) {
    return res.status(400).json({ error: "Wallet signature verification failed" });
  }

  // Bind the wallet to the session and burn the nonce so it can't be replayed.
  await updateSession(req.sessionId, {
    ...session,
    walletAddress: address.toLowerCase(),
    walletChainId: chainId,
    pendingWalletNonce: undefined,
    pendingWalletNonceExpiresAt: undefined,
  });

  res.json({ walletAddress: address.toLowerCase() });
});

viem's verifyMessage handles both EOA and ERC-1271 smart contract wallet signatures, which means smart accounts (Safe, Argent, etc.) work transparently. That's worth a lot.

The five things people get wrong

After reviewing a dozen open-source dapps, here are the bugs I see most often:

Trusting the client-supplied message. The signature is for a specific message. If the client picks the message, they pick the meaning. Always reconstruct server-side.
Storing the nonce in a cookie or localStorage. The whole point is the server holds the secret. A client-controlled nonce is no nonce at all.
No TTL. A nonce that lives forever can be replayed forever. 5 minutes is a good default.
No "burn after use". Once verified, delete the nonce. Otherwise a replay attack works the second time too.
Lowercasing inconsistently. EVM addresses are case-insensitive but checksums are case-significant. Pick one form (lowercase) and use it everywhere — comparisons, storage, env config.

Why I bothered

CoinHawk routes a 1% fee per trade on-chain to a verified admin wallet. The admin dashboard exposes server-side controls (nuke trades, surface alerts, run AI sentinel scans). If the wallet check were spoofable, anyone could promote themselves to admin by claiming to be the admin wallet's address. Game over.

With the signed-nonce pattern, the server has cryptographic proof — at the moment of binding — that the requester controls the private key for the address they're claiming. That proof gates everything downstream.

Try it

CoinHawk is live at https://71554e3f-e544-4c13-9297-83c480d696c1-00-3dqa8py4myltw.worf.replit.dev/ — connect your wallet, watch the MetaMask sign prompt, and you've just experienced the flow above.

Source patterns and the full implementation walkthrough live in the previous post: How I built an AI crypto trading dashboard in a weekend with Replit + Base.

Built with viem, Express, and a healthy paranoia about clients.

How I built an AI crypto trading dashboard in a weekend with Replit + Base

Heath Mcintyre — Sat, 25 Apr 2026 17:56:29 +0000

A practical breakdown of CoinHawk: what it does, how it's wired, and what surprised me along the way.

I've been trading crypto on the side for about three years and the same thing kills me every time: I miss the move. The signal hits at 3am, my Discord pings, I'm asleep, and by the time I wake up the trade is gone.

So last weekend I built CoinHawk — an AI-powered trading dashboard that watches the markets so I don't have to. It scans 2,400+ tokens across 14 exchanges, surfaces real-time signals, and executes trades in one click via MetaMask on Base. Free tier available, $29/mo for the full version.

Live demo → https://71554e3f-e544-4c13-9297-83c480d696c1-00-3dqa8py4myltw.worf.replit.dev/

Here's how it came together.

The stack (and why)

Replit for the dev environment, Postgres, and one-click deploy. The whole thing — frontend, backend, database, secrets, deploy — lives in one project. No DevOps required.
React + Vite for the dashboard frontend. Dark theme, accent green, Space Grotesk + DM Sans. Feels like a Bloomberg terminal got a redesign.
Express + Drizzle + Postgres for the API and trade log.
OpenAI gpt-5.4 (via Replit's AI proxy, so I never had to manage an API key) for signal generation and the security monitor.
viem for wallet signature verification.
Base as the on-chain settlement layer. 1% per trade routes to a verified admin wallet — no custodial layer, no third-party processor.

Total time from blank Replit to live production: about 11 hours over a weekend.

The three things I'm proud of

1. Real wallet ownership verification

The naive way to do "wallet-based admin" is: client says "I'm wallet 0xABC" → server trusts it. That's broken. Anyone can claim to be anyone.

CoinHawk does it properly:

Server issues a one-time nonce (5-minute TTL, stored in session)
Client signs the reconstructed message via MetaMask personal_sign
Server verifies the signature with viem's verifyMessage against the server-stored nonce, not the client-supplied one
Only then is the wallet bound to the session

No gas, no transaction, no spoofing. ~80 lines of code.

2. Server-driven AI security monitor

Every dashboard has a "Sentinel" status pill in the header. It's not a static badge — it's the output of a real LLM scan of the deployment that runs every 5 minutes server-side. Cache is shared across all users, so a single API call serves everyone. Frontend polls every 30s.

The trick: ignore user-supplied parameters when computing the cached scan. One bad-actor user shouldn't be able to steer what every other user sees.

3. Tier-aware dashboard from a single URL

Pricing tiers (Scout/Hunter/Apex) on the landing page link to /dashboard?plan=apex. The dashboard reads it via useSearchParams, persists to localStorage, then strips the query. Refresh-safe, share-safe, simple.

What surprised me

Replit's "publish" button is faster than I expected. I push a code change, click Publish, and 60 seconds later it's live on .replit.app. No Dockerfile, no CI/CD pipeline, no AWS console. For a solo project this is the right tradeoff.

OpenAI's gpt-5.4 is wildly good at structured trade analysis when you give it real market context. The hard part wasn't the model — it was building the data pipeline that feeds it.

Wallet UX is still terrible. Even with all my polish, MetaMask popping up two prompts (connect + sign) is friction. Smart accounts will eventually fix this. Until then, it's the cost of Web3.

What's next

Smart account support (gasless trades for the Hunter tier)
WalletConnect for non-MetaMask users
A mobile app via Expo
Backtest mode so users can replay AI signals against historical data

If you want to try it: https://71554e3f-e544-4c13-9297-83c480d696c1-00-3dqa8py4myltw.worf.replit.dev/ — Scout tier is free, no credit card.

Built on Replit and Base. Roast my code or my color choices in the comments.