Building Ask Leya: Lessons Learned from Local ONNX Embeddings, Prisma Migrations, and Express Async Handlers

Why build another chat widget? Because every widget I tried either followed a rigid script or confidently made things up. Ask Leya answers from a business's own content and hands off to a human when it can't.

Here are three core architectural hurdles I ran into and how I solved them:

1. Cross-language retrieval is local, not the LLM Retrieval uses paraphrase-multilingual-MiniLM-L12-v2 through ONNX, in-process — 384 dims, trained across 50+ languages. A business writes its knowledge base in English; an Arabic visitor asks in Arabic; the local model embeds that question into the same vector space as the English entries and finds the right one.

No translation step, no API call, no per-query embedding cost, and nothing leaves the server to be embedded. The chat model then only has to phrase the answer back in Arabic, which is the easy half.

The trade: A cold start while the model loads, and it pulls in sharp, which cost me a 90-second production hang when a macOS binary got rsynced onto a Linux box.

1. Managing a database shared with unreleased modules
   The database is shared with the rest of my broader app under development — 67 tables total right now, many of which belong to unreleased modules. Since Prisma migrate tries to treat unreleased tables as drift, every migration currently requires hand-filtering additive statements via prisma migrate diff until the full suite ships.

2. Express 4 doesn't forward async rejections
   An unguarded async handler that throws gives you no response at all — the caller just hangs until it times out. That took /chat down for 90 seconds a request before I tracked it down.

The Stack
Frontend: Next.js App Router, React (with a dependency-free vanilla JS iframe widget so it can't collide with customer CSS).

Backend: Node.js, Express.

Database: Postgres (Neon) + pgvector.

Real-time & AI: Socket.IO for live human handoff, gpt-4o-mini via OpenRouter for replies.

Free tier, no credit card required. Features like WhatsApp, Instagram, and Messenger integrations are built out but kept behind the scenes until fully polished for self-serve.