The latest articles on DEV Community by Derin Karakoç (@kahveciderin). https://dev.to/kahveciderin https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4102183%2Fe3a97dbf-bf28-4fea-981f-a5de086b40fb.png https://dev.to/kahveciderin en Derin Karakoç Mon, 31 Aug 2026 06:18:50 +0000 https://dev.to/kahveciderin/the-six-problems-between-a-chat-demo-and-a-multi-tenant-agent-1aoe https://dev.to/kahveciderin/the-six-problems-between-a-chat-demo-and-a-multi-tenant-agent-1aoe <p>A chat demo is a weekend. Putting that demo in front of paying customers is usually a quarter, because the model was never the hard part.</p> <p>I kept hitting the same six problems while wiring agents into other people's SaaS products. Isolation, citations, permissions, a real computer, work that outlives the request, and a cost number you can actually bill back. Each one is "just engineering" until it isn't.</p> <p>This is the shape that held.</p> <h2> 1. Keeping customers apart </h2> <p>If tenant id comes from the request body, someone will change it. If it lives in a metadata filter on a shared vector index, one forgotten <code>filter=</code> leaks another customer's neighbors. Namespaces help. They still don't cover the cache, the logs, or the write path.</p> <p>The boundary has to be in the data plane: a project is a tenant, the API takes the tenant from a short-lived token, never from the body. A missing filter should fail closed, not fan out.</p> <h2> 2. Grounding the answer </h2> <p>"Here are some chunks" is not a citation. Founders of document-heavy products need document, page, and quote, or they need the model to refuse. Layout-aware parsing matters; OCR only on the pages that need it matters; a reindex when you swap embedding models matters.</p> <p>If you cannot point at the clause, do not ship the sentence.</p> <h2> 3. Deciding who may do what </h2> <p>Tool calling makes prompt-injection load-bearing. A permission system the model can talk its way past is not a permission system. Capabilities belong on the token, intersected with a project ceiling, checked again per request. A tool the model decides to call without the capability is refused before it runs.</p> <h2> 4. Letting it do real work </h2> <p>Answers are half the product. The other half is: clean this CSV, chart it, hand me a PNG. That means a Linux sandbox per conversation, files in and out, no network unless you allow it, idle sleep, and an audit trail. Rolling your own is a graveyard of leftover containers.</p> <h2> 5. Outliving the request </h2> <p>"Every Monday, summarise new filings" dies the first time you deploy on a Sunday. You want durable one-shot and cron, DST-safe, signed webhooks, retries, dead letters. The schedule is part of the agent, not a sidecar you forget.</p> <h2> 6. Knowing what it cost </h2> <p>Provider invoices arrive a month late and they are not per customer. If you cannot slice spend, latency, and traces per project and per end-user as the calls happen, you cannot bill it back and you cannot cap it.</p> <h2> What I shipped </h2> <p>I got tired of rebuilding those six, so they are the product. <a href="https://oberik.com" rel="noopener noreferrer">Oberik</a> is the agent layer you drop into a SaaS: mint a capability-scoped JWT from your backend, stream the answer into your own UI. Your models, your keys, your product. Free while we launch.</p> <p>Docs need no signup: <a href="https://oberik.com/docs" rel="noopener noreferrer">https://oberik.com/docs</a></p> <p>If you are mid-LangChain on a multi-tenant app, I would rather you try this than spend the quarter on Pinecone namespaces. Roast the landing page if it is wrong for you.</p> saas ai rag typescript