The latest articles on DEV Community by Namchinbayar Naran (@bayar). https://dev.to/bayar https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4118728%2F831c3ad9-b35e-405b-8403-d949e09ba19a.png https://dev.to/bayar en Namchinbayar Naran Thu, 10 Sep 2026 07:36:51 +0000 https://dev.to/bayar/building-bayardev-streaming-ai-responses-and-designing-for-zero-retention-boundaries-507d https://dev.to/bayar/building-bayardev-streaming-ai-responses-and-designing-for-zero-retention-boundaries-507d <p>Hey DEV Community!</p> <p>I’m building <a href="https://bayar.dev" rel="noopener noreferrer">bayar.dev</a>, an early-stage software product company developing AI infrastructure, B2B workflow products, and multi-agent systems.</p> <p>The foundation starts with a simple goal: make AI applications feel immediate while keeping their data boundaries explicit.</p> <h2> What’s live today </h2> <p>The public chat endpoint at <code>/api/chat</code> is stateless and streams responses token by token using Server-Sent Events (SSE). The backend connects to Azure AI infrastructure through an OpenAI-compatible interface.</p> <p>That architecture keeps the request path small:</p> <ol> <li>The client submits a message.</li> <li>The API validates and forwards the request.</li> <li>The model begins generating.</li> <li>Tokens stream back as they become available.</li> </ol> <p>The user sees progress immediately instead of waiting for the entire response to finish.</p> <h2> What I’m designing toward </h2> <p>The broader infrastructure is evolving around several priorities:</p> <ul> <li> <strong>Edge-aware routing:</strong> directing inference requests based on latency, availability, and model capability.</li> <li> <strong>Semantic caching:</strong> reusing suitable results without weakening privacy boundaries.</li> <li> <strong>PII controls:</strong> detecting and redacting sensitive data before it crosses an external model boundary.</li> <li> <strong>Private deployment:</strong> supporting enterprise environments that require private networking, customer-controlled keys, or stricter data residency.</li> <li> <strong>Provider portability:</strong> keeping the application layer independent from any single model vendor.</li> </ul> <p>I’m treating privacy as an architectural constraint—not a policy added after the system is already built. The long-term objective is a clear zero-retention boundary where every component has an explicit reason to receive, process, or retain data.</p> <p>There is still plenty to solve, especially around cache invalidation, regional failover, observability, and the tradeoff between lower latency and stronger isolation. That is what makes this layer interesting.</p> <p>If you’re building streaming AI applications, inference gateways, semantic caches, or privacy-sensitive agent systems, I’d love to compare notes.</p> <p><strong>What does your current inference stack look like, and where are you encountering the hardest latency or data-boundary problems?</strong></p> ai architecture webdev azure