Building bayar.dev: Streaming AI Responses and Designing for Zero-Retention Boundaries

Hey DEV Community!

I’m building bayar.dev, an early-stage software product company developing AI infrastructure, B2B workflow products, and multi-agent systems.

The foundation starts with a simple goal: make AI applications feel immediate while keeping their data boundaries explicit.

What’s live today

The public chat endpoint at /api/chat is stateless and streams responses token by token using Server-Sent Events (SSE). The backend connects to Azure AI infrastructure through an OpenAI-compatible interface.

That architecture keeps the request path small:

1. The client submits a message.
2. The API validates and forwards the request.
3. The model begins generating.
4. Tokens stream back as they become available.

The user sees progress immediately instead of waiting for the entire response to finish.

What I’m designing toward

The broader infrastructure is evolving around several priorities:

• Edge-aware routing: directing inference requests based on latency, availability, and model capability.
• Semantic caching: reusing suitable results without weakening privacy boundaries.
• PII controls: detecting and redacting sensitive data before it crosses an external model boundary.
• Private deployment: supporting enterprise environments that require private networking, customer-controlled keys, or stricter data residency.
• Provider portability: keeping the application layer independent from any single model vendor.

I’m treating privacy as an architectural constraint—not a policy added after the system is already built. The long-term objective is a clear zero-retention boundary where every component has an explicit reason to receive, process, or retain data.

There is still plenty to solve, especially around cache invalidation, regional failover, observability, and the tradeoff between lower latency and stronger isolation. That is what makes this layer interesting.

If you’re building streaming AI applications, inference gateways, semantic caches, or privacy-sensitive agent systems, I’d love to compare notes.

What does your current inference stack look like, and where are you encountering the hardest latency or data-boundary problems?