Just built some FastAPI SSE backend streaming LLM responses token-by-token,
after i got some free time.
Two things broke in production that worked fine locally ๐:
nginx buffers SSE by default. The stream was delivering locally, dead
silent on Railway.proxy_bufferingoff, two hours later, fixed.The LLM API returns an
empty_retrievalevent when no docs match. I wasn't
handling it, so the frontend sat on a loading spinner indefinitely. Added
the handler, resolved.
Top comments (0)