DEV Community

Le Huy Hiep
Le Huy Hiep

Posted on

How to add UI for your RAG?!?

Just built some FastAPI SSE backend streaming LLM responses token-by-token,
after i got some free time.

Two things broke in production that worked fine locally ๐Ÿ™ƒ:

  • nginx buffers SSE by default. The stream was delivering locally, dead
    silent on Railway. proxy_buffering off, two hours later, fixed.

  • The LLM API returns an empty_retrieval event when no docs match. I wasn't
    handling it, so the frontend sat on a loading spinner indefinitely. Added
    the handler, resolved.

https://zooming-charm-production-547e.up.railway.app/

Top comments (0)