A practical breakdown of the architecture decisions, trade-offs, and frontend/backend boundaries behind Flux — an event-driven real-time dashboard platform I built.
While building Flux, I decided to build something that felt like a real-time system.
- Not a frontend that keeps polling every few seconds.
- Not a UI that directly calls five different APIs.
- Not a project where everything works only when the happy path works.
I wanted something closer to how production systems are usually designed:
- multiple data domains
- asynchronous communication
- real-time delivery
- graceful degradation
- and a frontend that stays simple even when the backend gets more complex
So this post is a breakdown of how I designed the architecture for Flux — and more importantly, why I made the decisions I made.
What Flux actually is
Flux is a real-time dashboard that streams and displays:
- Weather
- News
- Stocks
- Crypto
At first glance, it looks like a frontend-heavy project.
But the interesting part is actually the backend architecture.
Because the real problem wasn’t:
"How do I render cards on a dashboard?"
The real problem was:
"How can I design a system that cleanly ingests, processes, and streams multiple real‑time data feeds to clients?"
That changed everything.
The first architecture I didn’t want
The most obvious way to build this would have been something like this:
Frontend
├── calls weather API
├── calls news API
├── calls stocks API
└── calls crypto API
This works for a small demo.
But I expected that approach to become painful pretty quickly.
Why I avoided that approach
Because the frontend would slowly become responsible for things it should not own:
- request orchestration
- retries
- service-specific logic
- failure handling
- data normalization
- caching decisions
- reconnect behavior
That’s how “simple dashboards” become messy.
And honestly, this was one of the main design lines I kept repeating to myself while building this:
I wanted the frontend to stay thin and focused.
Meaning:
- render data
- send user intent
- keep a real-time connection alive
That’s it.
The architecture I settled on
I ended up designing Flux around 3 main layers:
Frontend (Web UI)
│
▼
BFF (Socket.IO + Cache + Kafka coordination)
│
▼
Domain Services (Kafka-based)
This became the core mental model of the whole project.
Each layer has one job.
And once I locked that in, the rest of the system became much easier to reason about.
1) Frontend: thin, reactive, and intentionally limited
The frontend in Flux is intentionally thin and focused.
That was a design choice, not a shortcut.
Its job is to:
- open a persistent Socket.IO connection
- send user context (like location)
- subscribe to real-time updates
- render whatever arrives
Its job is not to:
- talk to Kafka
- call backend services directly
- aggregate data
- implement retry policies
- decide caching behavior
That separation made the frontend much cleaner.
Why this mattered
Because when frontend code starts knowing too much about backend infrastructure, everything becomes harder:
- harder to debug
- harder to test
- harder to scale
- harder to change later
So in Flux, the frontend only talks to one thing:
the BFF
That one decision removed a lot of future complexity.
2) Why I used a BFF instead of exposing services directly
This was probably the most important architecture decision in the whole project.
I introduced a Backend-for-Frontend (BFF) layer between the UI and the backend services.
What the BFF does
The BFF is responsible for:
- maintaining client Socket.IO connections
- receiving events from backend services
- hydrating reconnecting clients quickly
- deciding what data to fan out to which users
- acting as the real-time gateway of the system
So instead of this:
Frontend → many services
I made it:
Frontend → BFF → services
On paper that sounds small.
In practice, it changed a lot.
Why I liked this model
Because the frontend now has:
- one connection model
- one integration boundary
- one real-time contract
And the backend can evolve without breaking the UI every time.
That gave me a much better separation between:
- presentation concerns
- system concerns
Which is exactly what I wanted from the start.
3) Why I used Kafka in the middle
Once I knew I wanted multiple real-time domains, I also knew I didn’t want everything tightly coupled.
If weather updates, crypto updates, stock updates, and news updates all depend directly on each other — or on one giant central service — that becomes painful fast.
So I used Kafka as the backbone.
What Kafka gave me
Kafka helped me design the system around events, not direct service-to-service coupling.
That gave me a few nice properties:
- services can evolve independently
- producers and consumers don’t need to know too much about each other
- scaling one domain doesn’t force scaling everything
- the architecture feels much closer to real production systems
That was important to me.
Because I didn’t want Flux to be a project optimized more for presentation than for system trade-offs.
I wanted it to feel like something that was designed with actual backend trade-offs in mind.
4) Why I chose Socket.IO for real-time delivery
For the client-facing real-time layer, I chose Socket.IO.
And yes — I know raw WebSockets are often the more low-level answer on paper.
But for this project, I cared more about reliability and developer ergonomics than sounding low-level.
Why Socket.IO made sense here
It gave me:
- automatic reconnection
- fallback transport support
- room-based fan-out
- simpler event semantics
- less boilerplate for real-time client communication
That mattered because Flux is not just:
“send one stream to one client”
It’s a multi-stream dashboard with different categories of data and different update patterns.
So having a stable, practical abstraction here was worth it.
Sometimes “more production-realistic” is not about choosing the lowest-level primitive.
Sometimes it’s about choosing the thing you can operate more reliably.
5) The problem I ran into: reconnects and hydration
This is where the architecture got more interesting.
Real-time apps are not just about live streaming.
They’re also about:
“What happens when a user reconnects?”
That question forced me to think beyond just pushing events.
Because if a user refreshes the page or reconnects after a network blip, I don’t want them staring at an empty dashboard waiting for all streams to naturally update again.
That creates a poor reconnect experience.
So I split the system mentally into two kinds of data:
A) Snapshot data
Data that should be shown immediately on reconnect
Examples:
- top news
- weather snapshot
- top crypto coins
- stock summaries
B) Stream data
Data that should continue flowing live
Examples:
- ticker updates
- incremental changes
- fast-moving live events
That separation ended up being very useful.
Because it let me design hydration and streaming differently instead of pretending all real-time data behaves the same way.
And honestly, that was one of the cleanest architecture decisions in the project.
6) Why I added cache, but made it optional
Once I started thinking about reconnect hydration, cache became the obvious next step.
But I also didn’t want to build a system that completely dies if cache is unavailable.
So I used Valkey (open-source Redis fork) as an optional accelerator, not as a hard dependency.
That distinction mattered a lot.
Why “optional cache” was important
Cache is amazing for:
- fast reconnect hydration
- reducing repeated work
- serving recent snapshot data quickly
But I didn’t want Flux to become:
“works only if every dependency is healthy”
So I designed it with this mindset:
- if cache is available → great, faster experience
- if cache is unavailable → system should still keep working
That’s a small detail, but it changes how resilient the system feels.
And personally, I’ve started appreciating this design style a lot more:
Acceleration should not become fragility.
7) One of the subtle problems: selective fan-out
Once I had a BFF pushing real-time data, I ran into another question:
“Should every client receive every event?”
In practice, the answer is no.
That would:
- waste bandwidth
- add unnecessary frontend filtering
- generally be inefficient
So I used Socket.IO rooms to scope event delivery.
That meant I could think in terms like:
- weather by city
- global news stream
- crypto stream
- stock stream
This helped keep the fan-out more intentional instead of just:
“broadcast everything and let the client figure it out”
That’s one of those things that sounds small when you say it in one sentence, but it makes the architecture much cleaner.
8) One frontend decision I’m glad I didn’t stay stubborn about
Initially, I was trying to keep the frontend very clean:
- hooks handle data and subscriptions
- components just render UI
And for the most part, that worked well.
Each domain had its own hook:
- weather
- news
- stocks
- crypto
So the general rule was simple:
Hooks deal with real-time logic. Components stay simple.
That kept things pretty clean.
But then I ran into a small UX problem
In the Crypto card, I had two tabs:
- Top Movers
- Live Ticker
And every time I switched between them, I didn’t like what I was seeing.
Some data in the Crypto card would reload again, and state didn’t feel stable across tab switches.
The UI didn’t feel as smooth as I wanted.
Nothing was actually broken.
It just didn’t feel good.
And I’ve started trusting that feeling more while building projects.
Because a lot of times, architecture looks clean in code but feels annoying in the actual product.
So this is where I bent my own rule a bit
Instead of forcing everything through local hook state, I used Redux selectively for the crypto section.
Not across the whole app.
Just where it actually helped.
Mainly to keep things like:
- ticker data
- top coins data
- price-related state
stable across tab switches.
The pattern that felt right was:
- hooks handle subscriptions and incoming socket events
- Redux keeps shared UI state stable where needed
- components just read and render
That ended up feeling much better.
Why I’m glad I did this
Because this was one of those cases where:
being too “architecturally pure” would have made the user experience worse.
And honestly, I’d rather have a slightly more practical architecture than a “perfectly clean” one that adds unnecessary friction to the user experience.
That small change made the Crypto card feel way smoother.
And I think that’s a useful reminder in general:
Sometimes the right architecture decision is just the one that makes the product feel better.
9) Failure isolation was not an afterthought
One thing I really wanted in Flux was this:
If one stream fails, the dashboard should still feel alive.
So I designed the UI and backend with failure isolation in mind.
Meaning:
- if weather is delayed → crypto still updates
- if news fails → stocks still render
- if one service lags → the whole app shouldn’t feel dead
That sounds like a UX decision, but it’s actually an architecture decision too.
Because if your system shape forces everything to depend on everything else, then partial failure becomes full failure.
And I wanted to avoid that.
A real-time system should degrade, not collapse.
10) I intentionally did not chase “perfect distributed systems purity”
This was a very conscious choice.
Because once you start building event-driven systems, it’s very easy to go down the rabbit hole of:
- exactly-once everything
- over-engineered delivery guarantees
- too many abstractions too early
- adding architectural complexity that doesn’t meaningfully improve the system
I tried hard not to do that.
So Flux is opinionated in a practical way.
I optimized for:
- clarity
- resilience
- clean boundaries
- realistic trade-offs
Not unnecessary complexity.
That was important to me because I wanted this project to reflect how I actually think as an engineer:
I like systems that are thoughtful, not just complicated.
Final architecture summary
If I had to describe Flux’s architecture in one sentence, I’d say:
It’s a real-time dashboard designed like a small event-driven platform, not like a frontend project with extra backend code attached.
That’s the difference I cared about.
The main ideas behind the design were:
- keep the frontend thin
- centralize real-time delivery in a BFF
- use Kafka for loose coupling
- use Socket.IO for practical real-time delivery
- separate snapshot hydration from live streams
- use cache as an accelerator, not a crutch
- isolate failures so the whole app doesn’t feel broken
And honestly, designing those boundaries was way more interesting than building the UI itself.
What I learned from building this
If I had to compress the biggest lesson into one line:
Most architecture problems become easier once each layer has one clear responsibility.
A lot of messy systems are messy because responsibilities are blurry.
Flux became much easier to build once I stopped asking:
“How do I make everything talk to everything?”
…and started asking:
“What is each layer allowed to know?”
That single shift made the architecture cleaner.
If you’re building a real-time dashboard too
A few things I’d strongly recommend:
- Don’t let the frontend talk to too many things directly.
- Decide early what is snapshot data vs stream data.
- Design for reconnects, not just the initial page load.
- Think about partial failure early.
- Keep boundaries boring and explicit.
That alone will save you a lot of pain later.
Final thoughts
If you’ve built something similar — or if you would have designed this differently — I’d genuinely love to hear your approach.
I always find architecture discussions more useful when they’re about trade-offs as well as best practices.
Disclosure: This article was proofread and polished with AI assistance, but the core ideas, architecture decisions, and final content are my own.
kaustubh-26
/
flux-platform
An event-driven real-time data platform
Flux Platform
Flux — An event-driven real-time data platform
Live Demo: https://flux.kaustubhalandkar.workers.dev
A production-style, event-driven real-time data platform showcasing modern Kafka-based streaming, WebSocket fan-out, and a clean Backend-for-Frontend (BFF) architecture.
This repository is intentionally built as a portfolio-grade, open-source system that mirrors how real-world, streaming data platforms are designed, operated, and documented.
🖥️ Live Dashboard Preview
📱 Mobile View
✨ What This Project Demonstrates
- Event-driven microservices using Kafka
- A resilient BFF layer for real-time fan-out
- Socket.IO–based client streaming
- Cache-accelerated hydration with graceful degradation
- Idempotency & deduplication patterns
- Structured logging & observability
- Self-healing Kafka connectivity
- Clean, scalable monorepo organization
The project emphasizes real-world system design concerns such as event-driven communication, failure handling, and scalability.
🧠 High-Level Architecture
┌──────────────────────┐
│ Client │
│ Browser / Mobile │
└─────────▲────────────┘
│ Socket.IO (real-time)
┌─────────┴──────────────┐
│ Backend-for-Frontend │
│ (BFF) │
│ - Socket.IO Server │
│ - Kafka Producer │
Top comments (0)