Middleware Stream Architecture: How it Breaks Away from Traditional Streams

Hellooo! I’ve been diving deep into async data handling lately, and I couldn’t shake the feeling that traditional stream architectures—like RxJS—weren’t quite cutting it for me.

That’s why I came up with something new: the Middleware Stream Architecture, baked into my project Asyncrush.

It’s my first shot at rethinking how we deal with async streams, and I’m excited to share how it stands apart from the usual suspects. Let’s break it down together and see what’s different!

---

What’s a “Traditional” Stream Architecture?

First, a quick refresher. If you’ve heard of RxJS or Java’s Reactive Streams, you’ve met the classics. These are built on the observer pattern, which works like this:

You’ve got a “stream” pumping out data over time.
An “observer” catches that data and does something with it.
“Operators” step in to transform or filter the data along the way.

Think of a live chat app: messages roll in, and you chain operators like “filter -> transform -> display.” It’s solid and structured, but it’s got some quirks. So, how does Asyncrush shake things up? Let’s dive into the key differences.

Approach: Operators vs. Middleware

• Traditional (Operator-Based): Tools like RxJS lean on predefined “operators.” You chain things like map, filter, or merge to shape your stream. Want to double a value? You’d write stream.pipe(map(x => x * 2)). It’s clean and systematic, but you’re stuck with the operator toolbox they give you.

• Asyncrush (Middleware-Based): I flipped the script here. Instead of operators, I went with “middleware”—plain old functions you slot into the stream. Doubling a value? Just stream.use(x => x * 2). No predefined list—you craft whatever function fits your vibe.

What’s Different?: Traditional feels like “pick from their toolkit,” while Asyncrush is “build your own tools.” It’s way more freeing and flexible.

Performance: Heavy & Complex vs. Light & Snappy

• Traditional: RxJS is a beast—packed with features, but that comes at a cost. Stack a bunch of operators, and it starts churning out objects and crunching complex logic, slowing things down. Big datasets can really bog it down.

• Asyncrush: I wanted lean and mean. Middleware runs as simple functions, and I ditched unnecessary abstractions.

What’s Different?:
Think of traditional as a “jack-of-all-trades longsword”—versatile but hefty. Asyncrush is a “sharp, quick dagger”—laser-focused on performance.

Practicality: Theory-Driven vs. Real-World Ready

• Traditional: RxJS is perfection on paper. Rooted in reactive programming math, it’s systematic but can take ages to learn. Even simple tasks might churn out long code.
• Asyncrush: I zoomed in on real problems. Need retries after an error? Want to split data to multiple spots? I made that stuff easy, skipping the theory for “how do I fix this now?”

What’s Different?:
Traditional chases “textbook elegance,” while Asyncrush goes for “get-it-done convenience.” It’s built for the trenches.

---

So, What’s the Big Picture?

Boil it down, and traditional stream architectures are like a “massive toolbox”—powerful, but heavy and a bit of a chore to wield.

Asyncrush’s Middleware Stream Architecture? It’s a “sleek multitool”—packing just what you need to move fast and free.

1. Complexity: Operator chains vs. middleware chains.
2. Speed: Bulky and slow vs. light and zippy.
3. Control: Add-on tools vs. built-in flow mastery.
4. Goal: Theoretical polish vs. practical wins.

---

Why This Matters to Me (and Maybe You)

Asyncrush came from me scratching my head over traditional limits, wondering, “Can’t this be better?” I wanted stream handling to feel fast, simple, and real—less stress, more joy.

It’s my first go at this, so it’s not perfect yet, but I’d love to hear what you think. Could this flip your async game? Hit me up in the comments—let’s riff on it together!

Comments

[Dario Mannu]

Hi, love that! Looks like we thought about the same thing at the same time.

I was asking myself the same question about Observables, then I came up with the idea of callforwards.

I agree with you that Observables — especially their operators — are the masterpiece, but if we talk about raw performance, making things the lightest and fastest we can think of, then all this middleware idea starts to make sense and was my motivation, too.

[miinhho]

Thank you for your comment!

I see some interesting parallels in our thinking about data processing.

While both approaches optimize data flow, the Middleware Stream Architecture I've proposed focuses on a reactive stream paradigm:

const stream = new RushStream<number>((observer) => {
  observer.next(1);
});
stream.use(
  (v: number) => v + 1,
  (v: number) => v * 2
).listen({
  next: (value) => {
    console.log(value);
  },
  complete: () => { },
});

This differs from Callforwards in how control flows through the pipeline - here using a composable observer pattern rather than explicit next() calls that Callforwards employs.

I'd be interested to hear more about your performance optimizations with Callforwards!

I agree that when focusing on raw performance, middleware approaches can offer significant advantages over traditional Observable implementations.

[Dario Mannu]

The shape of this middleware defined as plain functions is interesting, but I believe we may want to explore a few use cases whose solution is not obvious (to me) yet.

Do you think you could help me formulate an equivalent example of the following, using this functional middleware?

stackblitz.com/edit/drag-n-drop-ca...

[miinhho]

Of course!

But If it's okay, I want to work with cases like that with you in github.

I'd like to talk more in detail, is there an email where I can send my Discord contact information?