The Evolution of Signals in JavaScript

There has been some buzz recently in the frontend world around the term "Signals". In seemingly short order they seem to be everywhere showing up in everything from Preact to Angular.

But they are not a new thing. Not even remotely if you consider you can trace roots back to research in the late 1960s. At its foundation is the same modeling that enabled the first electronic spreadsheets and hardware description languages (like Verilog and VHDL).

Even in JavaScript, we've had them since the dawn of declarative JavaScript Frameworks. They've carried various names over time and come in and out of popularity over the years. But here we are again, and it is a good time to give a bit more context on how and why.

Disclaimer: I am the author of SolidJS. This article reflects the evolution from the perspective of my influences. Elm Signals, Ember's computed properties, and Meteor all deserve shoutouts although not covered in the article.

Unsure what Signals are or how they work? Check out this Introduction to Fine-Grained Reactivity:

A Hands-on Introduction to Fine-Grained Reactivity

Ryan Carniato ・ Feb 9 '21

#javascript #webdev #reactivity

---

In the Beginning...

It is sometimes surprising to find that multiple parties arrive at similar solutions around exactly the same time. The starting of declarative JavaScript frameworks had 3 takes on it all released within 3 months of each other: Knockout.js (July 2010), Backbone.js (October 2010), Angular.js (October 2010).

Angular's Dirty Checking, Backbone's model-driven re-renders, and Knockout's fine-grained updates. Each was a little different but would ultimately serve as the basis for how we manage state and update the DOM today.

Knockout.js is of special importance to the topic of this article, as their fine-grained updates were built on what we've come to call Signals. They introduced initially 2 concepts observable (the state) and computed (side effect) but over the next couple of years would introduce the 3rd pureComputed (derived state) to the language of the frontend.

const count = ko.observable(0);

const doubleCount = ko.pureComputed(() => count() * 2);

// logs whenever doubleCount updates
ko.computed(() => console.log(doubleCount()))

---

The Wild West

Patterns were a mix of patterns learned from developing MVC on the server and the past few years of jQuery. One particular common one was called Data Binding which was shared both by Angular.js and Knockout.js although in slightly different ways.

Data Binding is the idea that a piece of state should be attached to a specific part of the view tree. One of the powerful things that could be done was making this bi-directional. So one could have state update the DOM and in turn, DOM events automatically update state all in an easy declarative way.

However, abusing this power ended up being a foot gun. But not knowing better we built our applications this way. In Angular without knowledge of what changes it would dirty check the whole tree and the upward propagation could cause it to happen multiple times. In Knockout it made it difficult to follow the path of change as you'd be going up and down the tree and cycles were common.

By the time React showed up with a solution, and for me personally, it was the talk by Jing Chen that cemented it, we were more than ready to jump ship.

---

Glitch Free

What was to follow was the mass adoption of React. Some people still preferred reactive models and since React was not very opinionated about state management, it was very possible to mix both.

Mobservable (2015, later shortened to MobX) was that solution. But more than working with React it brought something new to the table. It emphasized consistency and glitch-free propagation. That is that for any given change each part of the system would only run once and in proper order synchronously.

It did this by trading the typical push-based reactivity found in its predecessors with a push-pull hybrid system. Notification of changes are pushed out but the execution of the derived state was deferred to where it was read.

For a better understanding of Mobservable's original approach check out: Becoming Fully Reactive: An in Depth Explanation of Mobservable by Michel Westrate.

While this detail was largely overshadowed by the fact that React would just re-render the components that read changes anyway, this was a monumental step forward in making these systems debuggable and consistent. Over the next several years as algorithms became more refined we'd see a trend towards more pull based semantics.

---

Conquering Leaky Observers

Fine-grained reactivity is a variation of the Gang of Four's Observer Pattern. While a powerful pattern for synchronization it also has a classic problem. A Signal keeps a strong reference to its subscribers, so a long-lived Signal will retain all subscriptions unless manually disposed.

This bookkeeping gets prohibitively complicated with significant use, especially where nesting is involved. Nesting is common when dealing with branching logic and trees as you'd find when building UI views.

A lesser-known library, S.js (2013), would present the answer. S developed independently of most other solutions and was modeled more directly after digital circuits where all state change worked on clock cycles. It called its state primitive Signals. While not the first to use that name, it is where the term we use today comes from.

More importantly, it introduced the concept of reactive ownership. An owner would collect all child reactive scopes and manage their disposal on the owner's own disposal or were it ever to re-execute. The reactive graph would start wrapped in a root owner, and then each node would serve as an owner for its descendants. This owner pattern is not only useful for disposal but as a mechanism to build Provider/Consumer context into the reactive graph.

---

Scheduling

Vue (2014) also has also made huge contributions to where we are today. Besides being in lockstep with MobX with advances in optimizing for consistency, Vue has had fine-grained reactivity as its core since the beginning.

While Vue shares the use of a Virtual DOM with React, reactivity being first-class meant it developed along with the framework first as an internal mechanism to power its Options API to, in the last few years, being front and center in the Composition API (2020).

Vue took the push/pull mechanism one step forward by scheduling when the work would be done. By default with Vue all changes are collected but not processed until the effects queue is run on the next microtask.

However, this scheduling could also be used to do things like keep-alive(preserving offscreen graphs without computational cost), and Suspense. Even things like concurrent rendering are possible with this approach, really showing how one could get the best of both worlds of pull and push-based approaches.

---

Compilation

In 2019, Svelte 3 showed everyone just how much we could do with a compiler. In fact, they compile away the reactivity completely. This is not without tradeoffs, but more interesting is Svelte has shown us how a compiler can smooth out ergonomic shortcomings. And this will continue to be a trend here.

The language of reactivity: state, derived state, and effect; not only gives us everything we need to describe synchronized systems like user interfaces but is analyzable. We can know exactly what changes and where. The potential for traceability is profound:

If we know that at compile time we can ship less JavaScript. We can be more liberal with our code loading. This is the foundation of resumability in Qwik and Marko.

---

Signals into the Future

Given how old this technology is, it is probably surprising to say there is much more to explore. But that is because it is a way of modeling solutions rather than a specific one. What it offers is a language to describe state synchronization independent of any side effect you'd have it perform.

It would seem unsurprising perhaps then that it would be adopted by Vue, Solid, Preact, Qwik, and Angular. We've seen it make its way into Rust with Leptos and Sycamore showing WASM on the DOM doesn't have to be slow. It is even being considered by React to be used under the hood:

And maybe that's fitting as the Virtual DOM for React was always just an implementation detail.

Signals and the language of reactivity seem to be where things are converging. But that wasn't so obvious from its first outings into JavaScript. And maybe that is because JavaScript isn't the best language for it. I'd go as far as saying a lot of the pain we feel in frontend framework design these days are language concerns.

Wherever this all ends up it has been quite a ride so far. And with so many people giving Signals their attention, I can't wait to see where we end up next.

Comments

[ecyrbe]

Hello Ryan,

Since we are in the evolution of things, i'd like to emphasize that all those javascript concepts where heavilly inspired by a pioneer c++ library : Qt

Indeed, all this is 30 years old at least. Advanced notions of signals where developped by trolltech who created Qt C++ widget library.

It's worth noting that signals in solid js are called Properties in Qt world and that Observable/Observers used to implement those Properties where called Signal/slots in Qt.

Here is a link to Qt Bound properties for the curious.

[Richey Ryan]

It seems like Bindable Properties were only released in QT 6 which was released in 2020. Is there some other reactive construct in QT that predates this? I'm always curious to see how these concepts have evolved over time, particularly if its in a language like C++ which is less permissive than a language like JavaScript.

[Boonyasuk Dheeravongkit]

Like I previously mentioned in Evan You's twitter thread, Observable-based reactivity is dated back to the dawn of modern GUI we know today. Back in the '70s, people at Xerox PARC created GUI with Smalltalk and introduced ValueHolder: a class that holds a mutable value and notifies its dependents whenever its value changes. Its invention came as to make their GUI interactive.

So no, Qt isn't, in any shape or form, a pioneer in this space.

AFAIK, many languages took after Smalltalk and provide Observable-based reactivity, though its use is no longer confined in UI development realm. Many languages introduce the concept as pairs: Observable/Observer, Listenable/Listener, etc.

If you like to read more on how Smalltalk approaches this subject, you can read more here.

[Nikola Stojaković]

In short, everything old is new again. People often keep thinking that their favorite technology / tool they use on the daily basis was the one which brought the innovation. But just like with many other things in life, timing is important too, not just the innovation.

[Joe Pea]

Good timing indeed! Surplus.js showed us that signals + JSX are a great DX. Then Solid.js had a stab at it, and for various reasons, the community took off. Although reactivity isn't new, this is the first time that it landed within the JSX era of web development. React Hooks look like signals and effects, and they were keen on the DX, but the way they work isn't signals and effects.

As the article hints, Meteor Tracker is a good prior art on signals and effects history in JS (although not using the same terminology). The really magical thing about Meteor is that their "signals" and "effects" are database connected: you use variables in your frontend template, and those values can come directly from a backend DB. In many ways it is still revolutionary. What's gonna be the next revolutionary full JS stack?

[Joe Pea]

Qt QML first appeared in 2009 (and surely was based on older ideas), and like Solid JSX is a declarative-reactive tree-defining language where when any variable in a QML expression changes (dependencies, aka Signals), the expression re-runs and that part of the tree is automatically updated. To explain with QML syntax, but hypothetically manipulating DOM objects, it looks like this:

form {
  input {
    type: "number"
    value: someValue + otherValue
  }
}

In this hypothetical QML-to-DOM example, the expression someValue + otherValue re-runs if either variable change (much like a "controlled input" in React, Preact, Solid, etc).

[ecyrbe]

Bindable properties where there in Qt5, Qt6 added them to lower level code.
So at minimum since 2012.

But, Properties where there since Qt 3 so 2001 and i was doing property binding whith them at the time, they where just not published with the lib, but it was a pattern known to those using Qt.

But like other have said, Xerox that pioneered ui was also doing them in the 70's

[Rajeev J Sebastian]

Signals and slots have been the cornerstone of Qt programming since atleast Qt 3 or even earlier. I moved from Wx to Qt simply to take advantage of signals and slots back in 2002 and by then it was quite well developed (used macros and a custom C++ pre processor)

[Ville M. Vainio]

They are called ”signals” in Qt but they are a different thing. Qt signals are just places where you can hook callbacks, not observable values.

[Rajeev J Sebastian]

In that case I would need to send a complaint to my English teacher.

Jokes aside, in Qt, signals come with values. The receiver of a signal receives the changed value.

Signals represent an observable attribute of any object in Qt, and allow to decouple the receiver from the sender.

[chamberian]

of course, the qml which is a part of qtquick released in Qt4.0 has support value and expression bindings, in my opinion, the signal-slot mechanism is far ahead of any web front-end framework.

[Ryan Carniato]

It's a component independent state management system that works off minimal language (signal, computed, effect) to guarantee consistent glitch-free synchronization of state and view.

This might sound like a lot of things admittedly. But for Angular specifically this means the ability to eventually deprecate Zone.js as Signals allows direct notification of what changes in the system without dirty checking everything and sits more naturally in templating solutions than things like RxJS that require direct pipe. If Rx is for async reactivity, Signals are for synchronous.

[Joe Pea]

In simple terms: signals are things that hold a value; reactive variables so to speak. Effects are blocks of code (defined with createEffect in Solid) that automatically re-run when any signals accessed inside of them change.

The way that this helps with practical performance, compared to React for example, is that JSX is compiled into effects that update DOM directly in the most efficient way possible, without an abstraction like vdom (more on the difference below). To demonstrate very simply, the following Solid.js JSX code:

const [
  count, // getter
  setCount // setter
] = createSignal(0)

setInterval(() => {
  // Increment count every second 
  setCount(count() + 1)
}, 1000)

createEffect(() => {
  // log count whenever it changes
  console.log(count())
})

// This HTMLDivElement will be updated every second
const div = <div>count: {count()}</div>

document.body.append(div) // It is DOM, not vdom.

compiles to essentially something like this:

const [
  count, // getter
  setCount // setter
] = createSignal(0)

setInterval(() => {
  // Increment count every second 
  setCount(count() + 1)
}, 1000)

createEffect(() => {
  // log count whenever it changes (runs once initially)
  console.log(count())
})

const div = document.createElement('div')

createEffect(() => {
  // This HTMLDivElement will be updated every second (runs once initially)
  div.textContent = `count: ${count()}`
})

document.body.append(div) // It is DOM, not vdom.

Example in Solid.js playground

If you don't have a build tool in place to compile JSX, you can use Solid.js's html template string tag function. The JSX part would be written like this:

// This HTMLDivElement will be updated every second
const div = html`<div>count: ${count}</div>`

document.body.append(div) // It is DOM, not vdom.

Here is that same example on CodePen, using no build tools (plain HTML and JS):

One more example of buildless Solid.js usage, making a function component:

How does this compare to vdom? In particular, how does it compare with Angular? Vdom is a diffing approach: every time React/Preact/Lit components update, they create a fake tree representation of the DOM, figure out the difference between that and the previous fake tree, then apply the differences to DOM. See this Angular article for more on that. TLDR, direct updates to the DOM (essentially like I showed in the example compile output) are memory efficient because they don't create trees (that need to be garbage collected) on every update. Now imagine you update in an animation loop: with vdom you'll be making a lot of garbage to collect, potentially tanking performance, and will be forced to either move your animation out of the declarative markup (not ideal for dev experience) or try to optimize your template by splitting it into the smallest components possible (not necessarily ideal if you didn't have a performance problem and otherwise didn't really need to make separate components).

By not using a VDOM, and updating the DOM directly using the most efficient methods, Solid.js achieves more performance. We can see (from that previous Angular article) that although Angular Ivy does fine grained updates -- a conceptually a better approach just like what Solid.js is doing -- benchmarks show that it still isn't as fast as it can be (I don't know exactly why, but probably due to the way change propagation is implemented, maybe related to Zone.js that Ryan mentioned, or something else, but it would be worse with the vdom in place).

The JSX compile output is actually more advanced than my naive hand-written output example. For example, in the expression <div>count: {count}</div>, Solid.js JSX output actually creates two Text nodes, one for the "count: " string, and one for the dynamic value, then updates only the text node that needs updating. The compiler has various sorts of optimizations, whereas my simplified hand-written output example is a simpler un-optimized concept that is more typical of hand-written code and is useful for describing what Solid.js JSX compiles to in a simplified way.

[Rasmus Schultz]

If you have async reactivity already, why do you need sync reactivity? What is it for? When does it matter?

What's lacking with RxJS in the context of Angular for example?

[Ryan Carniato]

Solving the diamond problem. We can make guarentees on execution propagation. The core team was talking to me a bit on stream about this problem of over fetching from APIs at times. Also it removes the need for the special mechanisms around Direct Pipe to my understanding. The auto tracking form is easier for templating as you can just drop expressions in.

[Joe Pea]

The diamond problem and sync vs async are tangential. Sync vs async can be a stylistic implementation choice, and although Solid has synchronous effects by default, primitives can be built on them that are totally async, including createAsyncSignal and/or createAsyncEffect (implementation left to the imagination).

Personally, my stylistic choice, were I to implement something from scratch, would be for JavaScript microtasks (queueMicrotask) to be the smallest unit of reactivity time, therefore making reactions (effects) always "async" (from the perspective of JavaScript code in general) by default: modify a signal, the effect for that signal is always in the next microtask. I would pick this as an out-of-box design because it inherently prevents too many things from running. In essence it would be similar to timing of ResizeObserver, MutationObserver, etc, which are async, and the purpose is to avoid unnecessary extra work.

Solid does have its own microtask-like timing within a Solid component, where it will not execute effects until synchronous code of a Solid component tree is done running. It is very similar to microtasks, but scoped only to Solid.js component execution. When integrating Solid.js code with external libraries, it becomes apparent that Solid.js runs effects "async" relative to its component tree, but not relative to JavaScript code in general.

These things are stylistic choices: a hypothetical like Solid.js alternative, with the same essential developer experience of signals and effects, can exist but be completely async based on microtasks. Signals and effects moreso decribe the DX pattern, but not the timing of the pattern.

As a parallel example using event emitter patterns, it could very be possible that one implementation of an event emitter runs event handlers synchronously (right when events are emitted) and some other event emitter implementation may always run event handlers in a future task (f.e. in a JS microtask). This doesn't fundamentally change the fact that either way we have an event emitter pattern. We have the same pattern, implemented with different timing semantics.

The same applies here.

[Rasmus Schultz]

You might want to have a look at Voby -

github.com/vobyjs/oby#effect

It's basically Solid with different opinions - looks like you'd agree with this one.

Unfortunately it's a one man project, and not really built to complete with Solid - although it could have been. I think we need more competition in this space to explore the pros and cons of various ideas and opinions.

[Peter Verswyvelen]

Very nice article!

Aren’t signals just another form of Functional Reactive Programming or Synchronous Dataflow Languages? The latter exists for a very long time, and is used to build mission critical software (e.g. Lucid Synchrone, Lustre, SCADE from Ansys, …). The former exists in many libs like BaconJS, SodiumFRP, ReactiveBanana … but never became popular for some reason. Google and Brown University made FlapJAX, but that didn’t catch on either.

I hope this time signals will finally become widespread!

PS: the past 11 years I have been working on ViKiD an educational/creative website to learn the concept of signals, not for HTML, but for 2D reactive graphics and simple games. It’s far from finished, but I hope it will help adoption of the signal-way of thinking, which is more declarative, and allows better reasoning. All schools use MIT Scratch to teach our kids how to code, but that feels so imperative… IMHO signals are much more interesting to learn, and they stimulate a mathematical and declarative way of thinking.

[Joe Pea]

Wow, Flapjax is interesting! It is like React, but without JSX, and from back in 2009!

Wow, ViKiD is really neat!

[3rdp]

While not the first to use that name, it is where the term comes from we use today.

“The term comes from we use today” I think that’s a typo or a missed word somewhere in there

UPD: actually it’s the order of words in a sentence, it probably means “it is where the term we use today comes from”.

[Dustin Brett]

Noticed similar thing with "We know can exactly what changes and where." likely being "We can know exactly what changes and where."

[Ryan Carniato]

Thank you both.. I Yoda speak a lot and it sometimes sneaks past Grammarly into my articles.

[B00MER]

UDP vs TCP, the same you could say for these frontend paradigms.

One quick but no guarantee of a return. The other slower & heavier on hardware but you're guaranteed a valid response.

[Ron Valstar]

I'm not that surprised about the beginning. The year 2010 was when 'Apple killed Flash' and a lot of developers were switching from ActionScript to JavaScript (both being ECMAScript). In ActionScript I was using as3-signals by Robert Penner (inspired by QT). When I went back to JavaScript I traded it for js-signals by Miller Medeiros. (there's also Robotlegs)
I've always loved signals, especially for smaller, framework-less projects. So I am surprised about it making a comeback in the bigger frameworks. Although signals now is a bit different; more state, less pub/sub.

[John Peters]

Ryan; I had studied 3270 and 5250 terminal protocols in the past. These were the streams to update content as the user typed. Both those protocols only sent one changed char at a time. The only change on the terminal was that single char. There was enough information in the push to identify the position, new attributes and the char itself. The whole exchange was less than 10ms. I guess we could say this was a signal which as you mentioned was developed 40 or 50 years ago.

[Ben Racicot]

I cannot find anything on the current state of native signals in JavaScript. Is there a proposal? What is the best way to work with them now?

[Ryan Carniato]

This is very very new idea. A proposal is being worked on, I'd expect some sort of initial draft published by November at the latest. At which point everyone can get involved.

[Ryan Carniato]

React's Hook complaint from React developers center around dependency arrays, needing hooks to indicate memoization, and how useEffect is almost an abstraction leak. The messy bloated code is almost completely related to those. The syntax is almost never a concern within the community.

It will be interesting to see what is done here give Angular does a lot with classes. They've been showing off the low level primitives. I suspect they will end up getting in there like that even if this. kinda defeats to the point. Being decoupled from the component is a super power.

[Jim Moore]

Is learning solid easier for react devs than poj devs.
I never used react. Played w vue n svelte but am not finding many examples of solid for framework newbies. Referrals pls. I really want to use solid cuz i do appreciate its philosophy.

• Old C guy

[Darryl Noakes]

Vue's hook-like Composition API was dead-on-arrival because of it.

I disagree. As Ryan said, the issues with React are the dependency arrays, which are not present in pretty much any other framework, including Vue.
As the Vue docs say (Comparison with React Hooks), the Composition API avoids these issues due to the different reactivity system, how component code is run, and how components are rendered.

Is it possible to get the benefit of signals with normal code, like this.count = this.count + 1 instead of setCount(count() + 1)?

This is the equivalent in Vue's composition API: myThing.value = myThing.value + 1 or myThing.value += 1

[Simon Thuillier]

Hi, good post providing an interesting approach about the troubled history of JS frameworks for the last 15 years.

I can't help but try to link it to my current activity involving reactive programming in Java/Spring ecosystem. There is a paper from 1985 "on the development of reactive systems" (Harel & Pnueli) which already highlighted some interesting properties about this paradigm stating notably : "A reactive system does not compute or perform a function, it maintains a certain ongoing relationship with its environment. "

While i'm unsure how to link it to your post I can't help but feel a relationship. I also notice this tendency of JS frameworks to go forward the principles of the reactive manifesto (2014) : message driven, elastic, responsive and resilient architectures.

[Robert Arteaga]

It seems like the direction the web is going is that there will a framework that precompiles html css js ahead of time like Qwik while also giving the DX of React with JSX and plug-in based functionality (hooks) and gives you the reactivity of Leptos/Solid. I don’t think the avg web dev will be writing rust in a day to day basis but there is some future bridge there with WASM interop finally opens. We are moving completely away from VDOM as it is non-reactive in every sense of the word and most importantly slow.