Building a Reactive Library from Scratch

In the previous article A Hands-on Introduction to Fine-Grained Reactivity I explain the concepts behind fine-grained reactivity through example. Now let's look at building a reactive library ourselves.

There is always something that seems a bit magical when you see it in action but mechanically it isn't that complicated. What makes reactivity feel so magical is once put in place it takes care of itself even under dynamic scenarios. This is the benefit of true declarative approaches as the implementation doesn't matter as long as the contract is kept.

The reactive library we will build won't have all the features of something like MobX, Vue, or Solid but it should serve as a good example to get a feel for how this works.

Signals

Signals are the core of our reactive system and are the right place to start. They contain a getter and setter so we might start with something like this:

export function createSignal(value) {
  const read = () => value;
  const write = (nextValue) => value = nextValue;
  return [read, write];
}

This doesn't do much of anything just yet but we can see that we now have a simple container to hold our value.

const [count, setCount] = createSignal(3);
console.log("Initial Read", count());

setCount(5);
console.log("Updated Read", count());

setCount(count() * 2);
console.log("Updated Read", count());

So what are we missing? Managing subscriptions. Signals are event emitters.

const context = [];

function subscribe(running, subscriptions) {
  subscriptions.add(running);
  running.dependencies.add(subscriptions);
}

export function createSignal(value) {
  const subscriptions = new Set();

  const read = () => {
    const running = context[context.length - 1];
    if (running) subscribe(running, subscriptions);
    return value;
  };

  const write = (nextValue) => {
    value = nextValue;

    for (const sub of [...subscriptions]) {
      sub.execute();
    }
  };
  return [read, write];
}

There is a bit to unpack here. There are two main things we are managing. At the top of the file, there is a global context stack that will be used to keep track of any running Reactions or Derivations. In addition, each Signal has its own subscriptions list.

These 2 things serve as the whole basis of automatic dependency tracking. A Reaction or Derivation on execution pushes itself onto the context stack. It will be added to the subscriptions list of any Signal read during that execution. We also add the Signal to the running context to help with cleanup that will be covered in the next section.

Finally, on Signal write in addition to updating the value we execute all the subscriptions. We clone the list so that new subscriptions added in the course of this execution do not affect this run.

This is our finished Signal but it is only half the equation.

Reactions and Derivations

Now that you've seen one half you might be able to guess what the other half looks like. Let's create a basic Reaction(or Effect).

function cleanup(running) {
  for (const dep of running.dependencies) {
    dep.delete(running);
  }
  running.dependencies.clear();
}

export function createEffect(fn) {
  const execute = () => {
    cleanup(running);
    context.push(running);
    try {
      fn();
    } finally {
      context.pop();
    }
  };

  const running = {
    execute,
    dependencies: new Set()
  };

  execute();
}

What we create here is the object that we push on to context. It has our list of dependencies (Signals) the Reaction listens to and the function expression that we track and re-run.

Every cycle we unsubscribe the Reaction from all its Signals and clear the dependency list to start new. This is why we stored the backlink. This allows us to dynamically create dependencies as we run each time. Then we push the Reaction on the stack and execute the user-supplied function.

These 50 lines of code might not seem like much but we can now recreate the first demo from the previous article.

console.log("1. Create Signal");
const [count, setCount] = createSignal(0);

console.log("2. Create Reaction");
createEffect(() => console.log("The count is", count()));

console.log("3. Set count to 5");
setCount(5);

console.log("4. Set count to 10");
setCount(10);

Adding a simple Derivation isn't much more involved and just uses mostly the same code from createEffect. In a real reactive library like MobX, Vue, or Solid we would build in a push/pull mechanism and trace the graph to make sure we weren't doing extra work, but for demonstration purposes, I'm just going to use a Reaction.

Note: If you are interested in implementing the algorithm for his push/pull approach I recommend reading Becoming Fully Reactive: An in-depth explanation of MobX

export function createMemo(fn) {
  const [s, set] = createSignal();
  createEffect(() => set(fn()));
  return s;
}

And with this let's recreate our conditional rendering example:

console.log("1. Create");
const [firstName, setFirstName] = createSignal("John");
const [lastName, setLastName] = createSignal("Smith");
const [showFullName, setShowFullName] = createSignal(true);

const displayName = createMemo(() => {
  if (!showFullName()) return firstName();
  return `${firstName()} ${lastName()}`
});

createEffect(() => console.log("My name is", displayName()));

console.log("2. Set showFullName: false ");
setShowFullName(false);

console.log("3. Change lastName");
setLastName("Legend");

console.log("4. Set showFullName: true");
setShowFullName(true);

As you can see, because we build the dependency graph each time we don't re-execute the Derivation on lastName update when we are not listening to it anymore.

Conclusion

And those are the basics. Sure, our library doesn't have batching, custom disposal methods, or safeguards against infinite recursion, and is not glitch-free. But it contains all the core pieces. This is how libraries like KnockoutJS from the early 2010s worked.

I wouldn't recommend using this library for all the mentioned reasons. But at ~50 lines of code, you have all makings of a simple reactive library. And when you consider how many behaviors you can model with it, it should make more sense to you why libraries like Svelte and Solid with a compiler can produce such small bundles.

This is a lot of power in so little code. You could really use this to solve a variety of problems. It's only a few lines away from being a state library for your framework of choice, and only a few dozen more to be the framework itself.

Hopefully, through this exercise, you now have a better understanding and appreciation of how auto-tracking in fine-grained reactive libraries work and we have demystified some of the magic.

---

Interested How Solid takes this and makes a full rendering library out of it. Check out SolidJS: Reactivity to Rendering.

Comments

[Ivan Jeremic]

I love this article! Why in Solid "batch" is a function you can import and use why no automatic baching, are there use-cases where you don't want to batch is that the reason?

[Ryan Carniato]

Yeah I really wanted to have things execute synchronously originally. And there are cases where injecting batch ourselves I thought was taking control away from the end user. I've decided since that probably isn't that big of a deal. But I generally start from a place of control.

[Lukas Bombach]

Hey Ryan,

I love this article, thank you for this!

You write that the algorithm you use here needs more work to prevent doing too doing too much work. Can you elaborate on that? I understood that each signal you update / set will set a chain in motion where every step will only trigger the next steps that are actually required. After each „chain reaction“ the whole dependency graph will be cleaned up and we‘re up to a fresh start. Sounds like no work can be done that should not be done, what am I missing, or getting wrong?

[Ryan Carniato]

This is true until you move to updating multiple signals in the same transaction. Then things get more complicated because you have to start somewhere but if you notify and execute off one signal and one of the descendants also depends on the other that changed you'd end up running it twice with a naive approach. Not to mention if you allow derivations to conditionally notify, like only if their value changes, simple approaches don't work.

[Lukas Bombach]

Ah, makes sense! Thank you!

[Evgenii Perminov]

Late reply but a simple example

const a = createSignal(1);
const b = createComputed(() => a.value * 3);
createEffect(() => logEffect(`A is ${a.value} and B is ${b.value}`));
a.value = 2;

This effect runs three times, not two - because update to a triggers update to b which triggers effect but also update to a itself triggers effect too

[Bill]

Hey Ryan,

Thanks for the blog post.
Q: What's the difference between

const displayName = createMemo(() => {
  if (!showFullName()) return firstName();
  return `${firstName()} ${lastName()}`;
});

vs

const displayName = () => {
  if (!showFullName()) return firstName();
  return `${firstName()} ${lastName()}`;
};

createEffect(() => log("My name is", displayName()));

and without the createMemo, for my understanding, the subscription will be the same outcome, right?

Cheers
Bill

[Ryan Carniato]

You are correct. Good on you for noticing. I often have to teach people where they can use less primitives. People new to reactivity tend to overuse them, and it probably doesn't help that intro examples tend to be overly simplistic to illustrate mechanics rather than practical. And here I am guilty of that here.

In this scenario where there is only a single observer that has no other dependencies this will effectively execute the same number of times. I explain this and the example a bit more in my previous article A Hands-on Introduction to Fine-Grained Reactivity. I show an example in that article illustrating the difference between using a thunk or using createMemo in an example where it does make a difference.

The main benefit of createMemo is to prevent re-calculations, especially expensive ones. This is only relevant if it is used in multiple places or the thing that listens to it could be triggered a different way which could also cause re-evaluation. It's just a caching mechanism, and like any memoization premature application actually can have negative impact since it costs extra memory and brings creation overhead. In real apps in any example where I was just calculating a string I'd probably not bother and just function wrap if there was reason not to inline it. But I'm also a bit of a performance nut.

Aside: This is actually one of the reasons I personally like explicit reactivity. A lot of libraries/APIs sort of push this memoization on you by default or hide it in their templating. It's excellent for update performance but you end up with higher creation costs. A typical sign of this behavior is when they have APIs like isObservable etc.. as it means they make decisions based on a value being a primitive so wrapping in a thunk would not be sufficient.

[Bill]

Thanks for the awesome and detailed reply. It does make sense. I really enjoy those reactive programming articles, hope to see more in the near future. ✌🏻

[Talin]

Very helpful article. One thing I would like to understand better is the lifecycle of effects, particularly when disposing of effects and signals that are no longer needed - it looks like the cleanup handler never gets called if you never set a signal.

I realize that in this simple example, effects aren't holding on to any resources, so everything would be garbage collected; but I'm having trouble seeing how you would extend this model to one where effects do need to be cleaned up.

My motivation, BTW, is I am working on a 3D game engine where character behavior is driven by a reactive framework that is similar to what you've outlined here. I'm in the process of refactoring it from a React hook style of reactivity to something more like Solid (with some important differences, such as reflection and type introspection).

[Ryan Carniato]

Yeah this article doesn't actually get into anything hierarchical, or implement any sort of disposal. In Solid we have an idea of ownership where child effects are owned by parents as well and follow the same disposal pattern. If you want to understand this a bit more I recommend reading the article linked at the end, Reactivity to Rendering. I go into how I built a framework on top of this and it talks more about disposal and ownership.

[alpatovdanila]

Hi! Thank you for the great article!

Reading comments and your answers and then running code from the article i found that nested effects will create new subscription everytime outer effect runs, without cleaning up previous subscriptions, causing zombie inner subscriptions to appear. codesandbox.io/s/creatememo-forked...

You have mentioned nested effects two times:
Answering the question about why do we need array of contexts and not a single context

Nested reactivity. Generally you can nest effects or derivations inside themselves. This is important to how Solid renders but I suppose wasn't needed for this simple example.

And the second time, answering the question about "why do we need clean-up in general"

The fact that we dynamically create and tear down subscriptions on each run has us doing this work anyway.

Concidering these answers, i had come to conclusion that your code should already deal with that situation, since it has context stack and is, indeed, tearing up subscriptions, but it fact that does not help with nested effects. You have mentioned clean-up is needed to solve that, did you meant some mechanism that is not presented in the code of that article? In my understanding, some will need to store tree of contexts and dispose children subscriptions before re-executing current context to avoid zombie children.

So i am wondering if i missed out something or read something wrong.

Thank you!

[Ryan Carniato]

Yes, an actual reactive system is more complicated. I'm missing cleanup on nested effects, but to be fair most reactive libraries do. Mobx and the like. They expect the user to handle final cleanup. Solid is special in that we build a hierarchical ownership graph to do this ourselves and this works really well with our rendering system, but it is far from the norm.

I guess showing a disposal API would make it simpler for this example, but it also distracts a bit since we don't use that in Solid. But implementing Solid's hierarchical system is also out of scope here.

I talk about Solid's ownership concept in my followup article Reactivity to Rendering.

[缪宇]

Hey Ryan, great article, I benefit a lot from it, thank you so much.

Q: Why use context stack?

I replaced the context stack, and it words fine. full code

let context = ""

const read = () => {
    const running = context;
    if (running) subscribe(running, subscriptions);
    return value;
};

const execute = () => {
    cleanup(running);
    context = running;
    try {
      fn();
    } finally {
      context = "";
    }
};

[Ryan Carniato]

Nested reactivity. Generally you can nest effects or derivations inside themselves. This is important to how Solid renders but I suppose wasn't needed for this simple example.

[缪宇]

I nested effects, but it also works fine. full code

[Ryan Carniato]

If you try to subscribe after the inner effect it doesn't track. Change the example to:

createEffect(() => {
  createEffect(() => {
    log("value is ", value());
  });
  log("name is ", name());
});

Keep in mind this is just a simple approximation. I don't actually use an array in my libraries and instead just store the out context and then restore it.

[Rasmus Schultz]

In order to better make sense of the types, I tried porting this to TypeScript - I can't make sense of the type of the dependencies property of a subscription.

Specifically, I can't make sense of these lines in the reader:

    if (running) {
      subscriptions.add(running);

      running.dependencies.add(subscriptions);
    }

Here, subscriptions is a Set - but then you add the set of subscriptions to dependencies, which is also a Set (which gets created in createEffect) soooo... is it a Set of Sets (of Subscriptions?) or what's going on here?

Are you sure the code is correct?

TypeScript playground link

[Ryan Carniato]

I mean I have a working example in the article. I'm basically backlinking the signal because on effect re-run we have to unsubscribe from all the signals that it appears in their subscription lists. So the link has to work both ways. The Signal here doesn't have an instance really and I just needed the subscriptions so I added that. I could have made it a callback maybe for clarity. This demo isn't how Solid actually works completely. It's just a way of doing the basics with next to no code.

[Rasmus Schultz]

Yeah, the example works - I guess I have trust issues with plain JS without types. I've seen too much JS that "works", but not for the right reason, or not the way it appeared to work, if you know what I mean?

I think types would better explain the data structures and concepts here - rather than making the reader reverse engineer the code to try to infer what's what. What's running, what's in dependencies, and so on.

It's challenging even after watching the video and reading the article.

I still can't make sense of the types.

[peerreynders]

I still can't make sense of the types.

Sometimes clarity comes from the chosen names.

Change #1:

// A Dependency has many different Subscribers depending on it
// A particular Subscriber has many Dependencies it depends on
type Dependency = Set<Subscriber>;
type Subscriber = {
  execute(): void;
  dependencies: Set<Dependency>;
};

Change #2:

  const subscriptions: Dependency = new Set();

Change #3:

function cleanup(running: Subscriber) {

Change #4:

  const running: Subscriber = {
    execute,
    dependencies: new Set(),
  };

TypeScipt Playground

[Rasmus Schultz]

// A Dependency has many different Subscribers depending on it
// A particular Subscriber has many Dependencies it depends on

Yes! This is what the article/video/example was missing. Thank you! 😀🙏

[Mehuge]

In the Reaction implementation, why is cleanup done before the next run and not after each run? (eg below)

export function createEffect(fn) {
  const execute = () => {
    context.push(running);
    try {
      fn();
    } finally {
      context.pop();
      cleanup(running);
    }
  };

  const running = {
    execute,
    dependencies: new Set()
  };

  execute();
}

[Ryan Carniato]

You wouldn't want to cleanup yet. Like picture it is a DOM widget like a jQuery Chart.. you wouldn't cleanup until the end. The cleanup being run is the previously registered one. It goes something like this:

// run 1
cleanup(); // no-op
fn(); // execute 1

// run 2
cleanup(); // cleanup 1
fn(); // execute 2

// run 3
cleanup(); // cleanup 2
fn(); // execute 3

// parent removes child
cleanup(); // cleanup 3

[Mehuge]

The dependencies need to remain so that if a signal happens it will trigger a reaction?

[Ryan Carniato]

Yes that too. That's probably the more foundational answer. The reason we do cleanup each time is it lets our dependencies be dynamic. There are optimizations to be done around this but this is the core mechanism that you will find in MobX or Vue, along with Solid.

[缪宇]

Hey Ryan, If I change [...subscriptions] to subscriptions. it will causes infinite loop.

  const write = (nextValue) => {
    value = nextValue;

    for (const sub of [...subscriptions]) {
      sub.execute();
    }
  };

to

  const write = (nextValue) => {
    value = nextValue;

    for (const sub of subscriptions) {
      sub.execute();
    }
  };

Error:

What is the cause? Thank you so much. Full Code

[Ryan Carniato]

Yep. It's because the first thing each computation execution does is remove itself from the subscription list and then during the process of execution if it reads the same signal it adds itself back on. What this means is if we don't clone the original list, while iterating over it the process of executing the computation will cause it to end up at the end of the list. And it will then execute it again within the same loop, and add itself to the end of the list and so on.

[chris-czopp]

Hey Ryan, I truly appreciate the very thoughtful architectural choices which shape Solid. Recently I've been switching from Virtual-DOM-based rendering to Solid for a rather complex project and it feels not only performant but so elegant! Keep up great work!

[SeongKuk Han]

Thank you so much, I just started to learn SolidJS and it was confusing to understand how createEffect detect changes of signals that used in the effect. Now, it's more clear then, thank you for this article!

[Itamar Silverstein]

Hi Ryan, Thanks for this post !

1. Why should the Effect itself hold a list of dependencies as well. The Signal holds a list of subscriptions that should be notified whenever the signal is modified, for example setCount(5), And that does the job right ? what am I missing ? In the code above effect's dependencies are being added but not really used anywhere except for cleaning itself but why do they exist in the first place).

2. In VueJS there is a hijacking of the state (data) itself and so at initialization the data properties are being replaced with reactive getters and setters (using defineProperty in the past and Proxy today) that notify listeners. In SolidJS is there hijacking as well ? or is it mostly 'pure functional' ?

Thanks !

[Ryan Carniato]

In the code above effect's dependencies are being added but not really used anywhere except for cleaning itself

1. That is why. Each Effect needs to unsubscribe itself on cleanup/re-run. We need to know since the Signal holds the subscriptions. A long lived Signal could fire an Effect indefinitely otherwise long after it should be disposed.
2. Solid has proxies but the prefered pattern is createStore which creates a readonly proxy and setter function combo. There is a createMutable like MobX/Vue available though.

[Ryan Carniato]

The observer pattern is inherently leaky. Signals don't need cleanup but any subscription does. If the signal outlives the subscriber it will have no longer relevant subscriptions. At minimum we need to mark the subscriber as dead. The fact that we dynamically create and tear down subscriptions on each run has us doing this work anyway. Consider this example:

const [a] = createSignal()
const [b] = createSignal()
createEffect(() => {
  console.log(a()); // track a
  createEffect(() => {
    console.log(b()); // track b
  })
})

Every time a changes you are creating a new reaction that listens to b. So a naive approach that didn't do cleanup would just keep appending more subscriptions to b. So if you updated a 3 times b would end up with 4 subscriptions. When you updated b it would console.log 4 times.

[piotrnajda3000]

Hey Ryan, thank you for the article series!

In the case of the demo you've built up in this article, the b would console.log 4 times. You writing "the fact that we dynamically create and tear down subscriptions on each run has us doing this work anyway," made me think it's not supposed to happen. But, it seems to make sense: each time a changes, a new effect is created, and so the b's will stack up.

I've read that with solid the nested effect, "would be disposed on each rerun of the "parent" effect. Is this what happens in Solid, that doesn't happen in this article?

Thank you.