DEV Community: Kristen

How Autotracking Works

Kristen — Fri, 28 Feb 2020 22:31:08 +0000

This blog post is the third in a series on autotracking, the new reactivity system in Ember.js. I also discuss the concept of reactivity in general, and how it manifests in JavaScript.

What Is Reactivity?
What Makes a Good Reactive System?
How Autotracking Works ← This post
Autotracking Case Study - TrackedMap
Autotracking Case Study - @localCopy
Autotracking Case Study - RemoteData
Autotracking Case Study - effect()

In the previous blog post, we discussed a number of reactivity models and extracted a few principles for designing reactive systems:

For a given state, no matter how you arrived at that state, the output of the system is always the same
Usage of state within the system results in reactive derived state
The system minimizes excess work by default
The system prevents inconsistent derived state

In this post, we'll dive into autotracking to see how it works, and how it fulfills these design principles.

Memoization

Last time, we ended on Elm's reactivity model and how (I thought) it used memoization as a method for minimizing excess work. Memoization is a technique where we cache the previous arguments that a function was called with along with the result they produced. If we receive the same arguments again, then we return the previous result.

But it turns out I was wrong about about Elm using it by default. An Elm user helpfully pointed out to me after reading that post that Elm does not memoize by default, but does provide a way to add memoization to components easily when you want to add it. I made my mistake here by taking the original Elm whitepaper for granted, without digging too deeply into the actual state of the framework today.

However, I still think memoization is the best way to understand what autotracking is doing. And it actually turns out that the reason Elm doesn't use it by default relates to the types of problems that autotracking solves quite a lot!

The issue comes down to equality in JavaScript. In JS, objects and arrays are not equal to one another even if they contain exactly the same values.

let object1 = { foo: 'bar' };
let object2 = { foo: 'bar' };

object1 === object2; // false

When memoizing, this presents us with a dilemma - if one of the arguments to your function is an object, how can you tell if any of its values have changed. Recall this example from the last post:

// Basic memoization in JS
let lastArgs;
let lastResult;

function memoizedRender(...args) {
  if (deepEqual(lastArgs, args)) {
    // Args
    return lastResult;
  }

  lastResult = render(...args);
  lastArgs = args;

  return lastResult;
}

In this example, I used a deepEqual function to check the equality of lastArgs and args. This function isn't defined (for brevity) but it would check the equality of every value in the object/array, recursively. This works, but this strategy leads to its own performance issues over time, especially in an Elm-like app where all state is externalized. The arguments to the top level component will get bigger and bigger, and that function will take longer and longer to run.

So, lets assume that's off the table! Are there any other options? Well, if we aren't memoizing based on deep-equality, then the only other option is to memoize based on referential equality. If we're passed the same object as before, then we assume nothing has changed. Let's try this on a simplified example and see what happens.

let state = {
  items: [
    { name: 'Banana' },
    { name: 'Orange' },
  ],
};

const ItemComponent = memoize((itemState) => {
  return `<li>${itemState.name}</li>`;
});

const ListComponent = memoize((state) => {
  let items = state.items.map(item =>
    ItemComponent(item)
  );

  return `<ul>${items.join('')}</ul>`;
});

let output = ListComponent(state);

In this example all we're trying to create is a string of HTML (much simpler than actually updating and maintaining real DOM, but that's a topic for another post). Does memoization based on referential equality help us if all we want to do is change the name of the first item in the list?

For starters, it depends on how we perform this update. We could either:

Create an entirely new state object, or...
Update the part of the state object that changed

Let's try strategy 1. If we blow away the state for every render, and start fresh, then memoization for any object will always fail. So, our ListComponent and ItemComponent functions will both always run again. So clearly, this doesn't work.

What if we try strategy 2? We update only the name property of the first item in the list.

state.items[0].name = 'Strawberry';

let output = ListComponent(state);

This won't work because the state object hasn't changed now, so the ListComponent function will return the same output as last time.

In order for this to work, we would have to update every object and array in the state tree that is a parent of the final, rendered state that has changed, and keep every other node in that tree the same. In a large application, which could have many state changes occur in a single update, this would be incredibly difficult to keep straight, and would almost definitely be as expensive (if not more expensive) than our deepEqual from before.

// This only gets worse in the general case
let [firstItem, restItems] = state.items;

state = {
  ...state,
  items: [
    { ...firstItem, name: 'Strawberry' },
    ...restItems
  ]
};

So that strategy doesn't work either. Even with all of our state externalized, we can't memoize by default - we have to opt-in every time and design a very particular part of the tree to be memoized.

This problem may be solved for Elm-like applications in the future, if TC39 ends up moving forward with Records and Tuples. This would allow value equality to work with object-like and array-like data structures, making this a non-issue for them. But the future there is uncertain (it's only stage 1 at the moment), and it only works for apps that follow the externalized state pattern to the extreme. Otherwise, all we have is referential equality.

But what if we could know which properties were used on that state object when rendering was happening? And what if we could know if one of them changed with very low cost? Would that open up some possibilities?

Enter Autotracking

Autotracking, at its core, is about tracking the values that are used during a computation so we can memoize that computation. We can imagine a world where our memoize function is aware of autotracking. Here's an inventory component that is slightly more complex than the previous example, with autotracking integrated:

class Item {
  @tracked name;

  constructor(name) {
    this.name = name;
  }
}

class State {
  @tracked showItems = true;

  @tracked selectedType = 'Fruits';

  @tracked itemTypes = [
    'Fruits',
    'Vegetables',
  ]

  @tracked fruits = [
    new Item('Banana'),
    new Item('Orange'),
  ];

  @tracked vegetables = [
    new Item('Celery'),
    new Item('Broccoli'),
  ];
}

const OptionComponent = memoize((name) => {
  return `<option>${name}</option>`;
});

const ListItemComponent = memoize((text) => {
  return `<li>${text}</li>`;
});

const InventoryComponent = memoize((state) => {
  if (!state.showItems) return '';

  let { selectedType } = state;

  let typeOptions = state.itemTypes.map(type =>
    OptionComponent(type)
  );

  let items = state[selectedType.toLowerCase()];

  let listItems = items.map(item =>
    ListItemComponent(item.name)
  );

  return `
    <select>${typeOptions.join('')}</select>
    <ul>${listItems.join('')}</ul>
  `;
});

let state = new State();
let output = InventoryComponent(state);

In this world, memoize will track accesses to any tracked properties passed to the function. In addition to comparing the arguments that were passed to it, it will also check to see if any of the tracked properties have changed. This way, when we update the name of an item, each memoized function will know whether or not to rerender.

state.fruits[0].name = 'Strawberry';

// The outer InventoryComponent reruns, and the
// first ListItemComponent reruns, but none of the
// other components rerun.
let output = InventoryComponent(state);

Awesome! We now have a way to memoize deeply by default without doing a deep-equality check. And for the functional programmers out there, this mutation could be handled as part of a reconciliation step (I imagine Elm could compile down to something like this for state changes, under the hood).

But is it performant? To answer that, we need to dig into the guts of autotracking.

Revisions and Tags

The core of autotracking revolves around a single number. This number is the global revision counter.

let CURRENT_REVISION: number = 0;

Another way to think of this is as a global "clock". Except rather than counting time, it counts changes. Whenever something changes in the application, the we increase the clock's value by 1.

So, each value of the clock represents a version of state that the application was in. We were in version 0 at one point, the initial state of the app. Then we changed something, creating version 1 of the state. By incrementing the clock, we are tracking the current version of state.

We can use a clock like this to check for very simple changes. Is the number greater than it was last time we looked? Yes? Alright, something is different, we need to update! But this doesn't help us with our memoization problem. We don't want our memoized functions to rerun whenever the clock changes, because it could have changed for completely unrelated state. We only want to rerun whenever tracked state within the function has changed. For that, we need tags.

Tags represent state within the application. For each unique piece of updatable state that is added to the system, we create a tag and assign it to that state.

Tags take their name from HTTP ETags, which were an optional part of the HTTP spec. ETags were essentially a way for browsers to check if anything had changed on a page before reloading it, and conceptually are very similar.

Tags have a single value, which is a version from the clock. Whenenever we modify the state that tag represents, we dirty the tag. To do this, we increase the value of the clock, and then we assign its new value to the tag.

So the tag essentially stores the last version that this state was updated at. Following the clock metaphor, this was the last point in time the state was updated.

Now for the memoization. As we run our program the first time, and we use every piece of state, we we collect these tags, and save them along with the result of the computation. This is called tag consumption.

We also save the current maximum version of all of the tags we've collected. This represents the most recent version for all of the state we accessed. Nothing has been modified within this computation since that version.

The next time we come back to this computation, we get the maximum version of all the tags again. If any one of them has been dirtied, it will be the most recent version of state. And that version will necessarily be higher than the maximum possible value last time we checked.

So, if the value is higher, then we know that something has changed! We rerun the computation and get the new result.

We can also look at the opposite case - what happens when we update state elsewhere in the application. Like before, we bump the global clock and assign its value to the tag that was updated.

But when we go to check if our memoized function needs to rerun, since we are only checking the values of the tags that were used within it, they will return the same maximum as last time. So our function only reruns when it should, unrelated changes will not affect it.

Fulfilling the Principles

The overhead of this form of memoization is, on its own, pretty low. Listing out the different actions involved:

Tag creation. We create an object with a single property for each piece of mutable root state, the first time that state is created and used.
Consumption. As the function is running, we keep a Set of values and push tags into it.
Dirtying. When we update state, we increase a number (++) and we assign its value once.
Validating. When we finish a computation, we take all of the revisions (Array.map to get them) and then get the maximum value from them (Math.max). When revalidating, we do this again.

Each of these operations is very cheap. They do scale as we add state to the system, but minimally so. In most cases, as long as we aren't adding excessive amounts of state, it will likely be very fast - much faster than rerunning the computations we want to memoize.

So, this system absolutely fulfills principle number 3:

3. The system minimizes excess work by default

But what about the remaining principles? Let's go through them one by one.

Principle 1: Predictable Output

1. For a given state, no matter how you arrived at that state, the output of the system is always the same

To answer this, let's start out with the original ListComponent from the beginning of this post, converted to use @tracked.

class Item {
  @tracked name;

  constructor(name) {
    this.name = name;
  }
}

class State {
  @tracked items = [
    new Item('Banana'),
    new Item('Orange'),
  ];
}

const ItemComponent = memoize((itemState) => {
  return `<li>${itemState.name}</li>`;
});

const ListComponent = memoize((state) => {
  let items = state.items.map(item =>
    ItemComponent(item)
  );

  return `<ul>${items.join('')}</ul>`;
});

let state = new State()
let output = ListComponent(state);

ListComponent is a pure function. It does not modify the state as it is running, so we don't have to worry about unpredictability caused by that. We know that if we don't memoize at all, and we pass a given state object to it, it will always return the same output. So, the question for this example is whether or not the memoization works correctly. Based on the way autotracking works, as long as all properties and values that are mutated are marked with @tracked or have a tag associated with them, it should.

So it works for simple functions that only use arguments and don't mutate any state. What about something a little bit more complex? What if the function had an if statement in it, for instance?

class Item {
  @tracked name;

  constructor(name) {
    this.name = name;
  }
}

class State {
  @tracked showItems = false;

  @tracked items = [
    new Item('Banana'),
    new Item('Orange'),
  ];
}

const ItemComponent = memoize((itemState) => {
  return `<li>${itemState.name}</li>`;
});

const ListComponent = memoize((state) => {
  if (state.showItems) {
    let items = state.items.map(item =>
      ItemComponent(item)
    );

    return `<ul>${items.join('')}</ul>`;
  }

  return '';
});

let state = new State();
let output = ListComponent(state);

In this example we would expect the output to be empty on initial render, since showItems is false. But that also means we never accessed the items array, or the names of the items in it. So if we update one of them, will our output still be consistent?

It turns out it will, since those values didn't affect the result in the first place. If showItems is false, then changes to the rest of the list items shouldn't affect the output - it should always still be an empty string. If showItems changes, however, then it will change the output - and it will consume all of the other tags at that point. The system works out correctly in this case.

So, complex functions with branching and loops work correctly. What about functions that don't just use the arguments passed to them? Many applications also end up using external state in their functions - JavaScript certainly allows that. Does autotracking still ensure predictable output if our function does this? Let's consider another example:

class Locale {
  @tracked currentLocale;

  constructor(locale) {
    this.currentLocale = locale;
  }

  get(message) {
    return this.locales[this.currentLocale][message];
  }

  locales = {
    en: {
      greeting: 'Hello',
    },

    sp: {
      greeting: 'Hola'
    }
  };
}

class Person {
  @tracked firstName;
  @tracked lastName;

  constructor(firstName, lastName) {
    this.firstName = firstName;
    this.lastName = lastName;
  }
}

let locale = new Locale('en');
let liz = new Person('Liz', 'Hewell');

const WelcomeComponent = memoize((person) => {
  return `${locale.get('greeting')}, ${person.firstName}!`;
});

let output = WelcomeComponent(liz);

In this example, we pass a person to the WelcomeComponent to render a greeting. But we also reach out to the local locale variable, which is an instance of the Locale class, used for translating.

What if we changed that language in the future? Would our WelcomeComponent's output properly update, the next time we called it?

The answer is once again yes - the tag associated with currentLocale was properly consumed when we ran it the first time, it doesn't matter that it was external. So, updating it to 'sp' will cause WelcomeComponent to rerender in Spanish, just like if that was the original state. As long as all mutable values that are used within the function are properly tracked, the function will update consistently, no matter where they come from.

Finally, what if the function mutates state as it's running? This one is trickier, and it's really one of the roots of many issues within reactive systems. For instance, let's consider a different version of a ListComponent:

class State {
  @tracked items = [];
}

const ListComponent = memoize((state) => {
  state.items = [...state.items, Math.random()];

  let items = state.items.map(item => `<li>${item}</li>`);

  return `<ul>${items}</ul>`;
});

let state = new State();
let output = ListComponent(state);

It seems like this component undermines our system! Every time this list re-renders, it'll add a new value, incrementing value. And since we memoize at the end of the function, it also means that we'll lock in that value until something else changes the items array. This is very different semantically than what would happen if we hadn't memoized the component.

This is a case where autotracking has a weakness - it is possible to write code that abuses its semantics like this. We could potentially lock down all tracked state and prevent it from changing at all during computation. But there are lots of valuable patterns where updating state - and even more often, creating new state_ - does make sense, so we unfortunately cannot prevent changes altogether. I'll be exploring some of these patterns in future case studies to show exactly what I mean there.

However, most real world use cases don't involve a constantly growing list of items. Let's look at something a bit more realistic.

class State {
  @tracked items = [];
}

const ListComponent = memoize((state) => {
  if (state.items.length === 0) {
    state.items = ['Empty List'];
  }

  let items = state.items.map(item => `<li>${item}</li>`);

  return `<ul>${items}</ul>`;
});

let output = ListComponent(new State());

In this case, we're only pushing into the array if we detect that it is empty. This seems more like something someone would actually write, but definitely has a codesmell. This type of mutation could cause quite a bit of unpredictability, since we won't know the final state of the program until after we run it.

However, in this case autotracking knows this, and prevents us from following this pattern. Autotracking has a rule, meant to help guide users toward more declarative and predictable code - if state has already been read during a computation, it can no longer be mutated. So, this series of statements:

if (state.items.length === 0) {
  state.items = ['Empty List'];
}

Would throw an error! We've just read state.items to get the current state, we can no longer update it during the same computation.

So, autotracking results in predictable output for most reasonable uses, and guides users towards predictable output. We had to go out of our way to get something quirky, and usually autotracking will throw errors if we're doing something bad (though there are still some failure cases).

I think this is pretty good personally! Computed properties in Ember Classic had the same quirks and edge cases along with others (such as depending on values you didn't use in the computation), but with significantly more overhead, both for the computer and for the programmer. And most other reactive systems, such as Rx.js or MobX, can also be abused in similar ways. Even Elm would have it, if it allowed mutations like JavaScript does (just part of the reason they invented a new language).

Principle 2: Entanglement

2. Usage of state within the system results in reactive derived state

Autotracking is entirely consumption based. Tags are added when tracked properties (and other reactive state) are accessed, and only when they are accessed. There is no way to accidentally access a value without adding its tag, so we can't end up in the types of situations that event listeners can cause, where we forgot to register something that should update.

In addition, state dirties its tag when updated, so there's no way we could accidentally forget to notify the system when something has changed. However, we probably want to also do something when we detect a change. Autotracking has this covered as well, via the setOnTagDirtied API:

let currentRender = false;

setOnTagDirtied(() => {
  if (currentRender) return;

  currentRender = setTimeout(() => {
    render();
    currentRender = false;
  });
});

This callback will be called whenever any tracked property is dirtied, and allows us to schedule an update in frameworks. It also does not receive any information about the tag that was dirtied, so it cannot be abused to add event-based patterns back into the system. It is a one way notification that allows us to schedule a revalidation, so our output will always be in sync with the input, and will always update based on usage.

Note: This API is currently called the setPropertyDidChange API in the Glimmer codebase, for historical reasons. I updated the name in this post to make it clearer what it does - it runs for any tag being dirtied, not just properties changing.

Principle 4: Consistent State

4. The system prevents inconsistent derived state

We already discussed how autotracking does allow for updates during computation, and how this can result in some edge cases that are problematic. The biggest issue that can arise is one that we discussed last time - inconsistent output during render. If we update our state halfway through, half of our output could contain the old version, while the other half contains the new version.

We saw how React handled this problem:

class Example extends React.Component {
  state = {
    value: 123;
  };

  render() {
    let part1 = <div>{this.state.value}</div>

    this.setState({ value: 456 });

    let part2 = <div>{this.state.value}</div>

    return (
      <div>
        {part1}
        {part2}
      </div>
    );
  }
}

In this example, setState wouldn't update the state until the next render pass. So, the value would still be 123 in part 2, and everything would be consistent. However, developers always need to keep this in mind when running code - any setState they do won't be applied immediately, so they can't use it to setup initial state, for instance.

Autotracking prevents this inconsistency differently. Like I mentioned before, it knows when you first use a value, and it prevents you from changing it after that first usage.

class Example extends Component {
  @tracked value;

  get derivedProp() {
    let part1 = this.doSomethingWithValue();

    // This will throw an error!
    this.value = 123;

    let part2 = this.doSomethingElseWithValue();

    return [part1, part2];
  }

  // ...
}

If any state has been used during a computation, it can no longer be updated - it is effectively locked in. This guides users to write better, more predictable code, and it also prevents any inconsistency from entering the output of memoized functions. This is a core part of the autotracking design, and one of the main helpers for writing declarative, predictable code within this system.

So, autotracking does fulfill all of the principles! And it does so with an incredibly minimal, low-overhead approach.

An Implementation is Worth a Thousand Words

Autotracking is, in many ways, the core that powers Ember.js and the Glimmer VM. Reactivity is one of the first things a framework has to decide on, because it permeates every decision the framework makes after that. A good reactivity model pays dividends for the entire lifetime of the framework, while a bad one adds debt, bugs, and bloat left and right.

I think I have a bit of a unique perspective on reactivity, since I got to see a framework fundamentally change its model (and even helped to lift the finishing pieces into place). I saw how much complexity and bloat the event-based chains model added under the hood. I've seen many, many bugs resulting from the most subtle tweaks to parts of the codebase. I've fixed a few of those bugs myself. And as an Ember user for the past 7+ years, I also dealt with the knock-on effects of that complexity in my own applications.

By contrast, autotracking is like a breath of fresh air. In part, because it's much more efficient. In part, because its pull-based nature makes it much easier to reason about code. And in part, because the new patterns and restrictions it adds encourage leaner, more consistent code.

But I think more than anything, I love it for its simplicity. And to demonstrate just how simple it is, here is the most minimal implementation of autotracking I could think of:

type Revision = number;

let CURRENT_REVISION: Revision = 0;

//////////

const REVISION = Symbol('REVISION');

class Tag {
  [REVISION] = CURRENT_REVISION;
}

export function createTag() {
  return new Tag();
}

//////////

let onTagDirtied = () => {};

export function setOnTagDirtied(callback: () => void) {
  onTagDirtied = callback;
}

export function dirtyTag(tag: Tag) {
  if (currentComputation.has(tag)) {
    throw new Error('Cannot dirty tag that has been used during a computation');
  }

  tag[REVISION] = ++CURRENT_REVISION;
  onTagDirtied();
}

//////////

let currentComputation: null | Set<Tag> = null;

export function consumeTag(tag: Tag) {
  if (currentComputation !== null) {
    currentComputation.add(tag);
  }
}

function getMax(tags: Tag[]) {
  return Math.max(tags.map(t => t[REVISION]));
}

export function memoizeFunction<T>(fn: () => T): () => T {
  let lastValue: T | undefined;
  let lastRevision: Revision | undefined;
  let lastTags: Tag[] | undefined;

  return () => {
    if (lastTags && getMax(lastTags) === lastRevision) {
      if (currentComputation && lastTags.length > 0) {
        currentComputation.add(...lastTags);
      }

      return lastValue;
    }

    let previousComputation = currentComputation;
    currentComputation = new Set();

    try {
      lastValue = fn();
    } finally {
      lastTags = Array.from(currentComputation);
      lastRevision = getMax(lastTags);

      if (previousComputation && lastTags.length > 0) {
        previousComputation.add(...lastTags)
      }

      currentComputation = previousComputation;
    }

    return lastValue;
  };
}

Just 80 lines of TypeScript, with a few comments for spacing. These are the low level tracking APIs, and are fairly similar to what Ember uses internally today, with a few refinements (and without a few optimizations and legacy features).

We create tags with createTag(), dirty them with dirtyTag(tag), consume them when autotracking with consumeTag(tag), and we create memoized functions with memoizeFunction(). Any memoized function will automatically consume any tags that are consumed with consumeTag() while running.

let tag = createTag();

let memoizedLog = memoizeFunction(() => {
  console.log('ran!');
  consumeTag(tag);
});

memoizedLog(); // logs 'ran!'
memoizedLog(); // nothing is logged

dirtyTag(tag);
memoizedLog(); // logs 'ran!'

The @tracked decorator would be implemented with these APIs like so:

export function tracked(prototype, key, desc) {
  let { initializer } = desc;

  let tags = new WeakMap();
  let values = new WeakMap();

  return {
    get() {
      if (!values.has(this)) {
        values.set(this, initializer.call(this));
        tags.set(this, createTag());
      }

      consumeTag(tags.get(this));

      return values.get(this);
    },

    set(value) {
      values.set(this, value);

      if (!tags.has(this)) {
        tags.set(this, createTag());
      }

      dirtyTag(tags.get(this));
    }
  }
}

And there are many other ways that they can be used to instrument state. We'll see one of these next time, when we dig into creating a TrackedMap class like the one provided by tracked-built-ins.

The core team expects to make these APIs publicly available in the near future, and while they may end up being a little different, this is the rough shape of what they'll look like. As such, I'll be using these APIs for future posts and examples. Don't worry about remembering them though, I'll re-explain them when I do!

Some notes on this implementation:

We use a symbol here to store the revision on Tag because it should be an opaque detail, not accessible to users normally. It's only for the autotracking system. Same reason for the createTag function - right now we return an instance of the Tag class, but that could be optimized in the future.
memoizeFunction does not take a function that receives arguments, unlike the memoize I used in earlier examples. Instead, it only focuses on memoizing based on autotracking/tags. This is because memoizing based on arguments actually becomes problematic at scale - you may end up holding onto cached values for quite a long time, bloating memory usage. The memoize shown in the code samples above could be implemented using this lower level API.

A Note on Vector Clocks

There's another reason why I called the global counter a "clock". In concurrent programming, there is a concept known as a vector clock, which is used for keeping track of changes to state. Vector clocks are usually used in distributed systems - on multiple machines that need to constantly synchronize their state.

Like our clock, vector clocks constantly "tick" forward as state changes, and check current values against previous values to see if things are in sync. Unlike our clock, there are more than one in a given system!

Currently we don't have to deal with this, which is nice, but in the future we actually might need to - with web workers and service workers for instance. Once you have more than one process, a single global clock no longer works on its own.

That's a ways out at the moment, but I'm excited to start exploring it when things calm down a bit. I had my start with distributed programming when I worked at Ticketfly, building a peer-to-peer ticket scanning system and it was some the most fun work I ever did.

Conclusion

As I've said before, autotracking is, to me, the most exciting feature that shipped in Ember Octane. It's not every day that a framework completely rethinks it reactivity model, and I can't think of one that did and was able to do so seamlessly, without any breaking changes.

Personally, I think that the next wave of Ember applications are going to be faster, less error prone, and easier to understand thanks to autotracking. I also think that Ember app's are just going to be a lot more fun to write 😄

I hope you enjoyed this deep dive, and I can't wait to see what the Ember community builds with this new reactive core. In the coming weeks, I'll start working through various use cases, and how to solve them with autotracking techniques, in a case studies series. If you have something you'd like to see solved, let me know!

(This blog post was originally published on pzuraq.com)

What Makes a Good Reactive System?

Kristen — Tue, 11 Feb 2020 05:05:09 +0000

This blog post is the second in a series on autotracking, the new reactivity system in Ember.js. I also discuss the concept of reactivity in general, and how it manifests in JavaScript.

What Is Reactivity?
What Makes a Good Reactive System? ← This post
How Does Autotracking Work?
Case Study - TrackedMap
Case Study - @localCopy
Case Study - RemoteData
Case Study - effect()

In the previous blog post, we discussed what it means for a system to be reactive. The definition I landed on for the purposes of this series was:

Reactivity: A declarative programming model that updates automatically based on changes to state.

I tweaked this slightly since last time so it reads better, but it's effectively the same. In this post, I'll discuss another aspect of reactivity in general: What makes a good reactive system?

Rather than trying to define this in a bubble, I'll start by taking a look at the reactivity of a few other languages and frameworks. From these case studies, I'll try to extract a few principles of good reactive design. This will, I think, both help to keep things grounded, and show a variety of different ways to accomplish the same fundamental goal. As I said in the first post of this series, there are many different ways to do reactivity, each with its own pros and cons.

I also want to say up front that I am not an expert in all of the technologies we'll be taking a look at. My understanding of them is mostly based on research I've done during my work on autotracking, to better understand reactivity as a whole. So, I may get a few things wrong and miss details here and there! Please let me know if you see something that's a little off (or completely backwards 😬).

HTML

In the last post, I used HTML as an example of a fully declarative language. Before we dive into some frameworks, I wanted to expand on that a little bit more, and also discuss the language's built-in reactivity model. That's right, HTML (along with CSS) actually is reactive on its own, without any JavaScript!

First off, what makes HTML declarative? And why is it so good at being a declarative language? Let's consider a sample of HTML for a login page:

<form action="/my-handling-form-page" method="post">
  <label>
    Email:
    <input type="email" />
  </label>

  <label>
    Password:
    <input type="password" />
  </label>

  <button type="submit">Log in</button>
</form>

This sample describes the structure of a form to the browser. The browser then takes it, and renders the fully functional form directly to the user. No extra setup steps are necessary - we don't need to tell the browser what order to append the elements in, or to add the handler for the button to submit the form, or any extra logic. We're telling the browser what the login form should look like, not how to render it.

This is the core of declarative programming: we describe what output we want, not how we want it made. HTML is good at being declarative specifically because it is very restricted - we actually can't add any extra steps to rendering without adding a different language (JavaScript). But if that's the case, how can HTML be reactive? Reactivity requires state, and changes to state, so how can HTML have that?

The answer is through interactive HTML elements, such as input and select. The browser automatically wires these up to be interactive and update their own state by changing the values of their attributes. We can use this ability to create many different types of components, like say, a dropdown menu.

<style>
  input[type='checkbox'] + ul {
    display: none;
  }

  input[type='checkbox']:checked + ul {
    display: inherit;
  }
</style>

<nav>
  <ul>
    <li>
      <label for="dropdown">Dropdown</label>
      <input id="dropdown" type="checkbox" />
      <ul>
        <li>Item 1</li>
        <li>Item 2</li>
      </ul>
    </li>
  </ul>
</nav>

My favorite example of these features taken to the extreme is Estelle Weyl's excellent Do You Know CSS presentation. See the ./index.html example for a pure HTML/CSS slideshow, with some stunning examples of the platform's native features.

In this model for reactivity, every user interaction maps directly onto a change in the HTML (e.g. the checked attribute being toggled on checkboxes). That newly modified HTML then renders, exactly as it would have if that had been the initial state. This is an important aspect of any declarative system, and the first principle of reactivity we'll extract:

1. For a given state, no matter how you arrived at that state, the output of the system is always the same

Whether we arrived at a page with the checkbox already checked, or we updated it ourselves, the HTML will render the same either way in the browser. It will not look different after we've toggled the checkbox 10 times, and it will not look different if we started the page in a different state.

This model for reactivity is great in the small-to-medium use cases. For many applications, though, it becomes limiting at some point. This is when JS comes into play.

Push-Based Reactivity

One of the most fundamental types of reactivity is push-based reactivity. Push-based reactivity propagate changes in state when they occur, usually via events. This model will be familiar to anyone who has written much JavaScript, since events are pretty fundamental to the browser.

Events on their own are not particularly very declarative, though. They depend on each layer manually propagating the change, which means there are lots of small, imperative steps where things can go wrong. For instance, consider this custom <edit-word> web component:

customElements.define('edit-word',
  class extends HTMLElement {
    constructor() {
      super();

      const shadowRoot = this.attachShadow({mode: 'open'});
      this.form = document.createElement('form');
      this.input = document.createElement('input');
      this.span = document.createElement('span');

      shadowRoot.appendChild(this.form);
      shadowRoot.appendChild(this.span);

      this.isEditing = false;
      this.input.value = this.textContent;

      this.form.appendChild(this.input);

      this.addEventListener('click', () => {
        this.isEditing = true;
        this.updateDisplay();
      });

      this.form.addEventListener('submit', e => {
        this.isEditing = false;
        this.updateDisplay();
        e.preventDefault();
      });

      this.input.addEventListener('blur', () => {
        this.isEditing = false;
        this.updateDisplay();
      });

      this.updateDisplay()
    }

    updateDisplay() {
      if (this.isEditing) {
        this.span.style.display = 'none';
        this.form.style.display = 'inline-block';
        this.input.focus();
        this.input.setSelectionRange(0, this.input.value.length)
      } else {
        this.span.style.display = 'inline-block';
        this.form.style.display = 'none';
        this.span.textContent = this.input.value;
        this.input.style.width = this.span.clientWidth + 'px';
      }
    }
  }
);

This web component allows users to click on some text to edit it. When clicked, it toggles the isEditing state, and then runs the updateDisplay method to hide the span and show the editing form. When submitted or blurred, it toggles it back. And importantly, each event handler has to manually call updateDisplay to propagate that change.

Logically, the state of the UI elements is derived state and the isEditing variable is root state. But because events only give us the ability to run imperative commands, we have to manually sync them. This brings us to our second general principle for good reactivity:

2. Usage of state within the system results in reactive derived state

In an ideal reactive system, using the isEditing state would automatically lead to the system picking up updates as it changed. This can be done in many different ways, as we'll see momentarily, but it's core to ensuring that our reactivity is always updating all derived state.

Standard events don't give us this property on their own, but there are push-based reactive systems that do.

Ember Classic

Ember Classic was heavily push-based in nature, under the hood. Observers and event listeners were the primitives that the system was built on, and they had the same issues as the browser's built in eventing system. On the other hand, the binding system, which eventually became the dependency chain system, was more declarative.

We can see this system in action with the classic fullName example:

import { computed, set } from '@ember/object';

class Person {
  firstName = 'Liz';
  lastName = 'Hewell';

  @computed('firstName', 'lastName')
  get fullName() {
    return `${this.firstName} ${this.lastName}`;
  }
}

let liz = new Person();

console.log(liz.fullName); 'Liz Hewell';

set(liz, 'firstName', 'Elizabeth');

console.log(liz.fullName); 'Elizabeth Hewell';

Under the hood in Classic Ember, this system worked via property notifications. Whenever we used a computed property, template, or observer for the first time, Ember would setup dependency chains through to all of its dependencies. Then, when we updated the property with set(), it would notify those dependencies.

Observers would run eagerly of course, but computed properties and templates would only update when used. This is what made them so much better than observers, in the end - they fulfilled the second principle of reactivity we just defined. Derived state (computeds and templates) became reactive when used, automatically.

This was the core of Ember's reactivity for a very long time, and drove most of the ecosystem as observers fell out of common usage. It was not without its weaknesses though. In particular, it was a very object-oriented system. It essentially required defining objects and classes in order to setup dependency chains, pushing developers in this direction. Object-Oriented Programming (OOP) is not a bad thing, but it can definitely be restrictive if it's the only programming model available.

Also, while computed properties were better for performance than observers and event listeners on average, dependency chains and event notifications were still costly . Setting up the dependency system had to be done on startup, and every property change produced events that flowed throughout the entire system. While this was good, it could still have been better.

Observables, Streams, and Rx.js

Another take on the push-based model that makes things more declarative is the Observable model. It was popularized in JavaScript by RxJS, and is used by Angular as the foundation for its reactivity.

This model organizes events into streams, which are kind of like a lazy-array of events. Every time you push an event into one end of the stream, it will get passed along through various transformations until it reaches subscribers at the other end.

// Plain JS
let count = 0;
document.addEventListener(
  'click',
  () => console.log(`Clicked ${++count} times`)
);

// With Streams
import { fromEvent } from 'rxjs';
import { scan } from 'rxjs/operators';

fromEvent(document, 'click')
  .pipe(scan(count => count + 1, 0))
  .subscribe(count => console.log(`Clicked ${count} times`));

This may seem similar to Ember's observers on the surface, but they have a key difference - they are passed the values that they are observing directly, and return new values based on them. This means that they fulfill the second principle of good reactivity, because derived state is necessarily reactive.

The downside with streams is that they are by default always eager. Whenever an event is fired on one end, it immediately triggers all of the transformations that are observing that stream. By default, we do a lot of work for every single state change.

There are techniques to lower this cost, such as debouncing, but they require the user to actively be thinking about the flow of state. And this brings us to our third principle:

3. The system minimizes excess work by default

If we update two values in response to a single event, we shouldn't rerender twice. If we update a dependency of a computed property, but never actually use that property, we shouldn't rerun its code eagerly. In general, if we can avoid work, we should, and good reactivity should be designed to help us do this.

Push-based reactivity, unfortunately, can only take us so far in this regard. Even if we use it to model lazy systems, like Ember Classic's computed properties, we still end up doing a lot of work for each and every change. This is because, at its core, push-based systems are about propogating changes when the change occurs.

On the other end of the spectrum, there are reactive systems that propogate changes when the system updates. This is pull-based reactivity.

Pull-Based Reactivity

I find the easiest way to explain pull-based reactivity is with a thought experiment. Let's say we had an incredibly fast computer, one that could render our application almost instantaneously. Instead of trying to keep everything in sync manually, we could rerender the whole app every time something changed and start fresh. We wouldn't have to worry about propagating changes through the app when they occured, because those changes would be picked up as we rerendered everything.

This is, with some hand-waving, how pull-based models work. And of course, the downside here is performance. We don't have infinitely powerful computers, and we can't rerender entire applications for every change on laptops and smart phones.

To get around this, every pull-based reactivity model has some tricks to lower that update cost. For instance, the "virtual DOM".

React and Virtual DOM

The virtual DOM is probably one of the most famous features of React.js, and was one of the original keys to their success. The concept takes advantage of the fact that adding HTML to the browser is the most expensive part. Instead of doing this directly, the app creates a model that represents the HTML, and React translates the parts that changed into actual HTML.

On initial render, this ends up being all the HTML in the app. But on rerenders, only the parts that have changed are updated. This minimizes one of the most expensive portions of a frontend application.

The second way that React's reactivity model optimizes is by only rerunning the part that something has definitely changed in. This is partially what the setState API (and the setter from the useState hook) are about.

class Toggle extends React.Component {
  state = { isToggleOn: true };

  handleClick = () => {
    this.setState(state => ({
      isToggleOn: !state.isToggleOn
    }));
  }

  render() {
    return (
      <button onClick={this.handleClick}>
        {this.state.isToggleOn ? 'ON' : 'OFF'}
      </button>
    );
  }
}

When a user changes state via one of these, only that component (and its subcomponents) are rerendered during the next pass.

One interesting choice here that was made to maintain consistency is that setState and useState do not update immediately when called. Instead, they wait for the next render to update, since logically the new state is new input to the app (and necessitates another rerender). This is counter-intuitive to many users at first before they learn React, but it actually brings us to our final principle of good reactivity:

4. The system prevents inconsistent derived state

React takes a strong stance here precisely because they can't know if you've already used state somewhere else. Imagine if in a React component we could change the state midway through render:

class Example extends React.Component {
  state = {
    value: 123;
  };

  render() {
    let part1 = <div>{this.state.value}</div>

    this.setState({ value: 456 });

    let part2 = <div>{this.state.value}</div>

    return (
      <div>
        {part1}
        {part2}
      </div>
    );
  }
}

If the state change were applied immediately, it would result in part1 of the component's template seeing the state before the change, and part2 seeing it after. While sometimes this may be the behavior the user wanted, it oftentimes comes from deeper inconsistencies that lead to bugs. For instance, you could render a user's email in one part of the app, only to update it and render a completely different email in another part. React is pre-emptively preventing that inconsistency from appearing, but at a higher mental cost to the developer.

Overall, React's two-pronged approach to reactivity is fairly performant up to a certain point, but definitely has its limitations. This is why APIs like shouldComponentUpdate() and useMemo() exist, as they allow React users to manually optimize their applications even further.

These APIs work, but they also move the system overall toward a less declarative approach. If users are manually adding code to optimize their applications, there are plenty of opportunities for them to get it just slightly wrong.

Vue: A Hybrid Approach

Vue is also a virtual DOM based framework, but it has an extra trick up its sleeve. Vue includes a reactive data property on every component:

const vm = new Vue({
  data: {
    a: 1
  }
});

This property is what Vue uses instead of setState or useState (at least for the current API), and it's particularly special. Values on the data object are subscribed to, when accessed, and trigger events for those subscriptions when updated. Under the hood, this is done using observables.

For instance, in this component example:

const vm = new Vue({
  el: '#example',

  data: {
    message: 'Hello'
  },

  computed: {
    reversedMessage() {
      return this.message.split('').reverse().join('')
    }
  }
})

The reversedMessage property will automatically subscribe to the changes of message when it runs, and any future changes to the message property will update it.

This hybrid approach allows Vue to be more performant by default than React, since various calculations can automatically cache themselves. It also means that memoization on its own is more declarative, since users don't have to add any manual steps to determine if they should update. But, it is still ultimately push-based under the hood, and so it has the extra cost associated with push-based reactivity.

Elm

The final reactivity model I want to discuss in this post isn't actually a JavaScript-based model. To me, though, it's conceptually the most similar to autotracking in a number of ways, particularly its simplicity.

Elm is a programming language that made a splash in the functional programming community in the last few years. It's a language designed around reactivity, and built specifically for the browser (it compiles down to HTML + JS). It is also a pure functional language, in that it doesn't allow any kind of imperative code at all.

As such, Elm follows the pure-functional reactivity model I discussed in my last post. All of the state in the application is fully externalized, and for every change, Elm reruns the application function to produce new output.

Because of this, Elm can take advantage of caching technique known as memoization. As the application function is running, it breaks the model down into smaller chunks for each sub-function, which are essentially components. If the arguments to that function/component have not changed, then it uses the last result instead.

// Basic memoization in JS
let lastArgs;
let lastResult;

function memoizedRender(...args) {
  if (deepEqual(lastArgs, args)) {
    // Args
    return lastResult;
  }

  lastResult = render(...args);
  lastArgs = args;

  return lastResult;
}

Because the function is "pure", and the arguments passed to it are the same, there is no chance anything changed, so Elm can skip it entirely.

This is a massive win for performance. Unnecessary work is completely minimized, since the code to produce the new HTML isn't even run, unlike React/Vue/other Virtual DOM based frameworks.

The catch is that in order to benefit from this, you have to learn a new language. And while there are many potential upsides to learning Elm, and it is a beautiful language, it's not always practical to switch to something less well-known and widely used.

Likewise, attempting to bring Elm's pure-functional approach to JavaScript usually has varying degrees of success. JavaScript is, for better or worse, a multi-paradigm language. The model of externalizing all state also has issues, from lots of overhead conceptually to issues with scale. Redux is a library built around this concept, but even leaders in that community don't always recommend it for those reasons.

What we really want are the benefits of memoization, but with the ability to store our state within the function - on components, near where it is used. And we want to fulfill all the other principles we've discussed as well.

But that's a topic for the next post!

Conclusion

So, in this post we looked at a number of different reactivity models, including:

HTML/CSS
Push-based reactivity
- Vanilla JavaScript
- Ember Classic
- Observables/Rx.js
Pull-based reactivity
- React.js
- Vue.js
- Elm

We also extracted a few general principles for designing a good reactive system:

For a given state, no matter how you arrived at that state, the output of the system is always the same
Usage of state within the system results in reactive derived state
The system minimizes excess work by default
The system prevents inconsistent derived state

I don't think this list is necessarily comprehensive, but it covers a lot of what makes reactive systems solid and usable. In the next post, we'll dive into autotracking and find out how it accomplishes these goals.

(This blog post was originally published on pzuraq.com)

What Is Reactivity

Kristen — Tue, 04 Feb 2020 23:17:05 +0000

Ember Octane has landed along with a large number of new features, but none of these features is more exciting to me personally than autotracking. Autotracking is Ember's new reactivity system, which is what allows Ember to know when stateful values (such as @tracked properties) have changed. This was a massive update under the hood, and involved completely rewriting some of Ember's oldest abstractions on top of this new core.

Autotracking is, at first glance, very similar to Ember Classic's reactivity model (based on computed properties, observers, and Ember.set()). If you compare the two side by side, the main difference seems to be where the annotations are placed. In Octane you decorate the properties that you depend on, while in Classic you decorate the getters and setters.

class OctaneGreeter {
  @tracked name = 'Liz';

  get greeting() {
    return `Hi, ${this.name}!`;
  }
}

class ClassicGreeter {
  name = 'Liz';

  @computed('name')
  get greeting() {
    return `Hi, ${this.name}!`;
  }
}

But if you were to dig deeper you'd find that autotracking is quite a lot more flexible and powerful. For instance, you can autotrack the results of functions, which was something that was impossible in Ember Classic. Take this model for a simple 2d video game, for example:

class Tile {
  @tracked type;

  constructor(type) {
    this.type = type;
  }
}

class Character {
  @tracked x = 0;
  @tracked y = 0;
}

class WorldMap {
  @tracked character = new Character();

  tiles = [
    [new Tile('hills'), new Tile('beach'), new Tile('ocean')],
    [new Tile('hills'), new Tile('beach'), new Tile('ocean')]
    [new Tile('beach'), new Tile('ocean'), new Tile('reef')],
  ];



  getType(x, y) {
    return this.tiles[x][y].type;
  }

  get isSwimming() {
    let { x, y } = this.character;

    return this.getType(x, y) === 'ocean';
  }
}

We can see that the isSwimming getter would update whenever character changed position (x/y coordinates). But it would also update whenever the tile that getType returned updated, keeping the system in sync . This type of dynamism popped up from time to time when working with CPs, without any great solutions. It usually required adding many intermediate values to calculate the derived state .

The point being, autotracking is pretty fundamentally different from Ember Classic. This is great, and exciting! But it's also a bit confusing, because many of the patterns that Ember developers have learned over the years don't work as well in Octane.

This is why I've decided to start a new blog post series, Thinking With Autotracking, discussing the design behind autotracking as a system. This series will show how to use autotracking in different ways for a variety of use cases. I'll say up front that these blog posts are not meant to be comprehensive. The goal is to setup the mental model for how autotracking works under the hood, and how to build patterns and libraries that work with it.

So far I have 7 posts planned. The first few will be introductory posts discussing the "big picture", and the rest will be case studies dealing with the nitty-gritty:

What Is Reactivity? <-- This post
What Makes a Good Reactive System?
How Does Autotracking Work?
Case Study - TrackedMap
Case Study - @localCopy
Case Study - RemoteData
Case Study - effect()

As we go along I'm hoping to add more case studies for interesting or difficult cases that new Octane users encounter. So, if you have anything that you'd like me to dig into as part of this series, let me know! You can reach out to me via email or on Discord.

Now then, let's get on to our first topic: What is "reactivity"?

Reactivity in Plain English

Reactivity, or "reactive programming", is one of those buzz words that has been thrown around a lot in the past decade or so. It has been applied to many different contexts, and is a very broad concept, so it can be hard to try to distill down exactly what it means. But, I'm going to try 😄

Reactivity: A declarative programming model for updating based on changes to state.

Note that I'm defining the term "reactivity" directly, rather than "reactive programming" or "reactivity model". In common conversation, I find that we tend to refer to "Ember's reactivity" or "Vue's reactivity", so I think it makes sense as a noun itself in the context of programming. In the end, all these terms essentially mean the same thing.

So, we have a definition that fits into a sentence! So far so good, I think we're beating Wikipedia here.

But now we have two more terms that our definition is based on that are possibly equally hazy to many: "declarative" and "state". What do we mean by "declarative programming", and how does it differ from other styles such as "imperative" and "functional" programming? And what exactly is "state"? We talk a lot about it in the programming world, but it can be hard to find a down-to-earth definition, so let's dig in.

State

"State" is in a lot of ways a fancy term for "variables". When we refer to the state of the program, we refer to any of the values that can change within it; that is, any values that aren't static. In JavaScript, state exists as:

Variables declared with let and const
Properties on objects
Values within arrays or other collections, such as maps and sets

These are forms of what I'll call root state, meaning that they represent real, concrete values in the system. By contrast, there is also derived state, which is state that is produced by combining root state. For instance:

let a = 1;
let b = 2;

function aPlusB() {
  return a + b;
}

In this example, aPlusB() returns a new value that is derived from the value of a and b. Calling the function does not introduce any local variables or values, so it does not have any of its own root state. This is an important distinction, because it means that if we know the value of a and the value of b, then we also know the value of aPlusB(). We also know that if a or b changes, then aPlusB() will also change, which is crucial for building a reactive system.

Declarative Programming

You may have heard the terms "imperative", "declarative", and "functional" tossed around before when talking about various styles of programming, but it can be difficult to know exactly what the difference is between them. Fundamentally, it comes down to this distinction:

In declarative programming, the programmer describes what they want to happen, without worrying about the details of how it is done.

In imperative programming, the programmer describes the exact steps for how something should be done.

"Imperative" means to give a command, and in imperative programming, the application runs as a series of steps (commands). This is a very broad definition and doesn't help much - aren't most programs a "series of steps"? Where it matters though is what it implies about imperatively derived state, specifically that it is not actually derived state (at least, as defined in the previous section).

Let's take a look at the first example again, and derive the value of aPlusB imperatively. Rather than defining a function that can be called to get the value at any time, we'll instead derive the value manually in a few steps:

Create a new variable (a new root state).
Assign it to the value of a + b.

let a = 1;
let b = 2;

let aPlusB = a + b;

At first, this example may appear to be much simpler than the previous one, as we're only assigning one more variable in one more step. But the complexity comes when we want to update a or b. Since aPlusB is its own root state, we must now update it as well:

let a = 1;
let b = 2;

let aPlusB = a + b;

// update `a`
a = 4;
aPlusB = a + b;

// update `b`
b = 7;
aPlusB = a + b;

Because we created a new root state imperatively, we must now always keep that root state in sync with the other state imperatively as well. Every time you command the computer to update a, you must also command it to update aPlusB. For a more concrete example, you could imagine a component API that forced you to manually rerender whenever you changed a value:

export default class Counter extends Component {
 count = 0;

  @action
    incrementCount() {
    this.count++;
    this.rerender();
  }
}

This would be a lot to remember, and could easily introduce bugs and errors if rerender() was called in the wrong way, or at the wrong time, or not called at all. The core issue here is that both of these designs require a command to tell them to update each time something changes.

By contrast, "declarative" programming does not force the user to manually sync things up every time. Let's consider the original example for aPlusB() again.

let a = 1;
let b = 2;

function aPlusB() {
  return a + b;
}

In this design, instead of assigning the value to a new variable which we have to update, we create a function that derives the value. Effectively, we are describing the "how" once, so that we never have to do it again. Instead, everywhere else in our code, we can call aPlusB() and know that we will get the right result. We can declare the values that we want to use, and then declare how they should interact with their surroundings.

This is what makes templates in frameworks like Ember, Vue, and Svelte so powerful, and (if perhaps to a lesser extent) what makes JSX feel better than manually returning the function calls that JSX compiles down to. HTML is fundamentally a declarative-only programming language - you can't tell the browser how to render, you can only tell it what to render. By extending HTML, templates are in turn extending that declarative paradigm, and modern frameworks use that as the basis for their reactivity. The "what" that programmers are describing is, in the end, DOM, and frameworks figure out how to translate your app's state into that DOM, without you ever needing to worry about it.

What About FP?

There are many different ways to accomplish declarative programming. You can use "streams" and "agents" and "actors", you can use objects or functions or a mixture of both. You can create your own language, such as HTML, that is purely declarative, and many UI systems have done this in the past because of how well it works in general.

One such way is functional programming (or FP), which has been very much in vogue recently for a variety of reasons. In what is known as "pure" functional programming, no new root state is ever introduced or changed when calculating a value - everything is a direct result of its inputs.

let a = 1;
let b = 2;

function plus(first, second) {
  return first + second;
}

// to get a + b;
plus(a, b);

In the most extreme form of this model, state for the entire application is fully externalized and then fed into the main function for the app. When state is updated, you run the function again with the new state, generating the new output.

This may seem like an expensive strategy, but there are ways to reduce the cost (we'll get more into that in the next post). What's important here is that pure FP is necessarily declarative, because everything boils down to a function that receives an input and produces an output, without making any changes. This means there's no wiggle room for imperative steps that can cause things to be out of sync.

This is one of the reasons why people like functional programming. It is a very strict form of programming, but it has a lot of benefits that come with that strictness, declarative-ness being one of them.

In the end, though, declarative programming is not limited to functional programming, and so reactivity as a whole is not either. Most reactivity systems end up being a blend of abstractions and paradigms, with some leaning more toward a "pure" FP style and others leaning into Object-Oriented Programming on some level (for instance, actor-based systems ultimately store state per-process, which isn't all that different from OOP. Check out the history of the term "message-passing" to see some more similarities!). What every reactive system has in common though is declarative-ness.

Summing Up

So, in this post we learned:

"Reactivity" roughly means "a declarative programming model for updating based on changes to state". This definition probably wouldn't pass academic muster, but it's the definition I'm going with for this series at least!
"State" means all of the values that can change in a program. There are two types of state (again, for the purposes of this series):
- Root state, which are actual values stored in variables, in arrays, or on objects directly.
- Derived state, which are values that are derived from root state via getters, functions, templates, or other means.
"Declarative programming" means a style in which the programmer describes what they want their output to be, without describing how exactly to do it. HTML is an example of a purely declarative programming language, and functional programming is a style that ensures code is declarative. There are many ways to write declarative programs, libraries, and frameworks using every programming paradigm.

Next time we'll dig into a number of different reactivity models at a high level, and discuss what the properties of a good reactive system are. We'll discuss, in particular, how it is possible to make reactive systems which incorporate object-oriented and imperative code, while still fundamentally being declarative. And we'll discuss where the pitfalls of those styles are, and how user's can "break" declarative-ness in some cases.

(This blog post was originally published on pzuraq.com)