Dive into MobX with Flowik: how auto-tracking works

In the first post I introduced Flowik, a MobX-style state library for Swing and Vaadin. Its pitch fits in one sentence: reading is subscribing. You read count.value inside an autoRun, and the autoRun re-runs when count changes. There is no subscribe() call anywhere.

The first time you see this it looks like magic. It isn't. The whole mechanism is a stack, a couple of sets and a counter, and it fits in about a hundred lines of Kotlin.

In this post we will build a toy version from scratch in three steps: observables with autoRun, then computed, then action. After that we will open Flowik's real Tracking.kt and see what changes when the toy has to survive a real UI. If you already use MobX in JavaScript, this is also a tour of what MobX does under the hood.

The picture in your head: a dependency graph

Every MobX-style app is a graph with three kinds of nodes:

• Observables: the source state. They are mutable and know who reads them.
• Computeds: values derived from other nodes. They are cached, and they are both readers and things being read.
• Reactions (autoRun in Flowik): the leaves, where effects happen, such as updating a label.

flowchart LR
    subgraph S["Observable state"]
        F(["firstName"])
        L(["lastName"])
        A(["age"])
    end
    subgraph D["Derivations"]
        N["fullName<br/><i>computed</i>"]
        R1{{"autoRun:<br/>update header label"}}
        R2{{"autoRun:<br/>update age field"}}
    end
    F --> N
    L --> N
    N --> R1
    A --> R2
    classDef obs fill:#e7f1fb,stroke:#3b78c4,color:#10243e
    classDef comp fill:#efedff,stroke:#6b5bd2,color:#1d1640
    classDef react fill:#e5f4e4,stroke:#3d8c40,color:#12300f
    class F,L,A obs
    class N comp
    class R1,R2 react

Arrows point the way changes flow. Change age and only the age field updates. Change firstName and the header updates, while the age field is left alone.

The interesting part is that you never draw this graph. Nobody writes "fullName depends on firstName". The library discovers each arrow at runtime, by watching what each derivation reads while it runs. Let's build the machinery that does that.

Step 1: who is reading right now?

The whole trick is one piece of global state: a stack of derivations that are currently running. When an observable is read, it looks at the top of that stack and adds whoever is there to its observers.

interface Derivation {
    fun onDependencyChanged()
}

object Tracking {
    private val stack = ArrayDeque<Derivation>()

    val current: Derivation? get() = stack.lastOrNull()

    fun <T> track(derivation: Derivation, block: () -> T): T {
        stack.addLast(derivation)
        try {
            return block()
        } finally {
            stack.removeLast()
        }
    }
}

It is a stack rather than a single variable because derivations nest: a reaction reads a computed, which runs its own function while the reaction is still running.

An observable is a value plus a set of observers. The getter records the current reader. The setter notifies everyone, but only if the value really changed:

class Observable<T>(initial: T) {
    private var _value = initial
    private val observers = mutableSetOf<Derivation>()

    var value: T
        get() {
            Tracking.current?.let { observers += it }   // reading is subscribing
            return _value
        }
        set(newValue) {
            if (newValue == _value) return
            _value = newValue
            observers.toList().forEach { it.onDependencyChanged() }
        }
}

And an autoRun simply runs its effect inside Tracking.track, and runs it again whenever told that something changed:

class AutoRun(private val effect: () -> Unit) : Derivation {
    fun run() = Tracking.track(this, effect)
    override fun onDependencyChanged() = run()
}

fun autoRun(effect: () -> Unit) = AutoRun(effect).also { it.run() }

That's it. Let's try it:

val count = Observable(0)
autoRun { println("count = ${count.value}") }   // count = 0
count.value = 1                                  // count = 1
count.value = 1                                  // nothing: the value did not change

Here is what happened during that first run:

sequenceDiagram
    autonumber
    participant R as autoRun
    participant T as Tracking stack
    participant O as count (observable)
    R->>T: push(this)
    R->>O: read count.value
    O->>T: who is reading? current
    T-->>O: the autoRun
    Note over O: observers += autoRun
    O-->>R: 0
    R->>T: pop()
    Note over O: later: count.value = 1
    O->>R: notify, run() again

Notice that the subscription is a side effect of reading. That is why dependencies can be conditional. In autoRun { if (showDetails.value) println(details.value) }, the reaction only depends on details while showDetails is true. Our toy doesn't handle the other half of that yet, which is forgetting old dependencies. We'll come back to it.

Step 2: computed, the node with two faces

A computed sits in the middle of the graph, so it plays both roles. To its readers it looks like an observable: reading it records the current derivation. To its inputs it looks like a derivation: it evaluates its function inside Tracking.track(this, ...).

class Computed<T>(private val compute: () -> T) : Derivation {
    private var cached: T? = null
    private var dirty = true
    private val observers = mutableSetOf<Derivation>()

    val value: T
        get() {
            Tracking.current?.let { observers += it }   // observable to its readers
            if (dirty) {
                cached = Tracking.track(this, compute) // a derivation to its inputs
                dirty = false
            }
            return cached as T
        }

    override fun onDependencyChanged() {
        if (dirty) return             // already stale, downstream already told
        dirty = true                  // push: mark stale, compute nothing
        observers.toList().forEach { it.onDependencyChanged() }
    }
}

fun <T> computed(compute: () -> T) = Computed(compute)

The key design choice is push invalidation, pull evaluation. When an input changes, the computed does not recompute. It only marks itself dirty and passes the news downstream. The actual work happens later, when someone reads value. If nobody reads it, it never runs.

sequenceDiagram
    participant A as firstName
    participant C as fullName (computed)
    participant R as autoRun
    Note over A: firstName.value = "Bob"
    A->>C: invalidate()
    Note over C: isDirty = true<br/>(nothing recomputed yet)
    C->>R: schedule
    R->>C: read fullName.value
    C->>A: recompute: read firstName
    A-->>C: "Bob"
    C-->>R: "Bob Smith"

val firstName = Observable("Alice")
val lastName = Observable("Smith")
var evaluations = 0
val fullName = computed { evaluations++; "${firstName.value} ${lastName.value}" }

autoRun { println("Hello, ${fullName.value}") }   // Hello, Alice Smith
firstName.value = "Bob"                             // Hello, Bob Smith
println(evaluations)                                // 2

Two evaluations for two distinct states. Add ten more readers of fullName and it still evaluates once per change. That is the caching MobX is known for.

Step 3: action, or "not yet, please"

Our toy has a problem. A user renames a person, so we write two fields:

firstName.value = "Carol"   // Hello, Carol Smith   <- a name that never existed
lastName.value = "Jones"    // Hello, Carol Jones

The reaction ran twice, and the first run showed a half-updated state. In a UI that is wasted work and sometimes a visible flicker.

The fix is a batch: a depth counter and a set of pending reactions. While the depth is above zero, reactions are queued instead of run. When the outermost batch ends, each queued reaction runs once. A LinkedHashSet keeps the order and removes duplicates for free.

object Batch {
    private var depth = 0
    private val pending = linkedSetOf<AutoRun>()

    fun schedule(autoRun: AutoRun) {
        if (depth > 0) pending += autoRun else autoRun.run()
    }

    fun <T> action(block: () -> T): T {
        depth++
        try {
            return block()
        } finally {
            if (--depth == 0) {
                val toRun = pending.toList()
                pending.clear()
                toRun.forEach { it.run() }
            }
        }
    }
}

fun <T> action(block: () -> T): T = Batch.action(block)

The only change elsewhere is one line in AutoRun: instead of running directly, it asks to be scheduled.

override fun onDependencyChanged() = Batch.schedule(this)

Now:

action {
    firstName.value = "Carol"
    lastName.value = "Jones"
}                              // Hello, Carol Jones  (once)

sequenceDiagram
    participant U as action { }
    participant T as Tracking
    participant O as observables
    participant R as autoRun
    U->>T: beginBatch() depth = 1
    U->>O: firstName = "Carol"
    O->>T: schedule(autoRun)
    Note over T: batching: add to pending set
    U->>O: lastName = "Jones"
    O->>T: schedule(autoRun)
    Note over T: already pending, set dedupes
    U->>T: endBatch() depth = 0
    T->>R: flush: run() once

The counter, rather than a boolean, lets actions nest. A store method that is an action can call another action, and nothing flushes until the outermost one ends.

The diamond: why batching is about correctness, not only speed

Here is a shape that appears constantly in real screens: one input, two derived values, one view that shows both.

flowchart LR
    A(["price"]) --> B["net<br/><i>computed</i>"]
    A --> C["tax<br/><i>computed</i>"]
    B --> R{{"autoRun:<br/>show net + tax"}}
    C --> R
    classDef obs fill:#e7f1fb,stroke:#3b78c4,color:#10243e
    classDef comp fill:#efedff,stroke:#6b5bd2,color:#1d1640
    classDef react fill:#e5f4e4,stroke:#3d8c40,color:#12300f
    class A obs
    class B,C comp
    class R react

val price = Observable(100)
val net = computed { price.value }
val tax = computed { price.value / 5 }
autoRun { println("net=${net.value} tax=${tax.value}") }   // net=100 tax=20

price.value = 200
// net=200 tax=20   <- glitch: tax is stale
// net=200 tax=40

Without a batch, price notifies net first. net schedules the autoRun, which runs immediately and reads tax. But tax has not been told about the change yet, so it returns its old cached value. Then tax is notified and the autoRun runs a second time. For a moment the screen showed a net price and a tax that never belonged together. In reactive systems this is called a glitch.

Inside an action the problem disappears. Both computeds are marked dirty before any reaction runs, so the single run sees a consistent world:

action { price.value = 300 }   // net=300 tax=60  (once)

MobX handles this by treating every write as a tiny batch of its own, and by tracking stale and ready counts on each node so that a reaction waits until all its inputs are settled. The practical rule for our toy, and for Flowik today, is simple: change state inside actions. That is good style anyway, because an action names the intent ("rename", "delete") and gives you one place to put it.

From toy to Flowik

Our toy is about 90 lines and already has the right shape. Open Flowik's Tracking.kt and you will recognise it immediately: a stack of Trackers, a batch depth and pending sets. What the real version adds is mostly about surviving a long-lived UI.

Concern	Toy	Flowik
Tracking context	One global stack	A ThreadLocal stack, so each thread has its own context. State is meant to be used from the UI thread.
Stale dependencies	Never forgotten	Before each re-run, AutoRun and Computed unsubscribe from all old dependencies, then re-subscribe to what they actually read
Reaction kinds	autoRun only	autoRun, reaction(supply, effect) and whenThen, each with its own pending queue
Change detection	==	equals by default, plus observableRef (identity), observableStruct (deep) or a custom comparer
Derived values that notify only on real change	Not possible	computedStruct / computedRef, refreshed at the end of a batch before reactions run
Errors	Exception escapes into the writer	Caught per reaction, passed to onError or logged; the reaction stays alive
Lifetime	Lives forever	dispose() stops a reaction and releases its subscriptions

A few details are worth reading in the source.

Batches end even when things throw. endBatch() refreshes derived values inside a try/finally, so a failing derivation can't leave the thread stuck in batching mode, where it would silently swallow every later reaction.

Derived values settle before reactions run. Policy-based computeds such as computedStruct have to evaluate to know whether they changed, and one refresh can invalidate another downstream. So flushRefreshes() loops until nothing is left, with a guard of 100 rounds to report a cycle instead of hanging. Only then are reactions flushed.

untracked is a stack swap. Reading without subscribing needs no special flags on observables. Flowik just replaces the stack with an empty one for the duration of the block:

fun <R> untracked(block: () -> R): R {
    val saved = stack.get()
    stack.set(ArrayDeque())
    return try {
        block()
    } finally {
        stack.set(saved)
    }
}

Errors are reported outside tracking. AutoRun.run() pops itself off the stack before calling the error handler, so whatever the handler reads does not become a dependency of the failed reaction.

Two ways to listen. Observer.kt is just a fun interface with onChange(). It is the explicit subscribe() channel, which the UI bindings and collections use to forward changes, for example from a list element up to its list. Auto-tracking and explicit subscription live side by side, and you rarely need the second one in application code.

Wrapping up

The "magic" of MobX comes down to three ideas:

1. A stack of who is running. Reading an observable subscribes whoever is on top.
2. Push invalidation, pull evaluation. Changes only mark things dirty; work happens when someone reads.
3. Batching. An action delays reactions until the state is consistent, then runs each one once.

Everything else (disposal, error handling, comparison policies, async loading) is engineering around those three. Once you have built the toy yourself, the real source stops being intimidating. Flowik's whole core is under 3,000 lines of Kotlin, and Tracking.kt is about 150 of them.

The complete toy from this post fits in a single file, so paste it into a scratch file and break it. Try making the observable setter wrap its notifications in action { } and watch the diamond glitch disappear.

• Flowik on GitHub: jreznot/flowik
• The gist of MobX