DEV Community

Cover image for Configs are programs. Hash what they evaluate to.
VX
VX

Posted on Originally published at vznjs.github.io

Configs are programs. Hash what they evaluate to.

Turborepo's turbo.json and Nx's project.json are data. The tools
hash the file and call the config part of the key. That works exactly
as long as the file is the whole story.

A vx.config.ts is a program:

import { defineProject } from '@vzn/vx/config'
import { lib } from '../../vx-preset.ts'

export default defineProject({
  tasks: {
    ...lib({ entry: 'src/index.ts' }),
    docs: {
      exec: { command: `typedoc --out ${process.env.DOCS_OUT ?? 'docs'}` },
      cache: { inputs: { files: ['src/**'] }, outputs: { files: ['docs/**'] } },
    },
  },
})
Enter fullscreen mode Exit fullscreen mode

Hashing this file's bytes would miss every change to vx-preset.ts.
It would also miss the value of DOCS_OUT. Both change what the task
does.

The key sees the object

vx evaluates the config and hashes the resolved task object, the
thing the scheduler is about to act on. Part five of the
key derivation is
xxh3(JSON.stringify(hashableConfig(node.config))) after the plugin
project stage has run — one field wide, dropping exec.remote,
which says where a task runs rather than what it does. Whatever a preset returned,
whatever a template literal expanded to, whatever a plugin added or
removed: all of it is in the key, because all of it is in the object.

Two properties fall out:

  • Presets are safe to share. Edit vx-preset.ts and every task that spread it re-keys, with no globalDependencies list to keep in sync.
  • Plugins are in the key. turbo() from @vzn/vx-migrate fills the project stage from a turbo.json; the resolved tasks it produces are what gets hashed. A plugin cannot change a task's behaviour behind the key's back.

Placement is stripped before hashing. exec.remote says where a task
runs, which is not what it produces. timeout, retries and description are
folded, because a task that was allowed to run longer may have finished
where the shorter one was killed.

Evaluating a program has a cost, so gate the cache

Evaluating a hundred TypeScript files per run is not free, and the
obvious fix, caching the evaluation, is unsound for a program that
reads the environment or the clock. vx caches evaluation results only
where it can prove soundness. Every file in the import closure is
read, with its string literals and comments stripped first (a command
string is not code: node -e "process.exit(0)" is an ordinary task).
If what is left names a global through which an evaluation can observe
something the file bytes do not capture, the config is refused the
cache and evaluated live every run. The list is
process, Bun, globalThis, global, self, fetch, Date,
Temporal, Intl, crypto, performance, navigator, require,
eval, Function, constructor, localeCompare, await, any
toLocale* method, the reflective primitives that reach Function
without naming it (Reflect, getPrototypeOf, setPrototypeOf,
getOwnPropertyNames, getOwnPropertyDescriptor,
getOwnPropertyDescriptors, __proto__, prototype,
__defineGetter__, __defineSetter__, __lookupGetter__,
__lookupSetter__), import.meta, random (as a word, so a
destructured Math.random too), prompt, confirm and alert (they
read the terminal), arguments (a CommonJS config's arguments[1] is
require), Worker (it runs a file outside the hashed closure) and a
dynamic import() — the aliases and the property-name routes to each
(global['proc' + 'ess'], ({}).constructor.constructor) included.
A string literal naming constructor, __proto__ or prototype
evaluates live too. The list stops accidental impurity; a config built
to defeat it can assemble a key at run time.
Five of those names were listed only after a config using them had
been cached as pure. An identifier
escape is the one spelling a name list cannot see, so a backslash in
code position is refused on sight, and a bare import of anything but
@vzn/vx/config (the entry vx init writes) and @vzn/vx's pure
helpers is refused too: a pure closure is relative files.

A config that passes is keyed by the git blob ids of its whole
import closure
— so an edit to the preset invalidates the cached
evaluation of every importer — together with the workspace
fingerprint and the Bun and vx versions that evaluated it. A closure
of more than 32 files evaluates live: a preset tree that big is not
the case this serves.

What the gate buys: load configs is 16–25 ms per 1,000 configs
served from the cache, against ~200 ms of evaluations. The refusal is
what makes having it at all sound.

Freezing the evaluation: vx lock

Sometimes you want the evaluation pinned rather than repeated. vx lock
evaluates every config now and writes the resolved objects plus a
content hash of each file to vx-lock.json; vx run --frozen consumes
the lock with no evaluation at all, and vx lock --check re-evaluates
everything against it and exits non-zero on drift, including drift a
byte hash cannot see: an env value read at eval time, an import that
changed. The CI recipe is vx lock --check && vx run … --frozen. That
gets its own post.

Why not named inputs

Turborepo's globalDependencies and Nx's namedInputs exist because
JSON cannot compose. A vx.config.ts can: a shared input list is a
constant in a file you import, and the resolved-config hash sees the
result. Named inputs, global inputs and global env are on the
repository's rejected list for that reason, and they will stay there.


Originally published on the vx blog. vx is an MIT task runner and build cache for JS monorepos: vznjs.github.io/vx · GitHub.

Written with AI assistance.

Top comments (0)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.