Sentinel: The Semantic Model

A UUID is the obvious primary key. It is opaque, globally unique, immune to renames, and every tutorial reaches for it. It is also the wrong choice here, and the reason has nothing to do with database design.

boiler.ch1.flow-temp shows up in a NATS subject, a Postgres row, a rule definition, a log line and a stack trace. Every one of those is a place a person reads under time pressure. f47ac10b-58cc-4372-a567-0e02b2c3d479 is correct in all five and useful in none of them: it moves the question "which point is this?" one join away, and one join away is far enough that nobody makes the trip at eleven at night.

So the identifier is a dotted slug, and it is the primary key rather than a label hanging off one. That choice has a price, and it is paid at rename time and nowhere else. The rest of this document is the same kind of trade made about twenty more times - what a point is, what it is allowed to mean, and what the system is required to admit it does not know.

What this document owns: point identity and hierarchy, the descriptor fields, the five closed vocabularies, units and conversion, the seven quality states and their legal transitions, the three timestamps, and the observation envelope. Everything downstream reads the registry defined here - the log schema, the watchdog, the rules engine and the command plane all resolve against it. It is the first build step for the same reason it is the first document: nothing later can correct a semantic model that was wrong, only migrate away from it.

Identity

point_id is a dotted path: {device}.{component}.{point}, each segment [a-z0-9-]+. This maps 1:1 onto NATS subjects, survives being pasted into logs and grep, and reads correctly in a stack trace.

The slug is the primary key in Postgres, not a UUID with the slug as a label. Stability comes from the ID being assigned at registration and never derived from anything the network can change. A rename is a deliberate migration event, and it should feel like one.

The registry holds addresses separately: MQTT topic, Modbus unit ID and register, BLE MAC, UniFi site path. These change freely without touching identity.

Hierarchy

Device → Component → Point, where Component may be trivial (main) so a one-sensor device does not pay ceremony for structure it does not have.

Component earns its place because it is the unit of independent failure and hot-swap: the four channels of a multi-channel relay, a sensor behind one leg of an I²C multiplexer. When the multiplexer leg goes dead, one component is marked unavailable, not the device.

Points are scalar. If a device emits a struct, the driver explodes it into points. No exceptions — the moment a point can hold a composite, every downstream consumer has to understand payload shapes, which is the coupling this model exists to prevent.

Point descriptor

The declarative half, living in the registry:

id                 boiler.ch1.flow-temp
device_id          boiler
component_id       ch1
direction          in | out | inout
kind               measurement | state | setpoint | command | diagnostic
value_type         float | int | bool | enum | string

quantity           temperature
unit_native        Cel                    # UCUM
unit_canonical     Cel                    # derived from quantity
subject            water
subject_ref        zone:upstairs-ch       # optional, nullable
modifiers          { aggregation: instant }
tags               [ heating, primary-loop ]

valid_range        { min: -10, max: 120 }
precision          1
change_policy      absolute(0.2)
freshness          periodic(30s, grace=15s)
retention_class    measurement            # defaults from kind

# out / inout only
init_strategy      read_on_connect | restore_from_log | assume(v) | assert(v)
confirmation_mode  inline | async_event | poll_verify(500ms) | optimistic
default_value      false                  # optional
require_intent     false
interlock_protected false                 # derived at rule load, not authored

kind and direction are not redundant. kind describes what the value means; direction describes flow. A setpoint is inout; a command is out with no retained value; a diagnostic is in but must never appear in a measurement dashboard or share a freshness policy with real sensors.

default_value and require_intent are the point's half of the arbitration model in the command plane. default_value is optional; when present, the registry materialises a non-expiring claim at the lowest band, which is the resting state the point falls back to when every other claim is released. When absent, releasing the last claim leaves the point alone rather than driving it anywhere.

require_intent: true says the point must never be without a resolved intent, and registry load fails if it has no default_value. interlock_protected is set at rule load rather than authored, and implies require_intent. Between them, a protective point is one that cannot accidentally be left with nothing asserting a state.

Retention class

retention_class is a closed vocabulary — measurement, state, diagnostic — and it decides how the point's data is stored, aged and summarised. The policy behind each class is in Retention & Compaction.

It is rarely authored, because it defaults from kind:

kind	default class
measurement	measurement
state, setpoint, command	state
diagnostic	diagnostic

Two properties belong here rather than in the retention document, because they are facts about the descriptor:

The default is resolved at registry load and stored explicitly. The class determines physical placement, and placement must not change silently because kind was edited for an unrelated reason. What the registry stores is the resolved class, not the rule that produced it.

Changing it afterwards is a migration, in the same category as a point rename: existing rows live in a table chosen by the old class and have to be moved. A deliberate operation, not a config edit.

The semantic axes

quantity is a closed vocabulary, and it owns the canonical unit. One table, one entry per quantity, and the canonical unit is a property of the quantity rather than a per-point decision.

Decision: pragmatic canonical units (°C, Wh) rather than strict SI ones (K, J). Strict would buy clean dimensional algebra — divide energy by time, get power, units fall out. Pragmatic buys rules and dashboards that read without conversion. This system does not do dimensional algebra, so pragmatic is the right choice.

unit uses UCUM codes (Cel, mA, V, W.h, %) rather than invented strings, with a separate display_symbol. UCUM is a real standard with libraries, and it permanently kills the °C / degC / C / celsius ambiguity. The point stores unit_native; the observation stores both native and canonical values.

subject is the class of thing measured: air, water, heatsink, core_body, skin, battery-cell, mains. Also a closed vocabulary.

Subject alone is underspecified. Two air-temperature sensors in different rooms have the same subject, and that difference is location rather than subject. So the axis splits:

• subject — the class of thing (closed vocabulary)
• subject_ref — an optional typed reference to a specific modelled entity: zone:loft, device:rpi-t2, person:x

subject_ref is nullable and unresolved for now. It is the hook that allows spatial or logical modelling to be added later without reworking every point. Declaring the field costs nothing; retrofitting it costs everything.

modifiers is a small typed map for things that qualify a reading without changing quantity or subject:

• current_type: ac | dc
• phase: l1 | l2 | l3 | n
• aggregation: instant | mean | min | max | sum | delta — this is what distinguishes a daily energy total from an instantaneous power reading, and it is a real semantic difference that must not hide in the point name

aggregation earns its place twice over: it is also what tells the rollup layer that a cumulative counter must not be averaged. See Retention & Compaction.

tags are free-form and exist to hold things not yet formalised. One governance rule, enforced strictly: tags are never load-bearing. No rule, no unit conversion, no aggregation may branch on a tag. The moment one needs to, the tag is promoted to a field. Without that rule, tags become a shadow schema within six months.

Vocabularies are closed

quantity, subject, enum_type, retention_class, and the modifier value sets live in the registry as versioned tables, not free strings. Adding heatsink is a deliberate registry change. This is the only thing that prevents accumulating heatsink, heat_sink, and heatSink alongside a rule that silently matches nothing.

Enums and codebooks

A point with value_type: enum references an enum_type:

enum_type: switch_state
members:   [ on, off, unknown-position ]

Drivers map native representations (Modbus 0/1, Shelly "on"/"off", UniFi status strings) to canonical members. A native value that maps to nothing produces quality bad with the unmapped value in the detail field. It does not invent a member and does not throw.

Quality

Seven states, plus a machine-readable reason and free-text detail:

State	Meaning
unknown	never observed
restored	loaded from log at startup, not yet reconfirmed
live	observed within its freshness window
stale	observed, now past its window, cause unknown
unavailable	driver or device known-disconnected
bad	last attempt gave an invalid or device-reported-error value
assumed	written, and unverifiable

stale and unavailable are separate because the difference is actionable: stale means the cause of the silence is unknown, unavailable means it is known. Alerting and rule tolerance differ.

Two states are not entered by any runtime transition:

• unknown is the state a point is in from registration until its first observation.
• restored is entered only at core start, when the checkpointed projection loads. A point checkpointed as unknown stays unknown; everything else becomes restored regardless of what it was.

Legal runtime transitions:

unknown     → live | bad | unavailable | assumed
restored    → live | stale | bad | unavailable | assumed
live        → stale | bad | unavailable | assumed
stale       → live | bad | unavailable | assumed
bad         → live | stale | unavailable | assumed
assumed     → live | stale | bad | unavailable
unavailable → live | bad
*           → unavailable          (driver reports disconnect)

Three of those need stating explicitly, because they are the ones an implementation gets wrong:

Anything readable can become assumed. A write with confirmation_mode: optimistic asserts a value irrespective of what the point held before, so unknown → assumed (a cold-boot relay with init_strategy: assert(v)), restored → assumed, stale → assumed and bad → assumed are all reachable. The prior state has no bearing on it; that is exactly what assumed means.

unavailable has only two exits, and both require real data. A point known to be off the network is never promoted synthetically, so there is no unavailable → stale and no unavailable → live by timer expiry. It leaves only when an observation arrives. There is deliberately no unavailable → assumed either: writing to a device known to be disconnected must not manufacture confidence in the result.

No transition returns to unknown or restored. Both are startup states. A point that has been observed once has been observed forever, and restored is a claim about a particular process start.

The rule that matters most: when quality is not live, the value field still holds the last known value. It is never nulled. Consumers receive (value, quality, observed_at) and decide for themselves. Clearing the value on staleness destroys information and reintroduces the null-means-three-things problem this model exists to escape.

Drivers may only assert live, bad, or unavailable. Everything else is the core's to derive. This keeps the driver contract small and prevents a badly-behaved driver from declaring things it cannot know.

Time

Three timestamps. Collapsing them to two destroys information that later mechanisms depend on:

• device_time — nullable, untrusted. Devices without an RTC will lie. Used for analysis only, and flagged when skew against observed_at exceeds a threshold.
• observed_at — when the driver saw it. This is what freshness is computed against.
• ingested_at — when the bus accepted the observation. This is what the log orders by.

ingested_at is taken from the bus message itself, not from the core's clock at the moment of handling. It is part of the dedup key, so it must be identical on every redelivery of the same message; the reasoning is in Ingest Path & Log Schema and it is not safe to change here in isolation. The practical reading is that it means "when this observation became durable in the system", which is also the more useful of the two candidate meanings.

The second and third diverge exactly when a driver host buffers through a network partition and then floods. Any host on a wireless link or a segment that drops makes this routine rather than exceptional, and BLE and radio hosts are the obvious candidates. Collapsed together, a burst of buffered readings all look fresh at the moment they land.

source_seq, monotonic per driver instance, exists for dedup and out-of-order detection.

Change vs refresh

change_policy (any | absolute(delta) | relative(pct) | none) belongs in the semantic model because it determines what the rules engine sees as an edge.

Every observation refreshes freshness. Only observations that clear the change policy emit a value_changed event. Without the deadband, a noisy analogue sensor generates an edge per sample and the rules engine spends its life on 0.01 °C.

The changed flag this produces has a second life in storage: it is what state-class compaction keeps and what it drops. See Retention & Compaction.

The observation envelope

What a driver publishes:

point_id, value_native, unit_native,
quality (live|bad|unavailable), reason, detail,
device_time?, observed_at, source_seq,
driver_id, driver_instance

What the core adds on ingest:

event_id, ingested_at,
value_canonical, unit_canonical,
prev_value, prev_quality,
changed (bool, per change_policy)

That second block is the full transition tuple, which is exactly what the rules engine consumes. Edge detection, staleness triggers, and bad-data triggers all read the same record.

Normalisation is a write-time decision

Core-side normalisation plus a log-first core means canonical values are computed once, at ingest, and then frozen in the log. If a wrong unit_native is corrected in the registry six months later, every historical event stays wrong.

The fix is cheap now and impossible later: the log stores native and canonical side by side, plus the registry version that did the conversion.

value_native, unit_native, value_canonical, unit_canonical, registry_version

Reprocessing then becomes a real operation — replay the log, reconvert from native under the current registry, rebuild the projection. Without value_native in the log, a unit error is permanent data loss. This is codified as a reprocess command from day one rather than a script written in anger later.

Registry authority and the two failure cases

With drivers dumb, the core needs explicit answers for two things that happen in week one:

Unregistered point. A driver publishes boiler.ch1.return-temp and the registry has never heard of it. It is neither dropped nor auto-registered. It goes to a quarantine stream with full native payload, raised for triage. Auto-registration guesses at semantics, and a guessed quantity is worse than no point at all. Quarantine also provides discovery as a side effect: connect a new device, let the driver run, and watch what accumulates.

Unit mismatch. Some sources self-describe (Shelly and UniFi JSON carry units), most do not. Where a driver supplies unit_native and it disagrees with the registry, the observation gets quality bad with reason unit_mismatch. Neither claim is converted from — either could be the wrong one, and silently picking is how a heatsink reading of 90 °F becomes a value a rule treats as 90 °C.

The quantity table

Pragmatic canonicals. Two columns that are easy to omit and expensive to add later:

quantity	canonical (UCUM)	affine	agg_safe
temperature	Cel	yes	yes
temperature_delta	Cel	no	yes
relative_humidity	%	no	yes
pressure	kPa	no	yes
voltage	V	no	yes
current	A	no	yes
power_active	W	no	yes
power_apparent	V.A	no	yes
power_factor	1	no	yes
frequency	Hz	no	yes
energy	W.h	no	yes
charge	A.h	no	yes
illuminance	lx	no	yes
flow_volumetric	L/min	no	yes
volume	L	no	yes
signal_strength	dBm	no	no
sound_level	dB	no	no
duration	s	no	yes
ratio	%	no	yes
count	1	no	yes

affine exists because °F → °C needs an offset, not just a scale factor. A multiplicative-only conversion layer is correct for every quantity except temperature, which is the one most likely to be converted. The conversion is value × factor + offset from the start.

agg_safe: false marks logarithmic quantities. The mean of two dBm values is not the dBm of the mean, and a naive moving average over RSSI produces a number that looks plausible and means nothing. Derived points refuse to compute mean on a non-agg-safe quantity unless the definition explicitly opts into log_mean, and the same flag governs which functions a rollup may use.

temperature_delta as a distinct quantity is the same class of trap. A 5 °C rise is not 41 °F.

Two notes on the table. pressure in kPa is awkward for both atmospheric and hydraulic readings, since neither is the natural unit; one quantity handled through display_symbol is preferred to splitting, and the decision is worth revisiting once real pressure points exist. And quantity is required for kind: measurement and null for boolean and enum state points — a state pseudo-quantity must not be invented to fill the column.

Subject and subject_ref

Starter vocabulary, flat, to be edited rather than accepted as given:

air, water, surface, heatsink, enclosure, soil, mains, circuit, battery, core_body, skin, ambient, self

self covers diagnostics — a device reporting its own uptime, free heap, or signal strength. Those are real points with real freshness needs and must not be forced into a physical subject.

subject_ref format is {kind}:{slug}, with kinds zone, device, circuit, asset, person. The format is validated, the string stored, and nothing resolved. No entity tables until a second consumer needs them.

The signal that flat subjects have stopped working is a point declared as subject: water with tags: [flow] and another with tags: [return], and then a rule branching on the tag. That is the tag governance rule firing as designed. When it happens the distinction gets promoted — either to a circuit_position modifier or to a nested subject — and by then there is real evidence for which.

Vocabularies as code

All five closed vocabularies (quantity, subject, enum_type, retention_class, modifier value sets) live as versioned files in the repository, loaded into Postgres by a migration. The registry version stamped on every observation refers to this.

Three checks belong in CI rather than at runtime:

• every point's unit_native is convertible to its quantity's canonical unit
• no point references a vocabulary member that does not exist
• no vocabulary member is removed while points still reference it

The last is the migration guard. Renaming heatsink to heat-sink fails the build rather than silently orphaning forty points.

Exit criteria

This step is done when a test suite demonstrates that the registry rejects what it is supposed to reject. Not that it accepts good input — that will be obvious — but that bad input fails at load rather than in production.

A fixture set of deliberately invalid descriptors, each expected to fail with a named reason:

• unit_native not convertible to its quantity's canonical unit
• a reference to a vocabulary member that does not exist
• on_change freshness with neither heartbeat nor an explicit never
• a retention_class whose rollup functions cannot apply to the point's value_type or quantity
• require_intent: true with no default_value
• removing a vocabulary member while points still reference it

Plus two positive checks that are cheap and catch real errors:

• Conversion round-trip against known pairs, affine included: °F → °C at 32 and 212, mA → A, W.h at scale. A multiplicative-only conversion layer passes every case except the one that matters, so the affine cases must be in the table.
• The quality transition table exists as a checked enum, with a test asserting that no code path can produce a transition outside it. The prose table above is for readers; this is what keeps it true.

---

On Monday 14 September - the Dev Diary will look at what needed to change when the model spec'ed above met reality.

Then on Wednesday 16 September - The Driver Contract. Why every driver speaks one northbound transport no matter what protocol it talks to the device in, and why converting a unit is on the list of things a driver is forbidden to do.

Start of the series: An Introduction. The map, the two decisions every later document is downstream of, and why a specification at this scale is being published in public while the system it describes gets built.