Thank you, @pengeszikra Really appreciate you reading and highlighting that point. In physical systems like railways, you simply cannot let an LLM have unchecked authority over safety-critical thresholds. Combining cognitive reasoning with deterministic code interlocks has to become standard practice if we want autonomous systems trusted in the real world. Thanks for the support!
Start many of them, for example on hacktober I just miss 2 weeks, but even that weeks also created something. Now we live in the golden ages of AI before singularity, so we have very strong tools to make our dreams comes to true.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
Two holes worth closing before this sits in front of a dispatcher: enforce_guardrails receives wind_speed and signal_state as arguments, so it only guards the values it was handed, and the bare wind_speed >= 65 has no unit binding attached to it -- an anemometer reporting m/s clears that threshold only at about 234 km/h. Bind the observation to the advisory (corridor id, source, timestamp) and re-check freshness inside the interlock; if the read is older than your poll interval or the tool call raised, return STOP instead of falling through to the model's number. The proof is two assertions: speed 140 with wind 70.6 must come out 80, and a stale observation timestamp must come out 0.
This is gold @dafili, thank you for such a sharp systems-engineering review! Both of those points hit right at the heart of real-world safety engineering:
Unit-bound types (km/h vs m/s): Passing raw floats is how Mars Climate Orbiter-style bugs happen. Wrapping telemetry fields in explicit unit-validated Pydantic types (or pint quantities) at the ingestion boundary so wind_speed_kmh rejects or converts raw m/s anemometer reads before enforce_guardrails ever runs is a must.
Binding the signed observation directly to the interlock: Instead of trusting parameters passed back through the agent loop, enforce_guardrails should pull the immutable, timestamped sensor struct (corridor_id, source_id, observed_at_utc) directly from the telemetry store and verify freshness inside the guardrail itself.
I love those two exact unit test assertions (speed=140, wind=70.6 -> 80 and stale_timestamp -> 0). I am going to add both of those directly into the repo's test suite!
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
Two traps when you write those assertions. Test the guardrail function, not the endpoint — if the assertion goes through the agent loop you are testing the model's mood, not the interlock:
deftest_wind_clamp_from_ms_reading():obs=WindObservation(raw=17.5,unit="m/s")# 63 km/h, under the 65 clamp
assertenforce_guardrails(speed_kmh=140,wind=obs).speed_kmh==140withpytest.raises(ValidationError):enforce_guardrails(speed_kmh=140,wind=70.6)# bare float must not be accepted
deftest_stale_telemetry_boundary():assertenforce_guardrails(140,wind_ok,age_s=15).speed_kmh==140assertenforce_guardrails(140,wind_ok,age_s=15.001).speed_kmh==0
Second: pin the freshness boundary, not just "a stale timestamp" — exactly poll_window and poll_window + 1ms must land on opposite sides, or the comparison flips silently the day someone rewrites > as >=. And assert on the returned JSON, never on the summary text: the failure that mattered was a model that named the crosswind in its explanation and still emitted 120 km/h, which any text assertion passes happily.
100% agreed @dafili, especially on pinning both sides of the exact 15.000s vs 15.001s poll-window boundary (> vs >= off-by-one regressions are classic silent killers during refactors) and testing enforce_guardrails() as a pure, isolated function rather than routing unit tests through the stochastic LLM loop.
One additional detail I like pairing with your age_s boundary test in production telemetry is using a monotonic clock delta or rejecting negative clock skew (age_s < -max_clock_skew_s -> speed_kmh == 0). If a wayside sensor's NTP clock drifts ahead of the central dispatch container, a naive now - observed_at check produces a negative age_s that silently passes age_s <= 15 even if the sensor is stuck repeating a future-stamped packet!
So the full boundary contract on the structured output becomes:
wind=19.61 m/s (70.6 km/h converted via WindObservation) -> 80
Really appreciate you sharing those exact pytest snippets. This thread is basically a mini-masterclass on why safety interlocks and LLM reasoning have to be tested with completely different harnesses!
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
Your skew case is the one that actually survives review, with one refinement: a monotonic delta on the consumer side does not fix a producer whose clock runs ahead, because the two sides share no origin. Stamp received_at from the consumer's own time.monotonic() at ingest and derive age_s from that; keep the producer's observed_at only as telemetry (producer_offset_s), never as input to the freshness decision. Then a future-stamped packet fails on arrival instead of on age.
The other half is making the clamp observable. age_s < -0.5 -> 0 is the right safety output, but it also hides a wayside box with permanently broken NTP, and the dispatcher just sees a train held at restrictive with no cause. So the structured output needs stale_reason="clock_skew" next to the speed, and you alarm on that counter being nonzero — a clamp without a reason field is how this class of bug stays silent for a month.
The edge I would pin, with the reason asserted separately:
An int-only assertion passes even when a refactor routes clock skew down the ordinary stale path, which is exactly the failure you want a test to catch and not excuse.
The 65 km/h clamp guards the decision, but what guards the input to the decision? A wind reading that is ten minutes stale is a number that passes every range check while describing weather that no longer exists. Three things make this boring instead of scary: every sensor value arrives with its age attached, the interlock checks age against a per-sensor TTL rather than checking the value against a threshold alone, and a read older than its TTL doesn't get clamped — it drives the interlock to its most restrictive state until fresh data lands. The dispatcher then sees "stale, holding at restrictive" instead of a confident number. It's a small contract change: the guardrail stops trusting values and starts trusting (value, age) pairs. Do you set the TTL per sensor from the telemetry's own publish rate, or is a single global staleness budget simpler to reason about in front of a dispatcher?
If you used a 60s global TTL, a 10-second-old signal state would be dangerously stale; if you used a 3s global TTL, the wind sensor would constantly trip false-stale holds. So each sensor type carries its own max_age_s = k * publish_interval_s in the Python interlock, but if any safety-critical input exceeds its own TTL, the operator cockpit surfaces one clear, unambiguous banner (STALE TELEMETRY [WIND_SENSOR]: HOLDING AT RESTRICTIVE) so the dispatcher never has to do mental math across different sensor clocks during an emergency.
Rail dispatch is a good test case because an optimistic but wrong schedule has a physical cost. I would make every recommendation carry a confidence band plus an operator-facing explanation of the constrained resources and counterfactual: which conflict it resolves, and what it displaces. That makes the model useful for triage without turning a black-box score into automatic authority.
Spot on @alexshev! Showing the counterfactual ("which conflict this resolves, and which downstream train or siding window it displaces") is what separates a real decision-support cockpit from a black-box oracle.
In single-track or bottlenecked rail corridors, every recovery action is a trade-off: holding a freight consist at Siding B clears the block for an oncoming express, but pushes the freight crew closer to their statutory hours-of-service limit. Extending the RailDispatchAdvisory Pydantic schema with explicit resolved_conflicts, displaced_resources (for example, secondary delay minutes added to opposing traffic), and an uncertainty/confidence band gives the human dispatcher immediate situational awareness to accept or override the plan in seconds. Love that framing of using the model for triage rather than unchecked authority!
I research AI tools, APIs & agents — comparing specs, pricing and real user reports — then publish honest rankings so you don't waste money. Research-first, never sponsored fluff.
The 65 km/h wind clamp is the detail that makes this real for me — a hard number in code, not a suggestion in a prompt. The part I'd want to see next is the one aifrontierpost already asked about: telemetry includes axle bearing temperatures, so where does the hot-bearing check live? If that's in the interlock too, this is the cleanest safety-critical agent pattern I've seen written up.
Really appreciate that @paullovelace Yes, hot-box (axle bearing temperature) detection belongs 100% inside the deterministic Python interlock right alongside the wind clamp and signal state, never left up to prompt instructions alone.
In rail operations, bearing temperature usually has two hard thresholds: an elevated warning band (for example, above 85°C where you clamp speed and route the train to the next siding for inspection) and a critical thermal threshold (above 105°C or a rapid delta spike compared to adjacent axles on the same bogie, which forces an immediate 0 km/h emergency stop to prevent axle seizure). Having Gemma summarize why the bearing is heating up alongside track gradient and load weight is super helpful for the human dispatcher, but the actual speed clamp has to live in deterministic code.
The deterministic safety interlock is the part that stuck with me — Gemma proposes a schedule recovery plan, but the hard-coded Python check against the 65 km/h wind threshold is what actually decides whether 145 km/h stands. That split between reasoning and enforcement is a clean answer to the 'never give an LLM sole authority over life-safety decisions' problem. The Pydantic output contract is a nice second layer too: the operator dashboard never has to parse freeform text. Did the interlock ever catch the model mid-recovery in testing, or did the wind-clamp scenario mostly surface in the DevFest demo?
Great question @onelifevibe ! It actually caught the model multiple times during our stress testing before the DevFest Baku demo, and in a very revealing way.
Whenever we gave the scenario a severe schedule delay (for example, a passenger train running 25 minutes late with a tight single-track siding window ahead) and borderline high crosswinds (around 68 to 72 km/h), smaller quantized models would occasionally prioritize "making up schedule time" in their reasoning chain and propose 110 to 125 km/h while still acknowledging the wind in the text summary! That was our biggest "aha" moment during testing: an LLM can literally mention a hazard in its explanation while still outputting an unsafe numeric parameter in the JSON tool call. Watching the Python interlock catch that 120 km/h proposal and slam it down to 80 km/h in real time proved why you always need code enforcement sitting after the model.
This is a great example of an agent built around a real operational constraint rather than a demo. I especially liked the deterministic SafetyInterlock — capping speed when wind crosses the 65 km/h threshold instead of letting Gemma make the final safety call — and the Pydantic RailDispatchAdvisory contract that keeps the dashboard from breaking on freeform text. How did you handle a case where the telemetry feed itself goes stale mid-dispatch? Does the interlock fall back to a conservative speed, or hold the last advisory until fresh data arrives?
Thank you @iictp! That is a great catch because in real SCADA and railway telemetry, dropped packets or stale sensor reads happen all the time, especially through mountain tunnels.
Holding the last advisory is actually dangerous if a train is entering a high-wind zone or approaching a signal change. In a production deployment, the interlock should treat stale telemetry as a degraded safety state rather than trusting the last cached number. Specifically, every telemetry payload carries a UTC timestamp, and if the delta between the current clock and the last sensor heartbeat exceeds the poll window (for example, 15 seconds), the interlock bypasses the LLM recommendation entirely, steps the speed target down to a conservative restricted-sight speed (or triggers a full hold depending on the block state), and flags a TELEMETRY_STALE warning on the operator cockpit. Never let the agent guess when the sensors go blind!
The obsession with wrapping remote APIs into multi-layered agent abstractions is creating massive hidden latency and failure cascades in production.
If you profile real-world workloads, bare-metal local inference (via direct C++ / llama.cpp or vLLM bindings) consistently beats fragile cloud wrappers on both predictability and cost. Tooling should eliminate abstraction penalties, not compound them.
Spot on @devkursat , and I actually agree with you on the latency and abstraction tax! That is one of the main reasons we chose open-weight Gemma models for this architecture rather than locking into a proprietary cloud-only API.
When you own the weights, you can run the exact same Gemma model on bare-metal edge hardware at the wayside or locomotive level via vLLM or llama.cpp for deterministic sub-second latency, or inside a self-contained GPU container on Cloud Run when you want elastic central dispatch without managing Kubernetes clusters. The agent layer here is kept intentionally thin (just structured tool bindings + hard Python safety interlocks) so there are no multi-hop prompt chains adding unpredictable tail latency.
We're a place where coders share, stay up-to-date and grow their careers.
👏👏👏
Congratulation!
Good to see someone who is care about real problem!
I hope many of us try to found a right solution for a real problem.
👏👏👏
Thank you, @pengeszikra Really appreciate you reading and highlighting that point. In physical systems like railways, you simply cannot let an LLM have unchecked authority over safety-critical thresholds. Combining cognitive reasoning with deterministic code interlocks has to become standard practice if we want autonomous systems trusted in the real world. Thanks for the support!
by the way how did you got this subscription badge @pengeszikra
I was won a copilot vibe contest
Cool .I wanted to win
Start many of them, for example on hacktober I just miss 2 weeks, but even that weeks also created something. Now we live in the golden ages of AI before singularity, so we have very strong tools to make our dreams comes to true.
Two holes worth closing before this sits in front of a dispatcher: enforce_guardrails receives wind_speed and signal_state as arguments, so it only guards the values it was handed, and the bare
wind_speed >= 65has no unit binding attached to it -- an anemometer reporting m/s clears that threshold only at about 234 km/h. Bind the observation to the advisory (corridor id, source, timestamp) and re-check freshness inside the interlock; if the read is older than your poll interval or the tool call raised, return STOP instead of falling through to the model's number. The proof is two assertions: speed 140 with wind 70.6 must come out 80, and a stale observation timestamp must come out 0.This is gold @dafili, thank you for such a sharp systems-engineering review! Both of those points hit right at the heart of real-world safety engineering:
km/hvsm/s): Passing raw floats is how Mars Climate Orbiter-style bugs happen. Wrapping telemetry fields in explicit unit-validated Pydantic types (orpintquantities) at the ingestion boundary sowind_speed_kmhrejects or converts rawm/sanemometer reads beforeenforce_guardrailsever runs is a must.enforce_guardrailsshould pull the immutable, timestamped sensor struct (corridor_id,source_id,observed_at_utc) directly from the telemetry store and verify freshness inside the guardrail itself.I love those two exact unit test assertions (
speed=140, wind=70.6 -> 80andstale_timestamp -> 0). I am going to add both of those directly into the repo's test suite!Two traps when you write those assertions. Test the guardrail function, not the endpoint — if the assertion goes through the agent loop you are testing the model's mood, not the interlock:
Second: pin the freshness boundary, not just "a stale timestamp" — exactly
poll_windowandpoll_window + 1msmust land on opposite sides, or the comparison flips silently the day someone rewrites>as>=. And assert on the returned JSON, never on the summary text: the failure that mattered was a model that named the crosswind in its explanation and still emitted 120 km/h, which any text assertion passes happily.100% agreed @dafili, especially on pinning both sides of the exact
15.000svs15.001spoll-window boundary (>vs>=off-by-one regressions are classic silent killers during refactors) and testingenforce_guardrails()as a pure, isolated function rather than routing unit tests through the stochastic LLM loop.One additional detail I like pairing with your
age_sboundary test in production telemetry is using a monotonic clock delta or rejecting negative clock skew (age_s < -max_clock_skew_s -> speed_kmh == 0). If a wayside sensor's NTP clock drifts ahead of the central dispatch container, a naivenow - observed_atcheck produces a negativeage_sthat silently passesage_s <= 15even if the sensor is stuck repeating a future-stamped packet!So the full boundary contract on the structured output becomes:
age_s == 15.000->140(fresh boundary inclusive)age_s == 15.001->0(stale boundary exclusive)age_s < -0.500->0(clock-skew / invalid future timestamp clamp)wind=19.61 m/s(70.6 km/hconverted viaWindObservation) ->80Really appreciate you sharing those exact
pytestsnippets. This thread is basically a mini-masterclass on why safety interlocks and LLM reasoning have to be tested with completely different harnesses!Your skew case is the one that actually survives review, with one refinement: a monotonic delta on the consumer side does not fix a producer whose clock runs ahead, because the two sides share no origin. Stamp
received_atfrom the consumer's owntime.monotonic()at ingest and deriveage_sfrom that; keep the producer'sobserved_atonly as telemetry (producer_offset_s), never as input to the freshness decision. Then a future-stamped packet fails on arrival instead of on age.The other half is making the clamp observable.
age_s < -0.5 -> 0is the right safety output, but it also hides a wayside box with permanently broken NTP, and the dispatcher just sees a train held at restrictive with no cause. So the structured output needsstale_reason="clock_skew"next to the speed, and you alarm on that counter being nonzero — a clamp without a reason field is how this class of bug stays silent for a month.The edge I would pin, with the reason asserted separately:
An int-only assertion passes even when a refactor routes clock skew down the ordinary stale path, which is exactly the failure you want a test to catch and not excuse.
The 65 km/h clamp guards the decision, but what guards the input to the decision? A wind reading that is ten minutes stale is a number that passes every range check while describing weather that no longer exists. Three things make this boring instead of scary: every sensor value arrives with its age attached, the interlock checks age against a per-sensor TTL rather than checking the value against a threshold alone, and a read older than its TTL doesn't get clamped — it drives the interlock to its most restrictive state until fresh data lands. The dispatcher then sees "stale, holding at restrictive" instead of a confident number. It's a small contract change: the guardrail stops trusting values and starts trusting (value, age) pairs. Do you set the TTL per sensor from the telemetry's own publish rate, or is a single global staleness budget simpler to reason about in front of a dispatcher?
I love how you phrased that @eye_java_420f1faa10ae8b86: "the guardrail stops trusting values and starts trusting
(value, age)pairs."In practice, a two-layer hybrid works best: per-sensor TTL in the backend interlock mapped to a single unified freshness state on the dispatcher UI.
At the ingestion and guardrail layer, a single global TTL breaks down quickly because different railway subsystems publish on very different cadences:
TTL ~ 2 to 3s)TTL ~ 10 to 15s, usually2x to 3xthe publish interval)TTL ~ 60 to 120s)If you used a 60s global TTL, a 10-second-old signal state would be dangerously stale; if you used a 3s global TTL, the wind sensor would constantly trip false-stale holds. So each sensor type carries its own
max_age_s = k * publish_interval_sin the Python interlock, but if any safety-critical input exceeds its own TTL, the operator cockpit surfaces one clear, unambiguous banner (STALE TELEMETRY [WIND_SENSOR]: HOLDING AT RESTRICTIVE) so the dispatcher never has to do mental math across different sensor clocks during an emergency.Rail dispatch is a good test case because an optimistic but wrong schedule has a physical cost. I would make every recommendation carry a confidence band plus an operator-facing explanation of the constrained resources and counterfactual: which conflict it resolves, and what it displaces. That makes the model useful for triage without turning a black-box score into automatic authority.
Spot on @alexshev! Showing the counterfactual ("which conflict this resolves, and which downstream train or siding window it displaces") is what separates a real decision-support cockpit from a black-box oracle.
In single-track or bottlenecked rail corridors, every recovery action is a trade-off: holding a freight consist at Siding B clears the block for an oncoming express, but pushes the freight crew closer to their statutory hours-of-service limit. Extending the
RailDispatchAdvisoryPydantic schema with explicitresolved_conflicts,displaced_resources(for example, secondary delay minutes added to opposing traffic), and an uncertainty/confidence band gives the human dispatcher immediate situational awareness to accept or override the plan in seconds. Love that framing of using the model for triage rather than unchecked authority!The 65 km/h wind clamp is the detail that makes this real for me — a hard number in code, not a suggestion in a prompt. The part I'd want to see next is the one aifrontierpost already asked about: telemetry includes axle bearing temperatures, so where does the hot-bearing check live? If that's in the interlock too, this is the cleanest safety-critical agent pattern I've seen written up.
Really appreciate that @paullovelace Yes, hot-box (axle bearing temperature) detection belongs 100% inside the deterministic Python interlock right alongside the wind clamp and signal state, never left up to prompt instructions alone.
In rail operations, bearing temperature usually has two hard thresholds: an elevated warning band (for example, above 85°C where you clamp speed and route the train to the next siding for inspection) and a critical thermal threshold (above 105°C or a rapid delta spike compared to adjacent axles on the same bogie, which forces an immediate
0 km/hemergency stop to prevent axle seizure). Having Gemma summarize why the bearing is heating up alongside track gradient and load weight is super helpful for the human dispatcher, but the actual speed clamp has to live in deterministic code.The deterministic safety interlock is the part that stuck with me — Gemma proposes a schedule recovery plan, but the hard-coded Python check against the 65 km/h wind threshold is what actually decides whether 145 km/h stands. That split between reasoning and enforcement is a clean answer to the 'never give an LLM sole authority over life-safety decisions' problem. The Pydantic output contract is a nice second layer too: the operator dashboard never has to parse freeform text. Did the interlock ever catch the model mid-recovery in testing, or did the wind-clamp scenario mostly surface in the DevFest demo?
Great question @onelifevibe ! It actually caught the model multiple times during our stress testing before the DevFest Baku demo, and in a very revealing way.
Whenever we gave the scenario a severe schedule delay (for example, a passenger train running 25 minutes late with a tight single-track siding window ahead) and borderline high crosswinds (around 68 to 72 km/h), smaller quantized models would occasionally prioritize "making up schedule time" in their reasoning chain and propose 110 to 125 km/h while still acknowledging the wind in the text summary! That was our biggest "aha" moment during testing: an LLM can literally mention a hazard in its explanation while still outputting an unsafe numeric parameter in the JSON tool call. Watching the Python interlock catch that 120 km/h proposal and slam it down to 80 km/h in real time proved why you always need code enforcement sitting after the model.
This is a great example of an agent built around a real operational constraint rather than a demo. I especially liked the deterministic SafetyInterlock — capping speed when wind crosses the 65 km/h threshold instead of letting Gemma make the final safety call — and the Pydantic RailDispatchAdvisory contract that keeps the dashboard from breaking on freeform text. How did you handle a case where the telemetry feed itself goes stale mid-dispatch? Does the interlock fall back to a conservative speed, or hold the last advisory until fresh data arrives?
Thank you @iictp! That is a great catch because in real SCADA and railway telemetry, dropped packets or stale sensor reads happen all the time, especially through mountain tunnels.
Holding the last advisory is actually dangerous if a train is entering a high-wind zone or approaching a signal change. In a production deployment, the interlock should treat stale telemetry as a degraded safety state rather than trusting the last cached number. Specifically, every telemetry payload carries a UTC timestamp, and if the delta between the current clock and the last sensor heartbeat exceeds the poll window (for example, 15 seconds), the interlock bypasses the LLM recommendation entirely, steps the speed target down to a conservative restricted-sight speed (or triggers a full hold depending on the block state), and flags a
TELEMETRY_STALEwarning on the operator cockpit. Never let the agent guess when the sensors go blind!The obsession with wrapping remote APIs into multi-layered agent abstractions is creating massive hidden latency and failure cascades in production.
If you profile real-world workloads, bare-metal local inference (via direct C++ / llama.cpp or vLLM bindings) consistently beats fragile cloud wrappers on both predictability and cost. Tooling should eliminate abstraction penalties, not compound them.
Spot on @devkursat , and I actually agree with you on the latency and abstraction tax! That is one of the main reasons we chose open-weight Gemma models for this architecture rather than locking into a proprietary cloud-only API.
When you own the weights, you can run the exact same Gemma model on bare-metal edge hardware at the wayside or locomotive level via
vLLMorllama.cppfor deterministic sub-second latency, or inside a self-contained GPU container on Cloud Run when you want elastic central dispatch without managing Kubernetes clusters. The agent layer here is kept intentionally thin (just structured tool bindings + hard Python safety interlocks) so there are no multi-hop prompt chains adding unpredictable tail latency.