DEV Community: Rafia

Industrial AI Needs Hard Boundaries: Hybrid Rules and LLMs for Energy Systems

Rafia — Fri, 30 Oct 2026 13:50:09 +0000

Large language models excel at synthesizing messy context, translating unstructured logs, and answering open-ended queries. However, they are fundamentally probabilistic. In production environments where physical infrastructure is on the line—such as high-voltage distribution networks, factory sub-stations, and thermal storage plants—probabilistic behavior can be a liability.

If an operator asks an operations assistant to summarize grid telemetry and suggest a load-shedding sequence, an LLM might hallucinate a safety threshold or confuse megawatt-hours with kilowatt-hours. In software engineering, this is a bug. In industrial facilities, this triggers breaker trips, equipment degradation, or violation of contractual peak-demand limits.

To build dependable tools for physical systems, you cannot let an LLM make raw control decisions or act as the primary validation layer. The solution is a hybrid architecture: deterministic, rule-based systems handle physics, state estimation, and safety limits, while the LLM acts strictly as an interface and context-aggregator.

The Failure Mode of Pure LLMs in Operations

LLMs operate over token distributions, not physical laws. Even with few-shot prompting and retrieval-augmented generation (RAG), three failure modes persist in industrial telemetry:

Numerical Drift and Hallucinated Units: A model can effortlessly read 480V in one paragraph and output 480 kW in the next when calculating phase load.
Silently Dropped Constraints: Prompts with complex nested conditions ("never shed Feeder 3 if Boiler 1 is running, unless Ambient Temperature exceeds 35°C") degrade in reliability as context lengths grow or when edge-case telemetry arrives.
Non-Deterministic Tool Execution: When models are granted direct access to APIs via function calling, variations in temperature sampling or minor prompt mutations can cause them to skip a required safety pre-check before calling a dispatch endpoint.

In energy monitoring, these failures carry tangible costs. Exceeding contracted peak power capacity for even a fifteen-minute window can alter an industrial consumer's utility tariff for months.

Architecture: Rules as the Substrate, LLMs as the Interface

A robust architecture decouples the execution layer from the language layer. Instead of asking the model to monitor the plant, the plant's monitoring engine continuously evaluates deterministic rules, computes state summaries, and hands verified facts to the model.

[ Raw Sensors / PLC Telemetry ]
               │
               ▼
┌──────────────────────────────┐
│ Deterministic Rule Engine    │
│ - Physics validation         │
│ - Threshold & rate checks    │
│ - Hard interlocks & trips    │
└──────────────┬───────────────┘
               │
       Validated State & Alerts
               │
               ▼
┌──────────────────────────────┐
│ LLM Translation Layer        │ ◄── [ Operator Natural Language Query ]
│ - Incident summarization     │
│ - Log cross-referencing      │
│ - Contextual explanation     │
└──────────────┬───────────────┘
               │
       Proposed Action Plan
               │
               ▼
┌──────────────────────────────┐
│ Action Firewall (Gatekeeper) │
│ - Schema validation          │
│ - Safety envelope assertion  │
└──────────────┬───────────────┘
               │
[ Physical Dispatch / Actuator ]

In this model, the LLM has two roles:

Downstream synthesis: Converting structured rule violations, raw time-series trends, and maintenance logs into clear status updates for human operators.
Upstream proposal: Translating unstructured human requests ("Prep line 2 for peak-shaving this afternoon") into structured function calls.

Crucially, those function calls do not hit the physical systems directly. They pass through an Action Firewall—a deterministic validation service that evaluates proposals against hard engineering constraints.

Concrete Implementation: Power Load Thresholding

Consider an energy monitoring system where an operator wants an LLM assistant to draft load-shedding configurations during high-tariff periods. Below is a simplified Python pattern illustrating how deterministic assertions surround the LLM interaction.

from pydantic import BaseModel, Field, model_validator
from typing import List, Literal

# 1. Deterministic Domain Models with Hard Invariants
class LoadShedAction(BaseModel):
    feeder_id: str
    curtailment_kw: float = Field(gt=0, description="Must curtail positive kW")
    priority: int = Field(ge=1, le=5)

class OperationalEnvelope(BaseModel):
    max_total_curtailment_kw: float
    protected_feeders: List[str]

class SheddingPlan(BaseModel):
    actions: List[LoadShedAction]

    def validate_safety(self, envelope: OperationalEnvelope) -> None:
        total_kw = sum(a.curtailment_kw for a in self.actions)
        if total_kw > envelope.max_total_curtailment_kw:
            raise ValueError(
                f"Total curtailment {total_kw}kW exceeds limit of "
                f"{envelope.max_total_curtailment_kw}kW"
            )

        for action in self.actions:
            if action.feeder_id in envelope.protected_feeders:
                raise ValueError(f"Feeder {action.feeder_id} is protected and cannot be shed")

# 2. Gatekeeper: LLM Output Verification
def execute_llm_proposal(raw_llm_json: dict, envelope: OperationalEnvelope):
    try:
        # Step A: Validate syntactic structure
        plan = SheddingPlan(**raw_llm_json)

        # Step B: Assert physical and business rules deterministically
        plan.validate_safety(envelope)

        # Step C: If and only if rules pass, allow dispatch
        print(f"Plan valid. Dispatching curtailment for {[a.feeder_id for a in plan.actions]}.")
        return {"status": "applied", "plan": plan.model_dump()}

    except ValueError as e:
        # The system rejects the action deterministically before state change
        print(f"Safety violation rejected by rule engine: {str(e)}")
        return {"status": "rejected", "reason": str(e)}

# Example Scenario:
envelope = OperationalEnvelope(
    max_total_curtailment_kw=500.0,
    protected_feeders=["FEEDER_CRITICAL_COOLING", "FEEDER_HVAC_HQ"]
)

# LLM erroneously generates a plan that targets a protected cooling feeder
llm_generated_output = {
    "actions": [
        {"feeder_id": "FEEDER_CRITICAL_COOLING", "curtailment_kw": 120.0, "priority": 2},
        {"feeder_id": "FEEDER_ASSEMBLY_B", "curtailment_kw": 150.0, "priority": 1}
    ]
}

result = execute_llm_proposal(llm_generated_output, envelope)
# Output: Safety violation rejected by rule engine: Feeder FEEDER_CRITICAL_COOLING is protected and cannot be shed

The LLM can write text, parse emails from the grid operator, and construct JSON payloads. But the Python layer ensures no execution occurs if an invariant is broken.

Evaluation and Safety Invariants

When testing software that controls physical assets, unit testing prompts is insufficient. You need distinct test suites for both halves of the hybrid system:

1. Rule Engine Test Suite

The rule engine should be evaluated with traditional property-based testing. Use tools like hypothesis to fuzz the boundary conditions: negative loads, sudden frequency drops, out-of-order time-series packets, and missing sensor values. Verify that safety interlocks trigger under all fault states without relying on any machine learning model.

2. LLM Interface Evaluation

Instead of evaluating the LLM for general conversational capability, score it on compliance and recovery:

Rejection Recovery: When the Action Firewall rejects a plan, does the model take the error string and output a valid alternative, or does it loop into repeated invalid outputs?
Information Extraction Precision: Given real telemetry logs, does the model accurately report out-of-bound variables without skipping entries?
Schema Adherence: What percentage of LLM responses adhere strictly to the target JSON schema on the first attempt?

If the LLM fails, the system must fail safe. For instance, if the LLM service suffers an outage or returns invalid JSON three times consecutively, the monitoring system falls back to predefined, deterministic heuristic tables for load management.

Key takeaways

Separate translation from execution: Use the LLM to ingest context, query logs, and draft plans, but keep it isolated from direct dispatch APIs.
Enforce an Action Firewall: Every model output that could alter system state must pass through deterministic schema and physics validations before execution.
Physics over probabilities: Never rely on system prompts or RAG to guarantee hard thresholds; token probabilities cannot replace mathematical assertions.
Design for deterministic fallbacks: If the generative layer produces invalid outputs or becomes unreachable, the rule engine must maintain baseline monitoring and safety operations autonomously.

Edge ML: Practical Quantization, Pruning, and Latency Trade-Offs

Rafia — Thu, 22 Oct 2026 13:49:52 +0000

Deploying machine learning models to microcontrollers and low-power embedded systems is fundamentally an exercise in resource accounting. Unlike server environments where compute is bounded by cost or thermal design power, edge devices operate under hard physical walls. An ARM Cortex-M4 or Cortex-M7 microcontroller often offers less than 1 MB of SRAM and a few megabytes of non-volatile flash storage. Even embedded Linux platforms, such as the Raspberry Pi Zero or entry-level NXP i.MX boards, face strict memory bandwidth bottlenecks and tight power limits.

To make deep neural networks execute reliably under these constraints, we rely on model compression techniques—primarily quantization and pruning. However, these techniques introduce non-linear trade-offs between precision, flash footprint, runtime memory usage, and operational latency.

The Memory Hierarchy on Microcontrollers

When optimizing a model for embedded hardware, distinguishing between static storage and dynamic memory is essential.

Flash memory holds the static model parameters (weights, biases) and the compiled program code. Read operations from flash are relatively fast, but writing during inference is not an option.

SRAM holds the runtime activations, the working tensor arena, and the stack/heap. SRAM is often the primary bottleneck. If a model's peak activation layer exceeds the allocated tensor arena size, the runtime aborts with an out-of-memory error. Because dynamic memory allocation (malloc) is discouraged or entirely disabled in real-time embedded environments, the entire activation arena must be allocated statically ahead of time.

This means that shrinking weights alone does not guarantee that a model can run on a microcontroller. The maximum activation tensor generated during the forward pass must fit within available SRAM alongside input/output buffers and peripheral drivers.

Quantization: INT8 vs FP32

Quantization maps continuous 32-bit floating-point values ($FP32$) to lower-bit representations, most commonly 8-bit integers ($INT8$).

The standard uniform affine quantization scheme maps a real-valued number $x$ to an integer $q$ via a scale factor $S$ and a zero-point $Z$:

$$q = \text{round}\left(\frac{x}{S}\right) + Z$$

Where $S$ is a positive floating-point number, and $Z$ is an integer of the same type as $q$.

Post-Training Quantization (PTQ)

In PTQ, a fully trained $FP32$ network is converted to $INT8$ using a small calibration dataset. The calibration run profiles the activation distributions across intermediate layers to determine optimal values for $S$ and $Z$. PTQ is straightforward and avoids the cost of full retraining. However, highly non-linear networks or models with narrow activation dynamic ranges often suffer noticeable accuracy degradation under PTQ alone.

Quantization-Aware Training (QAT)

QAT models the quantization noise during the training process itself. Forward passes simulate lower precision using fake-quantization nodes that clamp and round values, while backpropagation uses standard floating-point operations with a straight-through estimator (STE). QAT almost always recovers accuracy lost in PTQ, especially for lightweight architectures like MobileNetV2 or custom depthwise-separable CNNs, which are sensitive to weight clipping.

Here is a standard example of generating an INT8 full-integer quantized model using TensorFlow Lite for Microcontrollers:

import tensorflow as tf

def representative_dataset():
    for data in calibration_data_loader:
        # Yield normalized input batches for calibration
        yield [data.astype(np.float32)]

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset

# Enforce full INT8 execution across all operations
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

quantized_tflite_model = converter.convert()

with open("model_quantized_int8.tflite", "wb") as f:
    f.write(quantized_tflite_model)

Moving to full integer inference provides two distinct hardware benefits:

Model size decreases by roughly $4\times$, directly reducing flash usage.
Integer matrix multiplication instructions (like ARM CMSIS-NN SIMD instructions __SMLAD) execute significantly faster than software-emulated or even hardware single-precision floating-point units.

Pruning: Structured vs Unstructured

Pruning eliminates parameters that contribute minimally to the network's predictive output. We categorize pruning into two forms:

Unstructured Pruning: Individual weights below a given threshold are set to zero anywhere in the weight tensor. While this achieves high theoretical sparsity (often $70\text{--}90\%$), it creates arbitrary zero-patterns. Standard microcontrollers and embedded runtimes cannot leverage random sparsity without specialized sparse-matrix runtime engines. In practice, unstructured pruning usually bloats memory metadata and fails to reduce execution latency.
Structured Pruning: Entire channels, filters, or attention heads are removed. Because the pruned network retains dense matrix structures, it executes directly on commodity inference engines without custom sparse libraries. The resulting layer tensors are simply smaller dense arrays, leading to verified decreases in both activation size and latency.

If the target deployment platform lacks hardware acceleration for sparse matrices, structured pruning is the only reliable choice for achieving measurable latency gains.

A Worked Memory and Latency Budget

Consider an embedded sound-classification model deployed on an ARM Cortex-M4 running at 64 MHz, with 512 KB Flash and 128 KB SRAM.

Assume the architecture has three 2D convolutional layers followed by global average pooling and a fully connected output:

Total parameters: 95,000 weights.
Input shape: 1 channel, $32 \times 32$ spectrogram ($1{,}024$ samples).
Layer 1: 16 filters, $3 \times 3$, stride 1, padding 'same'. Output: $32 \times 32 \times 16 = 16{,}384$ values.
Layer 2: 32 filters, $3 \times 3$, stride 2, padding 'same'. Output: $16 \times 16 \times 32 = 8{,}192$ values.
Layer 3: 64 filters, $3 \times 3$, stride 2, padding 'same'. Output: $8 \times 8 \times 64 = 4{,}096$ values.

Footprint Comparison

FP32 Baseline:

Flash requirement: $95{,}000 \times 4\text{ bytes} \approx 380\text{ KB}$. This consumes $74\%$ of total flash.
Peak activation buffer: Layer 1 requires $16{,}384 \times 4\text{ bytes} = 65.5\text{ KB}$.
Double buffering for layer transitions (Layer 1 output + Layer 2 output): $65.5\text{ KB} + 32.7\text{ KB} = 98.2\text{ KB}$.
SRAM overhead: $98.2\text{ KB}$ exceeds the safe SRAM budget when operating systems, peripheral buffers, and stacks need memory within the remaining $29.8\text{ KB}$.

INT8 Quantized:

Flash requirement: $95{,}000 \times 1\text{ byte} \approx 95\text{ KB}$ (plus quantization parameters). Flash usage drops to under $20\%$.
Peak activation buffer: Layer 1 requires $16{,}384 \times 1\text{ byte} = 16.4\text{ KB}$.
Double buffering requirement: $16.4\text{ KB} + 8.2\text{ KB} = 24.6\text{ KB}$.
SRAM headroom: $24.6\text{ KB}$ leaves more than $100\text{ KB}$ of SRAM available for sensor ingestion, circular audio buffers, and communication stacks.

Execution Latency

On the Cortex-M4, an FP32 multiply-accumulate loop takes roughly 4 to 6 clock cycles per operation when using software floating-point emulation or simple single-issue FP instructions. With CMSIS-NN and an INT8 SIMD instruction set, two 8-bit multiply-accumulate operations occur per cycle.

For a layer executing 1.5 million MAC operations:

$FP32$: $\approx 1.5\text{M} \times 5\text{ cycles} = 7.5\text{ million cycles} \approx 117\text{ ms}$ at 64 MHz.
$INT8$: $\approx (1.5\text{M} / 2) \times 1.2\text{ cycles} = 900{,}000\text{ cycles} \approx 14\text{ ms}$ at 64 MHz.

Quantization in this scenario changes the inference from unusable to well within a real-time 50 ms audio polling loop.

Benchmarking Considerations on Physical Boards

Simulator profiles frequently diverge from real-world behavior on bare-metal systems. Three issues consistently emerge:

Flash Wait States: Microcontroller clock frequencies often exceed internal flash access speeds. Reading weights directly from flash introduces wait states unless an internal cache or prefetch buffer is present. Benchmarking must occur on hardware running at target clock rates.
Memory Alignment: ARM architectures penalize unaligned 32-bit accesses. Ensure tensor arenas are explicitly aligned to 4-byte or 16-byte boundaries using compiler attributes such as alignas(16).
Fallback Operations: If a quantization converter encounters an unsupported layer, it may quietly insert a float conversion step. This causes intermediate conversions between INT8 and FP32, dramatically degrading latency without raising a build-time failure.

Key takeaways

SRAM capacity limits the maximum activation tensor size, not just total parameter count; profile peak layer outputs early.
Post-training INT8 quantization yields an immediate $4\times$ reduction in weight footprint and unlocks SIMD instruction acceleration on modern microcontrollers.
Unstructured pruning provides minimal benefits on standard edge runtimes; rely on structured pruning to achieve true latency reductions.
Always verify on-device operator support to prevent covert FP32 runtime fallbacks in compiled computation graphs.

What Is Physical AI? Perception, Planning, and Control Explained

Rafia — Sun, 18 Oct 2026 12:51:45 +0000

Most machine learning operates purely in the digital domain. You give a model a text prompt, an image, or a database record, and it returns a prediction or generated text. The cost of an error is usually an incorrect log entry or an invalid API response.

Physical AI is different. It refers to systems where AI models interact directly with the physical world through sensors and actuators. In these systems, software must obey physics, respect strict latency bounds, and operate under mechanical and thermodynamic constraints. If a physical AI system fails, it does not just return a 500 error—it can damage equipment or cause physical harm.

At its core, Physical AI is not a single giant foundation model running in isolation. It is the integration of three distinct stages: perception, planning, and control.

The Classical Feedback Loop

To understand Physical AI, you have to look at the sense-plan-act loop that robotics has used for decades. Physical AI modernizes each stage of this loop with learning-based techniques, but the fundamental topology remains cyclical.

+------------+     Raw Sensors      +----------------+
|  Physical  | -------------------> |   Perception   |
|   World    |  (Cameras, IMU, etc) | (State Estim.) |
+------------+                      +----------------+
      ^                                     |
      | Actuator Commands                   v Current State
+------------+                      +----------------+
|  Control   | <------------------- |    Planning    |
| (PID, MPC) |     Target State     | (Trajectories) |
+------------+                      +----------------+

Let us break down how modern software and data pipelines handle each stage.

1. Perception: From Sensor Signals to State Estimates

Perception is the translation of noisy physical signals into a coherent estimate of the environment and the system's own state.

Physical agents rely on diverse sensor suites:

Inertial Measurement Units (IMUs): Accelerometers and gyroscopes operating at high frequencies (typically 200 Hz to 1 kHz).
Cameras: RGB and monochrome streams providing dense semantic data, usually at 30 to 60 Hz.
LiDAR and Radar: Direct time-of-flight depth measurements.
Encoders: Measurements of joint angles and wheel rotations.

Traditional systems rely on Extended Kalman Filters (EKF) or factor graphs to fuse these sources. Physical AI augments or replaces heuristic feature extraction with deep neural networks. For example, instead of relying on manually tuned computer vision pipelines to detect obstacles, convolutional nets or vision transformers produce 3D bounding boxes, occupancy grids, or point-cloud segmentations directly.

However, neural networks introduce latency and non-deterministic jitter. Raw camera frames cannot be fed directly into a motor controller operating at 500 Hz. Instead, the perception pipeline outputs a structured state estimate—such as coordinates, velocities, and semantic maps—which downstream systems consume.

2. Planning: Computing Safe Trajectories

Once the system understands where it is and where obstacles are, it must decide what to do next. This is the planning phase.

Planning generally happens in two stages: global planning and local planning.

Global Planning: Solves high-level routing. For an autonomous vehicle, this is finding a path across a road network. For a robotic arm, this is finding an inverse kinematics path from point A to point B without colliding with fixed geometry.
Local Planning: Evaluates the immediate trajectory over the next 1 to 5 seconds. It computes velocity profiles and lane assignments while accounting for dynamic obstacles.

In Physical AI, reinforcement learning (RL) and neural trajectory generators are often used here. Rather than running exhaustive graph-search algorithms (like A* or RRT*), a policy network can output a parameterized spline or a set of candidate trajectories that satisfy kinematic limits like maximum acceleration and jerk.

3. Control: Closing the Loop with the Hardware

Planning generates the desired trajectory, but physical systems never follow plans perfectly. Wind, friction, actuator backlash, and battery voltage drops create discrepancies between plan and reality. Control is the layer that enforces the plan in real time.

Unlike perception and planning—which often run at 10 to 60 Hz—control loops typically run at 200 Hz to several kHz on bare-metal microcontrollers or real-time operating systems (RTOS).

Engineers typically use one of two paradigms here:

Model Predictive Control (MPC): Solves an online optimization problem at every time step to find the optimal control inputs over a short horizon while respecting constraints (e.g., maximum torque).
Proportional-Integral-Derivative (PID) and LQR: Fast, reliable linear controllers that compute error signals and drive motors to minimize deviations.

While some end-to-end research models try to predict motor torques directly from camera pixels, production systems rarely do this. If a neural network hallucinates a value for a single frame, a high-gain motor controller could snap an axle or burn out a winding. Modern Physical AI keeps low-level control deterministic and bounded, using AI to inform setpoints rather than raw signals.

Concrete Example: A Closed-Loop Tracking System

Here is a minimal Python example demonstrating the interaction between a physical plant, a sensor with measurement noise, a basic state estimator, and a controller. It models a 1D cart being driven to a setpoint.

import numpy as np

class CartSystem:
    def __init__(self, mass=2.0, dt=0.01):
        self.m = mass
        self.dt = dt
        self.pos = 0.0
        self.vel = 0.0

    def step(self, force):
        # Physics update with simple drag
        accel = (force - 0.1 * self.vel) / self.m
        self.vel += accel * self.dt
        self.pos += self.vel * self.dt
        return self.pos

# Simulation setup
sim = CartSystem()
target_pos = 5.0
estimated_pos = 0.0
kp, kd = 25.0, 10.0
prev_error = target_pos

for step in range(500):
    # 1. Perception: read noisy sensor and update state
    noisy_measurement = sim.pos + np.random.normal(0, 0.05)
    estimated_pos = 0.8 * estimated_pos + 0.2 * noisy_measurement

    # 2. Planning: determine instantaneous error to target
    error = target_pos - estimated_pos
    deriv = (error - prev_error) / sim.dt
    prev_error = error

    # 3. Control: compute actuator force (PD law with saturation limit)
    force = np.clip(kp * error + kd * deriv, -20.0, 20.0)

    # 4. Actuation: apply to hardware
    actual_pos = sim.step(force)

print(f"Final Cart Position: {actual_pos:.3f} m (Target: {target_pos} m)")

In this example, the PD controller and physics update run in lockstep. In a real Physical AI system, the target position would be generated dynamically by a neural planner evaluating visual data, while the low-level loop maintains stability.

Edge Hardware and Latency Constraints

Deploying Physical AI introduces constraints that web servers never face:

Thermal envelopes: A mobile robot cannot run four 400-watt GPUs. It must run on embedded system-on-chips (SoCs) operating between 10W and 60W.
Quantization: Weights must often be quantized to INT8 or FP8 to fit within SRAM and utilize specialized neural processing units (NPUs).
Worst-case execution time (WCET): For cloud apps, the 99th percentile response time matters. In physical control, the absolute worst-case latency matters. If a loop misses a deadline because garbage collection triggered or a layer stalled, the system can become dynamically unstable.

Physical AI is engineering across the entire stack. Success requires balancing model capacity against memory bandwidth, sensor throughput, and actuator dynamics.

Key takeaways

Separation of concerns: Modern Physical AI divides tasks into perception (state estimation), planning (trajectory generation), and control (actuation) rather than relying exclusively on end-to-end black boxes.
Time scale divergence: Perception and high-level planning typically run at low frequencies (10–60 Hz), while low-level control loops must run deterministically at hundreds or thousands of Hertz.
Deterministic safety: Low-level controllers usually maintain hard mathematical bounds on torques, velocities, and currents to prevent AI policy hallucinations from damaging hardware.
Hardware limitations dictate architecture: Embedded power consumption, memory bandwidth, and strict latency ceilings are core constraints when designing Physical AI models.

Drafted with the help of an AI model from my own topic notes, then reviewed and edited by me before publishing.