Many pieces about “AIoT” only touch the surface: AI + IoT = Smart. But if you’re an engineer, setting up sensors to run in models is interesting because the pipeline between the device and the model lies in the engineering, the glue, between what’s on the ground and what can be predicted. When this glue fails, how good your model can be is irrelevant.
In this article, we go over system architecture models which do hold in a real production scenario and the common mistakes companies make when upgrading from the prototype level to production on a factory or warehouse floor.
Why are industrial data pipelines different from other IoT pipelines?
Consumer IoT pipelines are typically more forgiving: the odd minutes where the smart thermostat doesn’t get readings won’t cause a large impact. But this isn’t an option with industrial pipelines. When the sensor collecting vibrations from the compressor stops working, or a frame is dropped from a safety camera, it could very well be that the one data point that predicted an incident or failure.
This changes the system engineering requirements for everything:
The equipment might not have a reliable network, and it’s usually constrained. (industrial Wi-Fi, LTE-M, LoRaWAN)
Data should survive intermittence, not silent errors.
Requirements on speed vary depending on how it is used, with alerts for safety requiring <1 sec to trigger and trends for inventory only minutes late
Model implementation can be anything between edge and cloud, and is not fixed on the same place
An Example of an Architecture
An example that generalizes across multiple uses (asset tracking, employee safety, equipment monitoring):
The Edge layer – devices(sensors,RFID reader, cameras, PLCs) pushrawdata to a local Gateway.
The Edge layer preprocessing – filters, dedup, some mini model (maybe anomaly flags) is implemented on the device itself to reduce the network load and latency.
Buffered transport layer – a message queue (MQTT) usually includes local buffering to prevent losing data during network disruptions.
The data collection layer – incoming payloads are mapped to a standard scheme, as vendors have a tendency to report data differently.
A Time Series Database – designed for this specific use case (e.g., InfluxDB, TimescaleDB), unlike generic databases. This is because queries usually span time ranges.
The Model – Batch and/or stream inference, depending on what is required. Predictions are sent to an operational interface or automatically trigger actions.
Many people will bypass the whole stack to get raw sensor data to their models. This works when it’s just one device type under ideal circumstances. Once they introduce a different vendor's equipment or the network gets unreliable, it breaks down.
Edge vs. Cloud: When to Implement Models?
This is the big question I get, and the only accurate answer is that it depends on what risk you care about the most.
If the risk is in the event of not detecting it (e.g., employee safety, access control), it has to be run at the edge. It should not depend on a connection with the cloud.
If the risk lies in analysing trend data (e.g. Optimizing inventory across an entire warehouse), a batch inference in the cloud makes more sense. This is also typically easier to deploy upgrades.
Often, we encounter situations with both the so-called “hybrid setup”: a small edge model to flag urgent events, and a more powerful model in the cloud to perform deeper analyses.
The Problem of Data Quality
Data from industrial sensors is different from anything you find in tutorials. Sensors can drift and go out of calibration, devices may move location and the information on that can lag, and times are sometimes received out of order on different systems. Any model that assumes input is clean, ordered, and labelled properly will get faulty data long before it’s discovered.
Real-world solutions:
- Devices metadata can be versioned separately from sensor readings to prevent misinterpreting a shift in data as a problem after recalibrating the sensor or moving it
- Drift detection must be built in the system and not just into the model - identify when the baseline of a sensor is shifting before it impacts model performance
- Assume that receiving messages out of order is the standard case, not the exception, and plan your TSDB write actions appropriately
This Leads to…
These technologies are not complex; they've been around for a while (MQTT, time series DB, edge inference). What’s difficult with industrial AIoT is discipline-you need a data pipeline that doesn’t fall apart in the face of real network conditions, mismatched equipment, and dirty data. It must be built not for a clean simulation but for the realities on the ground.
Any team that has gone through the process of integrating systems in the past has a leg up over anyone just beginning from scratch. This is also the reason we see more and more AIoT solutions developed within large infrastructure providers as opposed to starting from zero in startups.
Aperture Venture Studio has already covered this strategy in detail.
If you are embarking on a similar venture and are experiencing challenges with the cloud/edge partitioning, buffers, or data quality issues that have been described here, I’d love to hear how others have tackled these. Drop your thoughts in the comments below!
Top comments (0)