DEV Community: ReductStore

Persistent Edge Storage for a Legacy IIoT System

Alexey Timin — Fri, 29 May 2026 00:00:00 +0000

In real-life projects, nothing is perfect. But usually we don't touch systems that already work. Especially when a customer has already invested almost a million euros into that system.

When you come with your solution, you don't have full freedom to change everything as you wish. You cannot just say: "Let's replace this part", "Let's

change the architecture", or "Let's use a better protocol". Maybe technically it would be better, but in reality the system is already there, people use

it, and the customer paid a lot of money for it.

So you have to adjust your solution to the existing system. You have to be flexible, understand the constraints, and adapt to the customer's needs.

This is a real story about how we enabled long-term persistent storage with ReductStore for a customer with a legacy IIoT system.

No persistent storage, no access to raw data

The customer had a pretty typical IoT setup:

Fleet of IoT devices which collect data from vibration sensors, do analysis on the edge, and send the results to the cloud via MQTT
Cloud backend based on GCP: Hono for authentication and data ingestion from MQTT to Pub/Sub, Cloud Run for processing, BigQuery for storage and analysis
Grafana for visualization and alerting

The system worked pretty well. But the customer wanted to store raw vibration data for training and validation of AI models. They didn't need all of it—only data collected when production was running at stable speed. That's what was useful for their models.

The current setup was designed for structured data and didn't have a data flow for raw data. On top of that, many devices had connectivity issues and couldn't store data persistently—so the data was lost when the connection went down. The IoT team had to copy data manually from devices after outages, which was a huge pain and still caused data loss.

This would be a perfect use case for ReductStore, if we had a chance to replace the existing MQTT → Hono → Pub/Sub flow with ReductStore directly. But the customer didn't want to change the architecture or have an additional service available publicly. They wanted us to integrate ReductStore into the existing system, so that raw data is stored in ReductStore on the edge, replicated to a cloud ReductStore instance, and also sent to Pub/Sub for processing and BigQuery storage.

So what we had to do:

Store all raw data persistently on edge with ReductStore
Replicate data from edge to a cloud ReductStore instance
Re-send all data to the existing cloud backend after connectivity outages
Store vibration data in the cloud only for certain conditions—when production is running at stable speed

Reliable delivery to the cloud

Storing all MQTT data locally in ReductStore wasn't a problem. We wrote a simple Python service that subscribes to MQTT and stores all messages in ReductStore with necessary labels. The system was based on MQTT v5 and used user properties to pass metadata, so we just extracted them and used them as labels in ReductStore.

The real challenge was how to re-send data to the cloud after connectivity issues.

The data lands in BigQuery, and BigQuery doesn't protect against duplicates, so we had to make sure we don't send the same data twice. If we could use ReductStore's replication directly, it wouldn't be an issue at all—we could create a replication task with a filter for the relevant labels, and it would take care of everything. But we had to send data via MQTT, so we had to be more creative. We explored two options:

Continuous Queries

ReductStore has an option to query data continuously with a filter. You specify a beginning timestamp, and it returns all data matching the filter that arrives after that timestamp. We could use this to re-send data to the cloud after connectivity issues.

However, if a device is restarted, it loses the timestamp of the last sent message, and we would have to re-send all data from the beginning. That's not acceptable.

We could store the timestamp in a file, but that's a bit hacky and not very reliable.

Another option would be to update labels of sent messages with a "sent" flag and use it in the filter. But that solution is also not ideal:

Updating labels on every message is not very efficient and could cause performance issues.
You still need to query all messages from the beginning to find the last sent one, which is also not efficient.

Looks like standard ReductStore features are not enough to solve this problem, so we had to come up with a custom solution.

Replication to a fake target

ReductStore has replication tasks to replicate data to another instance, selecting by labels. It has a transaction log and guarantees that all data is replicated in order and without duplicates.

But we have to send data via MQTT!? Actually, it's not a problem if you know how things work. We can create a middleman service which implements the necessary HTTP endpoints for replication and sends data to the cloud via MQTT.

This is the approach we chose, and it was the right one.

Building the HTTP-to-MQTT bridge

We already had an MQTT bridge service that sent MQTT messages from internal services to the cloud, so we just had to rework it into an HTTP service. It needed to receive HTTP requests from ReductStore's replication task, extract the data and labels, and send them to the cloud via MQTT.

In other words, the MQTT bridge becomes the replication target—ReductStore thinks it's talking to another ReductStore instance, but behind those endpoints sits our bridge that forwards everything over MQTT.

The diagram shows the data flow: vibration data enters the Connector, which writes it into a ReductStore bucket. The replication task inside ReductStore pushes batches of records to the MQTT bridge's HTTP API, and the bridge forwards them to the cloud over MQTT.

The replication task is configured on the edge ReductStore instance with a target URL pointing to the MQTT bridge's HTTP endpoint and a label filter that selects only analytics data collected during stable production speed. This way, only relevant records get replicated to the cloud—raw vibration data and irrelevant measurements stay on disk locally.

To imitate a target ReductStore instance, the service must implement the following HTTP endpoints:

GET /api/v1/b/:bucket_name - to get information about the bucket to initialize replication
POST /api/v1/b/:bucket_name/:entry_name/batch - to receive data from the replication task and send it to the cloud via MQTT

It is not much, but data comes in batches, so we had to parse the HTTP headers and extract data from the HTTP body before sending it to the cloud via MQTT. The batch protocol is documented here: https://www.reduct.store/docs/next/http-api/entry-api#batch-protocol.

If the MQTT bridge is able to send data to the cloud, it returns 200 OK, and the replication task removes the records from the transaction log. So we have a guarantee that data is sent to the cloud without duplicates and in order, and we don't have to worry about connectivity issues at all.

In case of connectivity issues, the MQTT bridge returns errors per record according to the batch protocol. If the connection drops mid-batch, successfully sent records are acknowledged and only the failed ones are retried by the replication task. This means no data is sent twice and no records are lost—even on partial failures.

Data is stored persistently on disk and keeps accumulating during outages. ReductStore uses a FIFO quota: once the disk quota is reached, the oldest records are evicted to make room for new ones. So data loss only happens if an outage lasts long enough for the quota to roll over unsent records—in practice, with a reasonable disk size and typical outage durations, this never happened.

What we achieved

Before this project, the customer had two painful gaps: raw vibration data was never stored—it was processed on the edge and discarded immediately, so there was no way to go back and use it for model training later. And processed MQTT data was regularly lost during the frequent connectivity outages their edge devices experienced. The IoT team spent hours after every outage manually pulling data from devices—and still lost some of it.

With ReductStore on the edge and a thin HTTP-to-MQTT bridge, both problems are gone. Raw vibration data is now persisted in ReductStore on each device and can be queried directly over HTTP whenever the data science team needs it for model training or validation. There is no need to push gigabytes of raw sensor readings to the cloud—it stays where it's most useful and cheapest to store.

For processed data that must reach the cloud, the replication task handles everything automatically. Data is persisted on disk the moment it arrives, and once connectivity returns the bridge drains the backlog in order, without duplicates, without manual intervention. The FIFO quota ensures the disk never fills uncontrollably, and label-based filtering means only data collected under relevant production conditions is replicated—everything else stays local.

The most important part: none of this required changes to the customer's existing cloud infrastructure. The MQTT → Hono → Pub/Sub → BigQuery pipeline remained untouched. The bridge simply slots in as a transparent layer between ReductStore and the existing MQTT ingestion point. For the cloud backend, nothing changed—it still receives MQTT messages in the same format as before.

How to Store and Manage Robotics Data

AnthonyCvn — Tue, 14 Apr 2026 07:53:15 +0000

Robots generate massive amounts of data, and managing it well is harder than it looks. Storage fills up fast, cloud transfer gets expensive, and real time ingestion is unforgiving when you're running cameras and sensors at high frequency.

This article covers practical strategies for handling robotic data, introduces ReductStore, and walks through a hands on example. Along the way, we cover native ROS integration, Grafana dashboards, MCAP export for Foxglove, a Zenoh API, and native S3 and Azure backends. We also compare ReductStore against Rosbag and MongoDB so you can pick the right tool for each part of your stack.

Challenges in Robotic Data Management

Robots operate in dynamic environments and continuously produce large volumes of data. The core challenges engineers run into are:

High frequency, real time requirements: A drone navigating a city must process camera and sensor data in milliseconds. Storage solutions need to keep up with these streams and make data accessible fast enough for real time decision making.
Limited on device storage: Most robots can't store everything they generate. Size, weight, and power constraints limit local capacity, so good data management strategies are essential.
High data volume: An autonomous vehicle can produce up to 5 terabytes per hour across camera feeds, LiDAR, radar, GPS, and sensors. Most databases weren't designed around that kind of throughput.
Cloud storage costs: Pushing everything to the cloud isn't practical. Providers charge per gigabyte stored and transferred, and with robots producing terabytes, the bill adds up fast. What you send to the cloud is a decision worth making deliberately.
Data reduction complexity: Filtering out irrelevant data without losing important information is tricky. Without a clear strategy, you either waste storage or throw away data you later need.

General Strategies for Managing Robotic Data

A few approaches that work well in practice:

Use a time series object store: Systems like ReductStore are designed specifically for high frequency, timestamped binary data. They offer fast writes, efficient querying, and manageable costs at scale.
Balance edge and cloud storage: Keep recent, critical data on the robot for fast access. Move older or lower priority data to the cloud for long term storage and analysis. Splitting storage this way cuts costs without adding much latency to what matters.
Use volume based retention policies: Rather than deleting data based on age, use a FIFO (first in, first out) quota. Data is only removed when the storage limit is reached, which avoids unnecessary deletion during downtime or low activity periods.
Compress where it makes sense: H.265 for video, JPEG for images. The right format can cut storage by an order of magnitude without losing anything you actually need.
Prioritize what to keep: Not all sensor data has equal value. Define what's critical upfront and discard the rest early in the pipeline.

ReductStore: Built for Robotic Data

ReductStore is a time series database built specifically for unstructured, binary data. It's designed to handle high frequency sensor streams from autonomous vehicles, drones, industrial robots, and IoT. It stores data with timestamps and labels, supports fast real time ingestion, and comes with batching, filtering, and edge to cloud replication built in.

Here's what's available today.

ROS Integration with reduct-bridge

reduct-bridge connects live ROS 1 and ROS 2 systems directly to ReductStore. You configure it with a simple TOML file that defines your ROS topic inputs, pipelines, and the ReductStore destination. It subscribes to topics and stores each message as an individual record.

It also writes a $ros attachment to each entry with the message schema, topic name, and encoding — that metadata is what powers MCAP export and Grafana visualization, both covered below. If your stack is already on ROS, setup takes minutes.

Cloud Storage: Native S3 and Azure

ReductStore supports native cloud storage backends for both Amazon S3 and Azure Blob Storage, no FUSE drivers needed. It uses a local cache for hot data and the cloud bucket for long term retention.

Option	What it means
S3 compatible	Works with AWS S3, MinIO, Ceph, Cloudflare R2, and any S3 compatible service
Azure Blob Storage	Switch backends by changing a few environment variables. Same deployment patterns.
Tiered access	Recent data stays in the local cache for fast reads. Older data lives in the cloud bucket.
Read replicas	Add read only replicas pointing at the same S3 bucket, close to your consumers.

For records around 100KB (e.g. JPEG images), ReductStore can be >10x faster than traditional time series object stores at a fraction of the cost.

Visualize ROS Data in Grafana

ReductStore has a Grafana integration through the ReductStore data source plugin. You can query ROS 2 messages directly in Grafana thanks to the ReductROS extension. The extension decodes binary CDR messages into JSON on the fly.

You can monitor sensor streams live, compare data across multiple robots, and set up alerts when metrics drift.

Export to MCAP for Foxglove

You can export raw ROS messages stored in ReductStore directly to MCAP and open them in Foxglove. This is powered by the ReductROS extension as well, which reconstructs valid MCAP files on demand.

When records are ingested via reduct-bridge, the $ros attachment carries the schema and topic information. The extension uses this to reconstruct valid MCAP files on demand. You can cover a full time range across multiple topics in a single query, and split by duration or file size for long recordings.

Zenoh Native API

ReductStore now includes a Zenoh native API alongside the existing HTTP API. Zenoh is a pub/sub protocol designed for robotics and distributed systems, and it's widely used in next generation ROS 2 deployments.

When enabled, ReductStore opens a Zenoh session with a subscriber for writes and a queryable for reads, both running in parallel with HTTP and sharing the same data. Zenoh keys become entry names, encodings map to content types, and attachments map to labels. Time range queries, conditional filters, and label lookups all work the same way they do over HTTP.

It's a natural fit if your robot stack already speaks Zenoh, or if you want to skip HTTP overhead on high frequency ingestion paths.

Query Language and Batching

ReductStore uses a JSON based query language that supports filtering, aggregation, and time range operations. You can query data from a specific robot within a time window, compare sensor streams across robots in parallel, or filter by any label.

Retrieval is optimized through batching: multiple records are grouped into a single response based on a time range, which cuts down on request overhead and improves throughput. SDKs are available for Python, C++, JavaScript, Go, and Rust. More in the querying guide.

Replication and Edge to Cloud Replication

ReductStore replicates at the bucket level based on conditions. That means you can choose to send only high priority sensor data to the cloud based on rules, labels, or events. Replication is incremental, so only new data is transferred. Labels stored alongside records let you define fine grained rules based on content, for example, only replicate records flagged as anomalies or sampled at 1 in 10 seconds.

Retention Strategies

Volume based retention follows the FIFO principle: data is only deleted when storage is full, making room for new records. This is different from time based retention, which deletes data after a fixed age regardless of whether storage is full. After an outage, a time based policy might delete data the system never had the chance to process. Volume based retention avoids that.

Example Applications

Application	How ReductStore helps
Autonomous Vehicles	Handles high throughput sensor streams with efficient querying and selective edge to cloud replication
Industrial Robots	Stores diagnostic data for predictive maintenance and continuous performance monitoring
Drones and UAVs	Supports offline operation in remote areas and syncs data when connectivity is restored

Comparing ReductStore with Rosbag and MongoDB

Three tools come up most often when robotics teams think about data storage: Rosbag and MCAP, MongoDB, and ReductStore. They're built for different things.

	Rosbag / MCAP	MongoDB	ReductStore
Data type	Binary ROS messages	Semi structured documents	Binary timestamped records
Query across recordings	No	Yes	Yes
Large binary payloads	Yes	Slow via GridFS	Yes, natively
Retention	Manual file management	Time based	Volume based FIFO
ROS integration	Native	None	Via reduct-bridge
Cloud storage	No	Atlas	Native S3 and Azure
Write (100 KB blobs)	—	694 blob/s	3,612 blob/s (+420%)
Read (100 KB blobs)	—	1,730 blob/s	6,250 blob/s (+260%)

Rosbag and MCAP are the standard for recording ROS sessions. They're great for capturing a snapshot during a test run. But they're file formats, not databases. Querying across many recordings requires custom scripts, there's no built in content indexing, and managing thousands of bag files quickly becomes its own problem. Use them for short recordings and local playback, not long term storage or fleet wide analysis.

MongoDB is flexible and works well for structured metadata, labels, and event logs. But it wasn't built for large binary payloads. For blob data, it relies on GridFS, which adds complexity and hurts performance. It also uses time based retention, so data can be deleted during idle periods even when storage isn't full.

ReductStore is designed specifically for the data robots actually produce: large, binary, timestamped records at high frequency. It connects to ROS via reduct-bridge, exports to MCAP for Foxglove, plugs into Grafana, and replicates selectively to S3 or Azure.

In practice: use Rosbag or MCAP for short test recordings, MongoDB for structured metadata and event logs, and ReductStore for raw sensor data that needs to be stored at scale, queried efficiently, and managed over time.

For a deeper look: MongoDB vs ReductStore: Choosing the Right Database for Robotics Applications.

Hands-On Example: Storing Robotic Data in ReductStore

Let's walk through a practical example of storing and querying robotic data with ReductStore. We'll use trajectory data (coordinates, speed, orientation) to keep things simple, but the same approach works for any sensor stream.

You'll need Python 3.8+ installed.

Setting Up ReductStore

Create a folder and add a docker-compose.yaml file:

version: "3.8"

services:
  reductstore:
    image: reduct/store:latest
    ports:
      - "8383:8383"
    volumes:
      - data:/data
    environment:
      - RS_API_TOKEN=my-token

volumes:
  data:
    driver: local

Start it with:

docker compose up -d

ReductStore will be available at http://127.0.0.1:8383. Check the container is running with docker ps, then install the Python libraries:

pip install reduct-py numpy

Store and Query Data

Create a Bucket

async def create_trajectory_bucket():
 async with Client("http://localhost:8383", api_token="my-token") as client:
  settings = BucketSettings(
     quota_type=QuotaType.FIFO,
     quota_size=1000_000_000,
    )
  await client.create_bucket("trajectory_data", settings, exist_ok=True)

The bucket uses a FIFO quota of 1 GB. Old data is only deleted when the limit is reached.

Generate Trajectory Data

async def generate_trajectory_data(frequency: int = 10, duration: int = 1):
 interval = 1 / frequency
 start_time = datetime.now()

  for i in range(frequency * duration):
    time_step = i * interval
    x = np.sin(2 * np.pi * time_step) + 0.2 * np.random.randn()
    y = np.cos(2 * np.pi * time_step) + 0.2 * np.random.randn()
    yaw = np.degrees(np.arctan2(y, x)) + np.random.uniform(-5, 5)
    speed = abs(np.sin(2 * np.pi * time_step)) + 0.1 * np.random.randn()
    timestamp = start_time + timedelta(seconds=time_step)

    yield {
      "timestamp": timestamp.isoformat(),
      "position": {"x": round(x, 2), "y": round(y, 2)},
      "orientation": {"yaw": round(yaw, 2)},
      "speed": round(speed, 2),
    }
    await asyncio.sleep(interval)

This simulates a robot moving in 2D at 10 Hz for 1 second. X and y are position coordinates, yaw is orientation, and speed is derived from position changes.

Calculate Metrics

def calculate_trajectory_metrics(trajectory: list) -> tuple:
  positions = np.array([[point["position"]["x"], point["position"]["y"]] for   point in trajectory])
  speeds = np.array([point["speed"] for point in trajectory])

  deltas = np.diff(positions, axis=0)
  distances = np.sqrt(np.sum(deltas**2, axis=1))
  total_distance = np.sum(distances)

  average_speed = np.mean(speeds)

  return total_distance, average_speed

Write to ReductStore

async def store_trajectory_data():
  trajectory_data = []
  async for data_point in generate_trajectory_data(frequency=10, duration=1):
    trajectory_data.append(data_point)

  total_distance, average_speed = calculate_trajectory_metrics(trajectory_data)

  labels = {
    "total_distance": total_distance,
    "average_speed": average_speed
  }

  packed_data = pack_trajectory_data(trajectory_data)

  timestamp = datetime.now()

  async with Client("http://localhost:8383", api_token="my-token") as client:
    bucket = await client.get_bucket("my-bucket")
    await bucket.write("trajectory", packed_data, timestamp, labels=labels)


def pack_trajectory_data(trajectory: list) -> bytes:
  """Pack trajectory data json format"""
  return json.dumps(trajectory).encode("utf-8")

The labels (total distance and average speed) are stored alongside each record. You can later filter or replicate records based on these values.

Once data is written, you'll see the bucket populate in ReductStore:

Query by Label

async def query_by_label(bucket_name, entry_name, label_key, label_value):
  async with Client("http://localhost:8383", api_token="my-token") as client:
    bucket = await client.get_bucket(bucket_name)

    async for record in bucket.query(
      entry_name,
      when={
        label_key: {"$gt": label_value}
      },
    ):
      # Do something with the record

Remove the 'when' condition to return all records with no filtering.

Main Function

async def main():
  await create_trajectory_bucket()
    await store_trajectory_data()
    label_query_result = await query_by_label("my-bucket", "trajectory", "&total_distance", HIGH_DISTANCE)
    if label_query_result:
      print(f"Data queried by label: {label_query_result}")

asyncio.run(main())

Conclusion

Robotics data pipelines don't have to be a mess of bag files, overpriced cloud storage, and custom scripts duct taped together. ReductStore covers ingestion, retention, replication, and querying in one place, with direct integrations into ROS, Foxglove, Grafana, S3, and Azure.

Start with what you need, and add the rest as your system grows. Check out reduct.store or read through the documentation to get going.

If you have any questions or comments, feel free to use the ReductStore Community Forum.

ReductStore v1.19: Open Data Backbone for Robotics and ROS

Alexey Timin — Wed, 08 Apr 2026 00:00:00 +0000

ReductStore 1.19.0 is now available. This release extends the storage model for robotics and telemetry workloads and introduces new integration points for ROS and Zenoh.

To download the latest release, visit the Download Page.

What's new in 1.19.0?

The first major change in v1.19 is licensing. ReductStore Core is now open source under Apache 2.0, which makes the core database easier to evaluate, integrate, and extend in production systems.

The second major change is the data model. ReductStore now supports hierarchical entry names, similar to ROS topics, and adds entry attachments for schemas and metadata. This makes it possible to represent structured robotics data without flattening topic hierarchies or moving context into external systems.

The release also introduces a native Zenoh API for direct ingestion and querying over Zenoh, and ReductBridge for ROS1 and ROS2 integration.

Nested Data Model with Attachments

The new hierarchical data model lets you organize data in a path-based structure, similar to ROS topics, Zenoh key expressions, or MQTT topics. Instead of relying on a flat namespace, ReductStore can now store data in a form that matches the structure used by upstream systems.

Each entry can also include attachments for schemas and metadata. These attachments preserve the context required by downstream tooling without changing the record payload itself. For example, the ROS Extension can use them to decode serialized ROS messages and export them to MCAP files.

Native Zenoh API

Zenoh is increasingly used in robotics and edge environments as a low-overhead protocol for distributed data exchange. With the native Zenoh API, ReductStore can participate directly in Zenoh-based systems without requiring an additional bridge or adapter.

You can start ReductStore with the Zenoh API enabled using a minimal Docker configuration:

docker pull reduct/store:v1.19.0docker run 
 --env "RS_ZENOH_ENABLED=ON" 
 --env "RS_ZENOH_CONFIG={}" 
 --env "RS_ZENOH_SUB_KEYEXPRS=**" 
 -p 8383:8383 -p 36597:36597 -p 7446:7446 reduct/store:v1.19.0

Once enabled, the API allows you to write data to ReductStore directly over Zenoh. If a sample includes a JSON attachment, ReductStore stores it as record labels:

import json
import zenoh

KEY = "factory/line1/camera"
PAYLOAD = b"<binary payload>"
LABELS = {"robot": "alpha", "status": "ok"}

with zenoh.open(zenoh.Config()) as session:
    session.put(
        KEY,
        PAYLOAD,
        attachment=json.dumps(LABELS).encode(),
    )

You can also query data through Zenoh. For conditional queries, pass a when expression in the query attachment. If you want all matching records returned individually, use zenoh.ConsolidationMode.NONE:

import json
import zenoh

KEY = "factory/line1/when-query"
CONSOLIDATION = zenoh.ConsolidationMode.NONE
attachment = json.dumps({"when": {"&status": {"$eq": "ok"}}}).encode()

with zenoh.open(zenoh.Config()) as session:
    replies = [
        reply
        for reply in session.get(
            KEY,
            timeout=5.0,
            attachment=attachment,
            consolidation=CONSOLIDATION,
        )
        if reply.ok
    ]

    for reply in replies:
        print(reply.ok.payload.to_bytes())

Data ingested through Zenoh is stored in ReductStore like data written through the HTTP API, so it remains available for querying, replication, and downstream tools and extensions.

ReductBridge for ROS Integration

This release also introduces ReductBridge, a new project for integrating ReductStore with ROS1 and ROS2. ReductBridge automatically labels ROS messages and stores related schemas and metadata as attachments.

This makes raw ROS payloads usable after ingestion: they can be decoded later with the ROS Extension or exported to MCAP files. ROS support is the first target, but the same pattern can be extended to operating system metrics, logs, and other telemetry to build a unified storage layer with consistent labeling and metadata.

What’s Next

The next area of work is data compression and storage efficiency, especially for robotics workloads where payload sizes and formats vary significantly. We plan to introduce backend compression optimized for ReductStore's storage model, where many records are packed into a single block.

We also plan to store metadata in Parquet format. This should improve query efficiency, make metadata accessible without scanning the underlying data blocks, and simplify integration with analytics and data lake tooling.

Compatibility and Migration

Starting with v1.19, ReductStore Docker images no longer run as root for security reasons. If you deploy with Docker, make sure the mounted data directory is writable by UID/GID 10001:10001 before upgrading.

If you have questions or feedback, join the ReductStore Community forum.

Thanks for using ReductStore!

ReductStore Core Adopts Apache 2.0 License

Alexey Timin — Tue, 17 Mar 2026 00:00:00 +0000

Hello, everyone!

Starting with ReductStore v1.19 , we are changing how we license and package the project. From this version onward, ReductStore is split into two editions: ReductStore Core and ReductStore Pro.

Previously, ReductStore shipped as a single edition under BUSL-1.1 (Business Source License). With v1.19, the core database is open source under Apache 2.0, while Pro continues under commercial terms.

What's changing

With this release, ReductStore Core becomes the open-source foundation of the project and is now available under the Apache License 2.0. It includes the database server and the core functionality that most users rely on for edge deployments and everyday data workflows.

Alongside Core, we will continue offering ReductStore Pro , which is distributed under a commercial license. The Pro edition provides additional capabilities and support options for teams with more advanced requirements. See Pricing for a clear comparison of what is included in each edition.

Why we're doing this

We want to better support open source communities and make it easier to contribute to, integrate with, and build on top of ReductStore Core. Just as importantly, we want users to be able to run ReductStore on the edge without licensing restrictions for the core database use cases.

At the same time, we want to keep a sustainable commercial model for companies that build more complex cloud setups or need advanced support and functionality around robotics and IIoT data formats. That is the role of ReductStore Pro.

What this means for you

If you use the core database functionality, you can freely adopt, integrate, and distribute ReductStore Core under the Apache 2.0 license. Existing workflows and upgrades remain straightforward.

For teams that rely on advanced functionality or require commercial support, ReductStore Pro remains available as the supported commercial edition. We will clearly document which features belong to Core and which are part of Pro in the documentation and release notes.

If you have questions or want to discuss which edition fits your use case, please reach out on the ReductStore Community forum or via our contact page.

Air-Gapped Drone Data Operations with Delayed Sync and Auditability

AnthonyCvn — Tue, 24 Feb 2026 00:00:00 +0000

Drones in air-gapped environments produce a lot of data (camera images, telemetry, logs, model outputs). Storing this data reliably on each drone and syncing it to a ground station later can be hard. ReductStore makes this easier: it's a lightweight, time-series object store that works offline and replicate data when a connection is available.

This guide explains a simple setup where each drone stores data locally with labels, replicates records to a ground station based on what it detects, and keeps a clear audit trail of what was captured and replicated.

What we'll cover:

Drone-to-Ground Architecture

The architecture has three main components:

Each drone runs a small ReductStore server to save images and telemetry locally on disk (this lets the drone operate fully offline).
A ground station runs a ReductStore instance that receives replicated data for analysis and archiving.
ReductStore replication tasks copy data from drone to ground based on labels and conditions (e.g., only records flagged as anomalies, plus context around them).

Each drone pushes its data to the ground whenever it is connected. If the network disconnects, replication continues when the drone reconnects. This approach provides offline capability, lets you decide which data to replicate, and keeps a clear record of what happened.

Setting Up the Drone Node

Start by running ReductStore on the drone's companion computer. Here is a minimal docker-compose.yml:

services:
  reductstore:
    image: reduct/store:latest
    ports:
      - "8383:8383"
    environment:
      RS_API_TOKEN: <DRONE_TOKEN>
      RS_BUCKET_1_NAME: mission-data
      RS_BUCKET_1_QUOTA_TYPE: FIFO
      RS_BUCKET_1_QUOTA_SIZE: 10000000000 # 10 GB
    volumes:
      - ./data:/data

Run it with:

docker compose up -d

This starts a ReductStore server with a mission-data bucket that uses FIFO retention. Old data is deleted only when the 10 GB limit is reached, so the drone always keeps as much history as possible.

FIFO quota is volume-based, not time-based. This means data is only deleted when disk space runs out, not after a fixed time period. This is important for drones that may sit idle between missions.

If you prefer Snap instead of Docker:

sudo snap install reductstore

That starts a ReductStore server on port 8383 by default. You can then create the bucket using the Reduct CLI:

reduct-cli alias add drone -L http://localhost:8383 -t "<DRONE_TOKEN>"
reduct-cli bucket create drone/mission-data --quota-type FIFO --quota-size 10GB

Storing Drone Data with Labels

Use labels to tag every record with mission context. This is what makes selective replication and auditing possible later. Here is an example using the Python SDK:

import asyncio
import time
import hashlib
from reduct import Client


async def main():
    async with Client("http://localhost:8383", api_token="<DRONE_TOKEN>") as client:
        bucket = await client.get_bucket("mission-data")

        # Read a camera frame
        payload = open("frame.jpg", "rb").read()
        checksum = hashlib.sha256(payload).hexdigest()
        timestamp = int(time.time() * 1_000_000)  # microseconds

        # Write with mission labels
        await bucket.write(
            "camera",
            payload,
            timestamp=timestamp,
            labels={
                "mission_id": "m-2026-02-24-01",
                "platform_id": "uav-07",
                "anomaly": "false",
                "confidence": "0.95",
                "checksum": checksum,
            },
            content_type="image/jpeg",
        )

        # Write telemetry as a CSV batch
        csv_data = "ts,lat,lon,alt,speed\n"
        csv_data += "1708771200000000,47.3769,8.5417,450.2,12.5\n"
        csv_data += "1708771201000000,47.3770,8.5418,451.0,12.8\n"

        await bucket.write(
            "telemetry",
            csv_data.encode(),
            timestamp=timestamp,
            labels={
                "mission_id": "m-2026-02-24-01",
                "platform_id": "uav-07",
                "anomaly": "false",
                "checksum": hashlib.sha256(csv_data.encode()).hexdigest(),
            },
            content_type="text/csv",
        )


asyncio.run(main())

The anomaly label is important: it lets the replication task decide what to sync based on what the drone actually sees. For example, if the drone detects something unusual (an object, a warning, a low confidence score), it sets anomaly=true. The replication task can then automatically sync that record — plus the context around it.

The checksum label gives you a simple way to verify data integrity during audits.

Setting Up Selective Replication

Once the drone connects to a trusted network, replication sends only the relevant records to the ground station. The simplest approach is to replicate based on a label, for example only records where the drone detected an anomaly:

reduct-cli alias add drone -L http://localhost:8383 -t "<DRONE_TOKEN>"

reduct-cli replica create drone/mission-to-ground \
    mission-data \
    https://<GROUND_TOKEN>@<ground-address>/drone-data \
    --when '{"&anomaly": {"$eq": "true"}}'

This creates a replication task that copies only records where anomaly=true from the drone's mission-data bucket to the ground station.

Replicating with context (before and after)

In many cases, you don't just want the anomaly record itself — you also want to see what happened before it. ReductStore supports this with the #ctx_before and #ctx_after directives. For example, to replicate each anomaly record plus 30 seconds of data before it and 10 seconds after:

{
  "&anomaly": { "$eq": "true" },
  "#ctx_before": "30s",
  "#ctx_after": "10s"
}

This is powerful for drone operations: imagine the drone's onboard model detects an unexpected object. ReductStore will replicate that record and the 30 seconds of camera frames leading up to the detection, so the ground team can review what happened.

You can provision this directly in Docker using environment variables:

services:
  reductstore:
    image: reduct/store:latest
    ports:
      - "8383:8383"
    environment:
      RS_API_TOKEN: <DRONE_TOKEN>
      RS_BUCKET_1_NAME: mission-data
      RS_BUCKET_1_QUOTA_TYPE: FIFO
      RS_BUCKET_1_QUOTA_SIZE: 10000000000
      RS_REPLICATION_1_NAME: mission-to-ground
      RS_REPLICATION_1_SRC_BUCKET: mission-data
      RS_REPLICATION_1_DST_BUCKET: drone-data
      RS_REPLICATION_1_DST_HOST: https://<ground-address>
      RS_REPLICATION_1_DST_TOKEN: <GROUND_TOKEN>
      RS_REPLICATION_1_WHEN: |
        {
          "&anomaly": { "$$eq": "true" },
          "#ctx_before": "30s",
          "#ctx_after": "10s"
        }
    volumes:
      - ./data:/data

With this setup, the drone can operate fully offline. Replication runs automatically when a connection is available and waits when it's not. It's also possible to pause replication tasks if needed. And because context is included, the ground team always has enough data to understand what triggered the event.

Querying for Audit Reports

After a mission, you can query the ground station to check what was captured and replicated. Here is a simple example that lists all records from a specific mission:

import asyncio
from reduct import Client


async def main():
    async with Client("https://<ground-address>", api_token="<GROUND_TOKEN>") as client:
        bucket = await client.get_bucket("drone-data")

        # Query all camera records from a specific mission
        async for record in bucket.query(
            "camera",
            when={"&mission_id": {"$eq": "m-2026-02-24-01"}},
        ):
            print(
                f"ts={record.timestamp}, "
                f"anomaly={record.labels.get('anomaly')}, "
                f"checksum={record.labels.get('checksum')}, "
                f"size={record.size}"
            )


asyncio.run(main())

This gives you a clear log of every record in that mission: timestamp, anomaly flag, checksum, and size. You can use this to verify that all expected data arrived on the ground side.

To go further, compare the checksums on the drone with the ground side to confirm nothing was altered during transfer. You can also check the error logs of the replication task to see if any records failed to replicate.

Why This Setup Works Well for Drones

Drones have specific constraints that general purpose databases don't handle well. Here is what makes this setup practical:

Full offline operation. Drones store everything locally and don't need a network connection during the mission. Data is safe on disk until sync happens.
Automatic sync when connected. When the drone lands or connects to a trusted network, replication picks up where it left off. No manual file transfers, no rsync scripts, no USB sticks.
Smart replication with context. You don't have to sync everything. The replication task filters by labels and can include past records around each event using #ctx_before. The ground team gets exactly what they need to understand what happened.
Disk never fills up unexpectedly. FIFO retention removes the oldest data only when the disk is full. The drone always keeps as much history as possible without running out of space mid mission.
Easy auditing. Every record has a timestamp, labels, and a checksum. After a mission, you can query the ground station and verify exactly what was captured and what was synced.
Store any file type. Camera frames, telemetry CSV, logs, MCAP files, model outputs. Everything goes into the same system with the same interface.

Next Steps

If you want to go deeper, check out these articles:

Distributed Storage in Mobile Robotics for a similar setup with mobile robots and S3 cloud backend
How to Store and Manage Robotics Data for a broader look at ReductStore features for robotics
Data Replication Guide for the full documentation on replication tasks, filters, and modes
Conditional Query Reference for all available conditional query operators you can use in replication filters and queries

I hope you found this article helpful! If you have any questions or feedback, don't hesitate to reach out on our ReductStore Community forum.

ReductStore v1.18.0 Released with Resilient Deployments and the Multi-entry API

Alexey Timin — Thu, 05 Feb 2026 00:00:00 +0000

We are pleased to announce the release of the latest minor version of ReductStore, 1.18.0. ReductStore is a high-performance storage and streaming solution designed for storing and managing large volumes of historical data.

To download the latest released version, please visit our Download Page.

What's new in 1.18.0?

In this release, we have added support for resilient deployments to build a more robust, fault-tolerant, and highly available ReductStore cluster. Now, you can implement hot-standby configurations, automatic failover, and seamless recovery to ensure uninterrupted service even in the face of hardware failures or network issues. You can also elastically scale read-only nodes to handle increased read workloads without impacting the performance of the primary nodes.

Additionally, we have introduced a new Multi-entry API that allows you to efficiently manage and query multiple entries in a single request. This API is designed to optimize performance and reduce latency when working with large datasets, making it easier to retrieve and manipulate data in bulk.

Resilient Deployments

In ReductStore v1.18.0, resilient deployments are now a first-class feature. Using the RS_INSTANCE_ROLE setting, you can build topologies that keep your ingestion endpoint available during node failures and scale reads independently from writes.

Hot standby (active-passive) for write availability

Run two nodes against the same backend (a shared filesystem or the same remote backend). Only one node is active at a time: the active node holds a lock file and refreshes it, while the standby waits and takes over when the lock becomes stale.

Set RS_INSTANCE_ROLE=PRIMARY for the active node and RS_INSTANCE_ROLE=SECONDARY for the standby.
Put both nodes behind a single virtual endpoint (load balancer / reverse proxy).
Route traffic only to the node that returns 200 OK on GET /api/v1/ready (the inactive node returns 503).
Tune failover behavior with RS_LOCK_FILE_TTL (how long the standby waits) and RS_LOCK_FILE_TIMEOUT (how long a node waits to acquire the lock).

To avoid split-brain writes, don’t run both nodes in STANDALONE mode against the same dataset.

Read-only replicas for read scaling

Add one or more REPLICA nodes to serve queries from the same dataset. Replicas never write and periodically refresh bucket metadata and indexes from the backend, so newly written data may appear with a small delay.

Route writes to a dedicated ingestion node (or the active node in a hot-standby pair).
Route reads to replicas to scale query workloads horizontally.
Tune staleness with RS_ENGINE_REPLICA_UPDATE_INTERVAL.

For an end-to-end walkthrough (including S3-based standalone, active-passive, and replicas), see the S3 Backend tutorial. For architecture options and operational notes, see the Disaster Recovery guide.

Multi-entry API

The new Multi-entry API makes it possible to work with multiple entries in a single request. In practice, this is most useful for querying : instead of running one query per sensor/stream and merging results on the client, you can request all the entries you need at once and process a single result stream (each returned record includes its entry name).

Here is a Python example using reduct-py to query multiple entries in one call:

import asynciofrom reduct import Clientasync def main() -> None: async with Client("http://localhost:8383", api_token="my-token") as client: bucket = await client.get_bucket("my-bucket") # Query multiple entries in a single request (since ReductStore v1.18). # You can mix exact names and wildcards. entries = ["sensor-*", "camera"] async for record in bucket.query( entries, start="2026-02-05T10:00:00Z", stop="2026-02-05T10:05:00Z", when={"&score": {"$gte": 10}}, ): payload = await record.read_all() print(record.entry, record.timestamp, len(payload))if __name__ == " __main__": asyncio.run(main())

What’s Next

We’re already working on the next improvements to make ReductStore easier to integrate into real-world data pipelines:

Native Zenoh API

Zenoh is becoming a common choice for data exchange in distributed, edge-first systems (robotics, industrial IoT, and telemetry). In upcoming releases, we plan to add a native Zenoh API so ReductStore can join Zenoh networks seamlessly.

This will make it easier to ingest and serve data directly through Zenoh—without custom bridges—so your storage layer fits naturally into existing Zenoh-based deployments.

Entry attachments (metadata)

Today, labels work well for filtering and replication, but many projects also need structured metadata tied to an entry itself: data format, schema version, units, encoding, calibration details, and other context.

We plan to introduce attachments for entries , allowing you to store and retrieve this kind of metadata alongside your data streams, making datasets more self-describing and easier to consume across teams and tools.

I hope you find those new features useful. If you have any questions or feedback, don’t hesitate to use the ReductStore Community forum.

Thanks for using ReductStore!

Comparing Data Management Tools for Robotics

AnthonyCvn — Thu, 04 Dec 2025 09:26:57 +0000

Modern robots collect a lot of data from sensors, cameras, logs, and system outputs. Managing this data well is important for debugging, performance tracking, and training machine learning models.

Over the past few years, we've been building a storage system from scratch. As part of that work, we spoke with many robotics teams across different industries to understand their challenges with data management.

Here's what we heard often:

Only a subset of what robots generate is actually useful
Network connections are not always stable or fast
On-device storage is limited (hard drive swaps is not practical)
Teams rely on manual workflows with scripts and raw files
It's hard to find and extract the right data later
ROS bag files get large quickly and are difficult to manage

In this article, we compare four tools built to handle robotics data: ReductStore, Foxglove, Rerun, and Heex. We look at how they work, what they're good at, and which use cases they support.

If you're working with robots and need to organize, stream, or store data more effectively, this overview should help.

Key Criteria for Comparison

When picking a data tool for robotics, focus on these areas:

Data Types Robotics is a large field with many sensor types. The tool should support the data you work with, such as:
- Telemetry: Lightweight (GPS, IMU, joints), ideal for monitoring.
- Downsampled Data: Lower-rate images or lidar for incident review without high storage cost.
- Full-Resolution: Raw sensor outputs for deep debugging or training. This is storage-intensive but essential for some applications.
Integration The tool should work with what you already use, like ROS, Grafana, MQTT, cloud platforms (S3, Azure, Google Cloud), and your development environment to avoid extra glue code and simplify workflows.
Performance and Scalability Data must move quickly (both locally and to the cloud). Large files or slow queries can block robots or delay analysis.
Ease of Use and APIs A simple UI and solid API support make it easier to automate, scale, and adapt the tool to different use cases.

Tool Overviews

ReductStore

ReductStore is a storage and streaming system designed for robotics data. It works both on the robot and in central storage (on-premise/self-hosted or in the cloud) with the same interface and SDKs (in Python, C++, Go, Javascript/TypeScript or Rust). That means your code stays the same whether you're reading local, remote data (or creating a browser-based dashboard).

To move data to the cloud, ReductStore uses conditional replication. You can define rules to upload only certain records: by label, rules, or event. For example, replicate all incident data, or just 1 out of 10 entries for routine monitoring.

ReductStore handles storage limits on edge devices with FIFO retention. Old data is deleted only when the device is full. Each bucket can have different rules, so you can keep more images and less telemetry, for example.

With an S3 backend, ReductStore batches small records together before uploading. This cuts down the number of requests and lowers cloud storage costs. For observability, you can connect Grafana to ReductStore to create dashboards with system metrics and sensor data. For MCAP files, ReductStore supports shareable query links that open directly in Foxglove v1/v2.

It also lets you filter or merge records server-side. For example, you can pull all temperature readings above a threshold over a time range without downloading full datasets.

Want more technical detail? Check out The Missing Database for Robotics Is Out.

Foxglove and MCAP

Foxglove is a browser-based visualization and observability tool for robotics. It supports ROS 1, ROS 2, and MCAP logs, and handles data types like telemetry, camera feeds, lidar, and depth maps.

It uses MCAP, an open-source log format built for robotics, to store high-resolution data efficiently. You can explore MCAP files interactively in Foxglove Studio or stream them programmatically.

Foxglove provides an agent that detects new MCAP files on the robot and uploads them to the cloud automatically. This requires robots to record short rosbag segments (typically a few minutes each) which are closed and rotated continuously.

It integrates natively with ROS topics, services, and actions, and offers WebSocket and REST APIs. It also connects to major cloud providers like AWS, Azure, and Google Cloud for scalable storage.

The interface is built for time-series and sensor data, with interactive 2D/3D views, plots, and drag-and-drop panels for quick setup and review.

Rerun

Rerun is an open-source visualization solution for time-series and multimodal data. It supports data types like images, point clouds, lidar, depth maps, tensors, and other sensor streams.

Its main strength is combining flexible logging with a fast, built-in 3D viewer designed for robotics and extended reality (XR) applications. For large datasets, Rerun provides a column-oriented API to speed up ingestion and reduce memory usage. It also uses efficient internal structures to minimize allocations and optimize performance on edge devices.

Rerun doesn't offer native ROS integration yet, but it can be used in ROS projects by adding custom logging to nodes.

You can embed Rerun in Jupyter notebooks or web pages, and use loggers for Python, Rust, and C++ to stream data into the viewer.

The UI is built for real-time 3D exploration, with overlays and live tracking that make it easy to inspect different data types in the same visual space.

Heex

Heex is a data capture and review platform for autonomous systems that focuses on collecting only key moments—like errors or specific events instead of logging everything. This reduces bandwidth and storage needs while keeping important context.

Robots using Heex record data continuously in short ROSbag segments. A small agent on the robot watches for triggers and uploads only selected segments to the cloud based on rules.

A core feature is RDA (Resource and Data Automation) for ROS 2, which automates what to record and when. Rules can be changed remotely without restarting the robot.

Data is stored in ROSbag and can be reviewed directly in the Heex dashboard, which includes built-in open-source version of Foxglove. This setup makes it easy to manage data across fleets and locations.

Heex supports both ROS 1 and ROS 2, and integrates with other systems through SDKs, APIs, and a CLI.

The interface includes customizable dashboards to monitor sensor data, errors, and system status. Timelines and streams are easy to navigate for quick analysis.

Comparative Analysis Table

To help visualize the differences between the tools, here is a comparison table summarizing their main characteristics:

Tool	Core Focus	Data Types	Storage Strategy	Visualization	ROS Integration	Unique Features
ReductStore	Time-series storage and streaming for robotics	Telemetry, camera images, lidar, logs	Local + cloud with same API (supports S3, FIFO retention, conditional replication)	Grafana, Foxglove (via MCAP links)	Integrated with ROS via extensions	Filter/merge on server, batch uploads, topic-level control, efficient on edge
Foxglove	Visualization and observability for robotics logs	MCAP logs (telemetry, lidar, camera, depth)	ROSbag short segments, auto-upload with agent	Foxglove Studio (2D/3D, timeline, plots)	Native ROS 1 & 2	Drag-and-drop views, real-time stream inspection, cloud integration
Rerun	Real-time 3D visualization of multimodal time-series data	Images, lidar, point clouds, tensors, metrics	User-defined logging; logs streamed into viewer or embedded in notebooks	Built-in viewer (3D overlays, tracking)	Not native (custom logging)	Column-oriented API, fast ingestion, selective logging, notebook/web integration
Heex	Event-driven data capture for fleets of robots	ROSbag (telemetry, images, lidar, metrics)	Continuous recording, uploads filtered by event-based rules via onboard agent	Built-in Foxglove in dashboard	Native ROS 1 & 2	RDA (automated capture rules), remote config, scalable fleet-wide dashboards

Conclusion

Each tool addresses a different part of the robotics data workflow. ReductStore is ideal for distributed storage across many robots, with selective replication to the cloud and flexible integration with tools like Grafana and Foxglove. Foxglove excels at visualizing MCAP logs and ROS topics. Rerun offers flexible, real-time 3D inspection for custom applications. Heex focuses on capturing just the important moments for efficient fleet analysis.

Choosing the right tool depends on what kind of data you collect, how you process it, and where you need it to go. In many cases, combining tools can give you the best of all worlds.

Thanks for reading. I hope this article helps you decide on the right storage strategy for your vibration data.
If you have questions or comments, feel free to visit the ReductStore Community Forum.

Distributed Storage in Mobile Robotics

AnthonyCvn — Mon, 17 Nov 2025 00:00:00 +0000

Mobile robots produce a lot of data (camera images, IMU readings, logs, etc). Storing this data reliably on each robot and syncing it to the cloud can be hard. ReductStore makes this easier: it's a lightweight, time-series object store built for robotics and industrial IoT. It stores binary blobs (images, logs, CSV sensor data, MCAP, JSON) with timestamps and labels so you can quickly find and query them later.

This introduction guide explains a simple setup where each robot stores data locally and automatically syncs it to a cloud ReductStore instance backed by Amazon S3.

Edge-to-Cloud Architecture

The architecture has three main components:

Each robot runs a small ReductStore server in order to save images and IMU data locally on disk (this let the robot operate offline).
A cloud ReductStore instance runs on a server (e.g., EC2) and uses S3 for long-term storage.
ReductStore replication tasks copies data from robot to cloud based on labels, events, or rules (e.g., 1 record every minute).

Each robot pushes its data to the cloud whenever it is connected to the network. This approach provides the robots with offline capability, allows you to decide which data to replicate, and easily scales to support many robots.

How Replication Works

ReductStore uses an append-only replication model:

The robot stores new data locally.
ReductStore automatically detects new records.
It sends them to the cloud in batches (or streams large files).
If the network disconnects, replication continues when the robot reconnects.

You can replicate:

everything
or only specific sensors
or only records with certain labels
or based on rules (e.g., 1 record every S seconds or every N records)

This can be configured per robot using environment variables (provisioning), with the web console or via the CLI (as shown in this guide).

Cloud ReductStore With S3 Backend

ReductStore supports storing all records directly in S3. It keeps a local cache for fast access and batches many small blobs into larger blocks to save on S3 costs.

By batching data into S3 objects, you can save significantly on storage costs compared to storing many small files individually.

Here is an example docker-compose.yml to run a ReductStore server that uses S3 as the remote backend and provisions buckets for robots:

services:
  reductstore:
    image: reduct/store:latest
    container_name: reductstore
    ports:
      - "8383:8383"
    environment:
      # AWS credentials and S3 bucket configuration
      RS_REMOTE_BACKEND_TYPE: s3
      RS_REMOTE_BUCKET: <YOUR_S3_BUCKET_NAME>
      RS_REMOTE_REGION: <YOUR_S3_REGION>
      RS_REMOTE_ACCESS_KEY: <YOUR_AWS_ACCESS_KEY_ID>
      RS_REMOTE_SECRET_KEY: <YOUR_AWS_SECRET_ACCESS_KEY>
      RS_REMOTE_CACHE_PATH: /data/cache
      # Bucket provisioning
      RS_BUCKET_ROBOT_1_NAME: robot1-data
      RS_BUCKET_ROBOT_2_NAME: robot2-data
      # .. additional buckets as needed
    volumes:
      - ./cache:/data/cache

Run it with:

docker compose up -d

This starts a ReductStore server that writes to S3 automatically. There are many more configuration options available in the configuration documentation.

Setting Up Replication

First spin up a local ReductStore on each robot. Here with Snap:

sudo snap install reductstore

That starts a ReductStore server on port 8383 by default. Then you can use the Reduct CLI to set up replication from the robot to the cloud instance:

# Point the CLI to the robot's local ReductStore
reduct-cli alias add local -L http://localhost:8383 -t "<ROBOT_API_TOKEN>"

# Create a bucket for that robot
reduct-cli bucket create local/robot1-data

# Create a replication task to the cloud
reduct-cli replica create local/robot1-to-cloud \
    robot1-data \
    https://<CLOUD_API_TOKEN>@<cloud-address>/robot1-data

This creates a replication task called robot1-to-cloud that copies all data from the robot's local robot1-data bucket to the cloud instance. You can customize replication further by adding filters or rules. See the replication guide for more details.

Storing Sensor Data

There are many ways to store data. When it comes to high-frequency sensor data like IMU readings, a common approach is to store them in 1-second files. Images can be stored as binary blobs (e.g., JPEG or PNG files). Here is an example of storing IMU data as CSV files and images as binary blobs using the Python SDK (this stores 10,000 samples and one camera image for a given timestamp as an example):

import asyncio
import time
import random
from reduct import Client


async def main():
    async with Client("http://localhost:8383", api_token="<ROBOT_API_TOKEN>") as client:
        bucket = await client.get_bucket("robot1-data")

        # Current timestamp to index the data by time in ReductStore
        timestamp = int(time.time() * 1_000_000)  # microseconds

        # Generate 10'000 IMU samples
        rows = []
        for i in range(10_000):
            rows.append(
                {
                    "ts": timestamp + i * 100,  # microseconds
                    "linear_acceleration_x": round(random.uniform(-2, 2), 3),
                    "linear_acceleration_y": round(random.uniform(-2, 2), 3),
                    "linear_acceleration_z": round(random.uniform(8.0, 10.0), 3),
                }
            )

        # Convert to CSV (store 1 seconds of data per file)
        csv = (
            "ts,linear_acceleration_x,linear_acceleration_y,linear_acceleration_z\n"
            + "\n".join(
                f"{r['ts']},{r['linear_acceleration_x']},"
                f"{r['linear_acceleration_y']},{r['linear_acceleration_z']}"
                for r in rows
            )
        )

        # Write the IMU batch
        await bucket.write(
            entry_name="imu_logs",
            data=csv.encode(),
            timestamp=timestamp,
            labels={"sensor": "imu", "rows": "1000"},
            content_type="text/csv",  # MIME type
        )

        # Write one camera image
        with open("camera_image.png", "rb") as img:
            await bucket.write(
                entry_name="images",
                data=img.read(),
                timestamp=timestamp,
                labels={"sensor": "camera"},
                content_type="image/png",
            )


asyncio.run(main())

If you are considering storing all IMU data as individual records in a time series database (TSDB) like Timescale or InfluxDB, keep in mind that high-frequency sensors (e.g., 1000 Hz) can lead to performance and cost issues. Batching samples into files (e.g., one second of data per CSV file) is a more efficient storage and querying method.

Querying Sensor Data Using ReductSelect

If your IMU data is stored as CSV, the ReductSelect extension lets you:

extract only certain columns
filter rows based on conditions

Example: filter CSV rows where acc_x > 10:

{
    "#ext": {
        "select": {
            "csv": {"has_headers": True},
            "columns": [
                {"name": "ts", "as_label": "ts_ns"},
                {"name": "linear_acceleration_x", "as_label": "acc_x"},
                {"name": "linear_acceleration_y"},
                {"name": "linear_acceleration_z"},
            ],
        },
        "when": {"@acc_x": {"$gt": 1.9}},
    }
}

Python example:

import asyncio
from reduct import Client

when = { ... } # the JSON condition from above

async def main():
    async with Client("https://<cloud-address>", api_token="<TOKEN>") as client:
        bucket = await client.get_bucket("robot1-data")

        async for rec in bucket.query("imu_logs", when=when):
            data = await rec.read_all()
            print(data.decode())

asyncio.run(main())

This returns only the rows where linear_acceleration_x > 1.9, along with the timestamp.

Why This Setup Works Well for Robotics

There are several advantages to using a specialized storage solution like ReductStore for mobile robotics:

Robots can store data locally and operate offline without network connectivity.
Automatic replication when connected to avoid manual uploads and simplify data management.
Selective replication lets you control what data is sent to the cloud (i.e. decide on your reduction strategy) to save bandwidth and storage.
Labels and timestamps make it easy to organize and query sensor data later.
Store files of any type (images, CSV, logs, MCAP) in a single system without needing separate storage solutions for each data type.

Next Steps

ReductStore also integrates into robotics observability stacks such as the Canonical Observability Stack (COS) for robotics. You can visualize sensor data, logs, and metrics in Grafana dashboards alongside your other robot telemetry. More details in our blog post The Missing Database for Robotics Is Out.

I hope you found this article helpful! If you have any questions or feedback, don't hesitate to reach out on our ReductStore Community forum.

The Missing Database for Robotics Is Out

AnthonyCvn — Wed, 22 Oct 2025 00:00:00 +0000

Robotics teams today wrestle with data that grows faster than their infrastructure. Every robot generates streams of images, sensor readings, logs, and events in different formats. These data piles are fragmented, expensive to move, and slow to analyze. Teams often rely on generic cloud tools that are not built for robotics. They charge way too much per gigabyte (when it should cost little per terabyte), hide the raw data behind proprietary APIs, and make it hard for robots (and developers) to access or use their own data.

ReductStore introduces a new category: a database purpose built for robotics data pipelines. It is open, efficient, and developer friendly. It lets teams store, query, and manage any time series of unstructured data directly from robots to the cloud.

What Makes It a New Category

ReductStore treats robotics with the respect it deserves. It captures everything in its raw form and stores it with a time index and labels for flexible querying and management. It ingests and streams any type of data (images, sensor frames, logs, MCAP files, CSVs, JSON, etc) without forcing developers to convert or reformat it.

It works on robots and in the cloud using the same interface and SDKs (Python, C++, Rust, Javascript, Go). This means developers can build data pipelines that run the same way on robots or in the cloud without needing to change code or learn new tools.

Developers can run ReductStore on an edge device for local data capture and replicate to a cloud instance (with S3 backend) for cloud analytics or archiving.

It is the first and only database designed specifically for unstructured, time series robotics data.

Data Handling and Querying

Developers can work directly with data using simple queries and SDKs. The focus is speed and flexibility.

1. MCAP topic filtering

You can filter topics directly from multiple MCAP files stored in ReductStore without needing to download and reprocess everything locally.

import json
import pandas as pd
from reduct import Client

# Extract only the IMU topic from MCAP files
ext = {
    "ros": {"extract": {"topic": "/imu/data"}},
}

async with Client("https://test.reduct.store") as client:
    bucket = await client.get_bucket("my-robotics-data")
    parts = []
    async for rec in bucket.query("mcap-entry", ext=ext):
        blob = await rec.read_all()
        data = json.loads(blob.decode("utf-8"))
        rows = [
            {
                "ts": data["header"]["stamp"]["sec"] * 1_000_000_000
                + data["header"]["stamp"]["nanosec"],
                "linear_acceleration_x": data["linear_acceleration"]["x"],
                "linear_acceleration_y": data["linear_acceleration"]["y"],
                "linear_acceleration_z": data["linear_acceleration"]["z"],
            }
        ]

This allows you to extract only the relevant topics from multiple bags. In this example, we extract only the IMU topic as a stream of JSON records, which would look like this:

ts	linear_acceleration_x	linear_acceleration_y	linear_acceleration_z
1633024800000	0.1	0.3	-9.8
1633024801000	0.0	0.1	-9.7
...	...	...	...

2. CSV/JSON field extraction and filtering

You can extract specific JSON fields or CSV columns when querying data. This lets you select only the information you need, for example, filtering and visualizing certain fields from streams of JSON or CSV sensor readings.

import io
import pandas as pd
from reduct import Client

# Select specific CSV columns and filter rows
ext = {
    "select": {
        "csv": {"has_headers": True},
        # Use "json": {}, for JSON data
        "columns": [
            {"name": "ts"},
            {"name": "linear_acceleration_x", "as_label": "acc_x"},
            {"name": "linear_acceleration_y"},
            {"name": "linear_acceleration_z"},
        ],
    },
    "when": {"$gt": [{"$abs": ["@acc_x"]}, 10]},
}

async with Client("https://test.reduct.store") as client:
    bucket = await client.get_bucket("my-robotics-data")

    # Loop over filtered CSV entries
    async for rec in bucket.query("csv_sensor_readings", ext=ext):
        blob = await rec.read_all()
        csv_data = pd.read_csv(io.BytesIO(blob))

The tabular result will only include the selected columns and rows that match the filter abs(linear_acceleration_x) > 10:

ts	linear_acceleration_x	linear_acceleration_y	linear_acceleration_z
1633024800000	12.5	0.3	-9.8
1633024801000	-15.2	0.1	-9.7
...	...	...	...

3. Query any type of data

ReductStore automatically batches small records and streams large ones for efficient storage and access. You can query any type of data, from lightweight telemetry to high-resolution images or point clouds, efficiently.

import io
from PIL import Image
from reduct import Client

# Every 5 seconds, limit to 5 records
when = {"$each_t": "5s", "$limit": 5}

async with Client("https://test.reduct.store") as client:
    bucket = await client.get_bucket("my-robotics-data")
    async for rec in bucket.query("camera_frames", when=when):
        blob = await rec.read_all()
        img = Image.open(io.BytesIO(blob))

The example above retrieves camera frames at 5-second intervals. You can then process or visualize these images as needed.

4. Browse petabytes of data

ReductStore is designed to handle massive volumes of data. Its indexing and storage architecture allows you to efficiently browse data at scale without downloading everything locally.

For example, you can quickly navigate records and preview your data directly in the ReductStore web console, even when working with petabytes of robotics data.

info

You can build custom applications on top of ReductStore using its SDKs for Python, C++, Rust, Javascript, and Go. This makes it easy to build data pipelines, dashboards that works in the browser, or integrate with existing tools.

Cloud Integration and Cost Savings

ReductStore connects robots and the cloud in a simple and flexible way. It works with S3-compatible storage and includes a robust replication system to transfer data from robots to the cloud (even when the network is unstable or intermittent), making it perfect for field robots that often go offline.

Replication tasks can be configured to replicate only specific data based on labels or any criteria (for example, only replicate data when the confidence score is below a threshold, or replicate everything from a 10-minute window around a specific event ).

In the cloud, by batching multiple records into single data blocks, ReductStore minimizes both the number of blobs and the number of API calls to S3. This design reduces storage and retrieval costs by leveraging S3's pricing model.

This approach can deliver major savings when working with large volumes of robotics data.

Observability Stack Integration

ReductStore works with the tools robotics engineers already trust.

Foxglove Studio

Foxglove is an amazing tool for visualizing robotics data and debugging robots for the MCAP format.

To share data from ReductStore to Foxglove, you can use the ReductStore web console (or the SDKs) to generate a query link that Foxglove can open directly.

You can then paste the query link into Foxglove Studio to visualize the data.

Grafana

Grafana is a popular open-source tool for creating dashboards and visualising time-series data. You can connect Grafana to ReductStore using the ReductStore data source plugin, which allows you to query and visualise data stored in ReductStore.

You can query data using labels, for example, localization coordinates, object detected, confidence score, etc:

Or you can query based on content, such as JSON files with sensor readings or other structured data:

Canonical Observability Stack (COS)

Canonical's COS (Canonical Observability Stack) for robotics is an end to end observability framework built on open source tools such as Prometheus, Loki, Grafana, and Foxglove.

The missing piece in this stack has always been a purpose built system for storing and managing robotics data efficiently from robot to cloud.

ReductStore closes that gap. It provides a data storage and streaming solution optimized for both edge and cloud environments, along with an agent that captures data directly from ROS and streams it into the observability pipeline.

Closing Thoughts

Robotics teams no longer need to choose between control and convenience. ReductStore gives full ownership of data from robot to cloud. It removes vendor lock, cuts cost, and keeps everything observable and connected. It is the new foundation for robotics data infrastructure (the missing database for robotics).

If you are interested to compare ReductStore with other databases (like MongoDB or InfluxDB), you can read our white paper that goes deeper into the architecture and design choices.

I hope you found this article helpful! If you have any questions or feedback, don't hesitate to reach out on our ReductStore Community forum.

ReductStore v1.17.0 Released with Query Links and S3 Storage Backend Support

Alexey Timin — Tue, 21 Oct 2025 00:00:00 +0000

We are pleased to announce the release of the latest minor version of ReductStore, 1.17.0. ReductStore is a high-performance storage and streaming solution designed for storing and managing large volumes of historical data.

To download the latest released version, please visit our Download Page.

What's new in 1.17.0?

This release includes several new features and enhancements. First, there are query links for simplified data access. Second, there is support for S3-compatible storage backends.

These new features enhance the usability and flexibility of ReductStore for various use cases in the cloud and on-premises environments and make it easier to share and access data stored in the database.

🔗 Query Links for Data Access

ReductStore now supports query links, enabling users to generate temporary, public URLs for specific data records — without requiring authentication. This makes it easier to share datasets with external collaborators , embed links into dashboards, or integrate with third-party systems that need read-only access to specific data.

You can create query links directly from the Web Console (or any SDKs):

Open the Data Browser page and select a record you want to share.
Click the “Share record” icon in the action panel.
Configure an expiration time to automatically revoke access after a defined period.

Once generated, anyone with the link can access the selected record via a simple HTTP(S) request — no access token required. The link only has access to the specific query for which it was created, along with the creator's permissions. This provides a secure and convenient way to expose selected data for collaboration and analysis.

☁️ S3-Compatible Storage Backend

ReductStore now supports S3-compatible storage backends , allowing you to use object storage instead of a local file system for your underlying data. This update brings greater flexibility and scalability for managing large datasets in the cloud.

Previously, ReductStore supported only local disk storage, and users had to mount S3 buckets as local disks via FUSE drivers. With this release, ReductStore can now natively integrate with S3-compatible backends — no additional software or mounting is required.

This feature is designed with performance and cost optimization in mind. ReductStore uses a local disk cache layer to speed up read and write operations, while batching multiple records into a single data block to reduce storage and retrieval costs. This approach works especially well with cost-efficient AWS S3 storage classes such as S3 Standard-IA or S3 Glacier.

To run ReductStore with an S3-compatible backend, use the following environment variables:

docker run -p 8383:8383 \
 -e RS_REMOTE_BACKEND_TYPE=s3 \
 -e RS_REMOTE_BUCKET=<YOUR_S3_BUCKET_NAME> \
 -e RS_REMOTE_REGION=<YOUR_S3_REGION> \
 -e RS_REMOTE_ACCESS_KEY=<YOUR_S3_ACCESS_KEY_ID> \
 -e RS_REMOTE_SECRET_KEY=<YOUR_S3_SECRET_ACCESS_KEY> \ 
 -e RS_REMOTE_CACHE_PATH=/data/cache \
 -e RS_LICENSE_PATH=<PATH_TO_YOUR_LICENSE_FILE> \ 
 -v ${PWD}/data:/data/cache \
 reduct/store:latest

Read more about configuring S3-compatible storage backend in the documentation

info

This feature requires a commercial license. Please see the Pricing page for more details.

What’s Next

We’re continuing to develop new features to make ReductStore even more powerful and user-friendly. Here’s a preview of what’s coming in the next releases:

📦 Multiple Entries in a Single Request

Currently, each write or query request must target a single entry. This can be limiting when dealing with multiple entries or dynamic lists of entries in your applications.

In upcoming versions, ReductStore will support batch operations across multiple entries within a single API request. This improvement will simplify integrations and reduce overhead for large-scale data ingestion and querying workflows.

🔒 Read-Only Mode for ReductStore

Like most databases, ReductStore currently requires exclusive access to its data directory while running. As a result, running multiple instances on the same dataset—for load balancing or high availability—is not yet possible.

To address this, we’re introducing a read-only mode that will allow one writer instance* and multiple reader instances to access the same dataset concurrently. This approach will enable scalable read operations and improved availability without adding the complexity of clustering or replication mechanisms.

I hope you find those new features useful. If you have any questions or feedback, don’t hesitate to use the ReductStore Community forum.

Thanks for using ReductStore!

Building a Resilient ReductStore Deployment with NGINX

Alexey Timin — Sat, 13 Sep 2025 00:00:00 +0000

If you’re collecting high-rate sensor or video data at the edge and need zero-downtime ingestion and fault-tolerant querying, an active–active ReductStore setup fronted by NGINX is a clean, practical pattern.

This tutorial walks you through the reference implementation, explains the architecture, and shows production-grade NGINX snippets you can adapt.

What We’ll Build

We’ll set up a ReductStore cluster with NGINX as a reverse proxy, separating the ingress and egress layers. This architecture allows for independent scaling of write and read workloads, ensuring high availability and performance.

Ingress layer

The ingress layer handles all writes and replicates data to the egress layer. Its nodes may have limited storage capacity, while they need only to handle writes and replicate data to the egress nodes. It can use high-rate storage like NVMe SSDs or even RAM disks, depending on your data volume.

Egress layer

The egress layer handles all reads and serves data to clients. Its nodes are optimized for read performance and can use larger, slower storage like HDDs or cloud object storage. Each egress node holds a complete copy of the dataset, allowing for high availability and load balancing.

NGINX Load Balancer

The NGINX load balancer sits in front of both layers, exposing two stable endpoints:

http://<host>/ingress → load balances writes across ingress nodes
http://<host>/egress → load balances reads across egress nodes

This separation allows you to scale each layer independently and ensures that writes and reads are handled optimally.

It is also important to note that NGINX must maintain session affinity (stickiness) for both ingress and egress requests to ensure that queries remain consistent and throughput is maximized.

Quick Start

Clone the example and bring it up:

git clone https://github.com/reductstore/nginx-resilient-setup
cd nginx-resilient-setupdocker 
compose up -d

This will start two ingress nodes and two egress nodes with NGINX in front, all configured to replicate data between them. Check the docker compose file for details on how the nodes are set up.

Now we need to write some data and verify that we can read it back.Install the reduct-cli tool if you haven't already, then run the following commands to set up aliases for the ingress and egress endpoints:

reduct-cli alias add ingress -L http://localhost:80/ingress --token secret
reduct-cli alias add egress -L http://localhost:80/egress --token secret

Then copy some data from our Demo Server to the ingress layer and read it back from the egress layer:

# Add demo server alias to the CLI
reduct-cli alias add play -L https://play.reduct.store --token reductstore
# Copy data from the demo server to ingress
reduct-cli cp play/datasets ingress/bucket-1 --limit 1000
# Read/export via egress
reduct-cli cp egress/bucket-1 ./export_folder --limit 1000

NGINX Configuration

Below is a distilled config you can adapt for open-source NGINX:

# Upstreams
# Separate pools for ingress (writes) and egress (reads)
upstream reduct_ingress {
    ip_hash;   # stickiness for writes
    server ingress-1:8383 max_fails=3 fail_timeout=10s;
    server ingress-2:8383 max_fails=3 fail_timeout=10s;
    keepalive 64;
}

upstream reduct_egress {
    ip_hash;   # stickiness for queries
    server egress-1:8383 max_fails=3 fail_timeout=10s;
    server egress-2:8383 max_fails=3 fail_timeout=10s;
    keepalive 64;
}

server {
    listen 80;
    server_name _;

    client_max_body_size 512m;
    proxy_read_timeout 600s;
    proxy_send_timeout 600s;

    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $remote_addr;

    location /ingress/ {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_pass http://reduct_ingress/;
    }

    location /egress/ {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_pass http://reduct_egress/;
    }
}

In the config above, we define two upstream blocks: reduct_ingress for handling write requests and reduct_egress for handling read requests. Each block uses ip_hash to ensure session affinity, which is crucial for maintaining consistent writes and reads.

ReductStore Configuration Notes

The configuration between nodes of each layer is identical. To reach the desired architecture, you need to provision buckets and replication tasks for ingress nodes and buckets only for egress nodes. See the configuration files in the example repo for details.

Failure Drills

When the setup is running, you can simulate failures to see how it behaves:

Kill an ingress node → writes continue via other ingress nodes.
Kill an egress node → reads continue via other egress nodes; replication resyncs when it’s back.
Simulate total ingress outage → analysis continues on egress; for true ingestion continuity, pair with a pilot-light instance in another location.

Runbook

Here’s a high-level runbook for deploying this architecture in production:

Provision ingress + egress ReductStore nodes
Create buckets and replication tasks
Expose /ingress and /egress via NGINX with ip_hash
Test with demo dataset
Validate reads from egress
Run failure drills

References

I hope you find this article interesting and useful. If you have any questions or feedback, don’t hesitate to use the ReductStore Community forum.

ReductStore v1.16.0 Released With New Extensions and Context Replication

Alexey Timin — Sat, 30 Aug 2025 00:00:00 +0000

We are pleased to announce the release of the latest minor version of ReductStore, 1.16.0. ReductStore is a high-performance storage and streaming solution designed for storing and managing large volumes of historical data.

To download the latest released version, please visit our Download Page.

What's new in 1.16.0?

The v1.16.0 release introduces two new extensions designed to enhance data workflows for robotics and columnar data, along with support for replicating context records during queries.

Querying and Replicating Data with Context

We’ve extended the conditional query syntax with directives that allow users to modify global query behavior. The first directives introduced are #ctx_before and #ctx_after, which enable the inclusion of context records that occur before or after each matching record in a query.

This feature is particularly useful when analyzing specific events or conditions in your data, as it helps provide a clearer picture of the surrounding context. For instance, you can use these directives to include records from a few seconds before or after an anomaly or incident, aiding in root cause analysis or pattern recognition.

Here’s an example of how to use the #ctx_before directive in a query:

{ "#ctx_before": "5s", "&anomaly_score": { "$gt": 0.8 }}

This query returns all records with an anomaly score greater than 0.8, along with the context records that occurred within 5 seconds before each matching entry.

New ReductSelect Extension

ReductStore is fundamentally a blob storage system and does not allow direct manipulation of stored data. However, with its extension mechanism, we can introduce new capabilities while keeping the core system simple.

The new ReductSelect extension enables users to query and transform data stored in CSV or JSON formats, making it easier to build flexible and efficient data processing workflows.

For example, the following query uses ReductSelect to extract specific columns from CSV data and filter rows using the same conditional syntax available in ReductStore's native query language:

{
  "ext": {
    "select": {
      "csv": {
        "has_headers": true
      },
      "columns": [
        { "name": "temperature", "as_labels": "temp" },
        { "name": "humidity" }
      ]
    }
  },
  "when": {
    "&temperature": { "$gt": 30 }
  }
}

This query selects the temperature and humidity columns from a CSV file, renames temperature to temp, and filters rows where the temperature is greater than 30°C.

These simple transformations enable you to ingest structured data very quickly and retrieve only subsets of it for further processing and analysis.

New ReductROS Extension

Another exciting addition is the ReductROS extension, which provides tools for extracting and transforming data stored in ReductStore into formats compatible with the Robot Operating System (ROS).

With this extension, you can extract data from MCAP files containing ROS 2 messages and convert it into JSON format, making it easier to analyze and visualize. It also supports transforming raw binary data—such as images—into more accessible formats like JPEG or base64 strings.

For example, the following query extracts data from a ROS 2 topic and encodes the image payload as a JPEG:

{
  "ext": {
    "ros": {
      "extract": {
        "topic": "/camera/image",
        "encode": { "data": "jpeg" }
      }
    }
  }
}

ReductROS is still in active development, and we plan to expand its capabilities with support for additional ROS message types and more flexible extraction options in future releases. Stay tuned for updates!

What next?

We are constantly working on improving ReductStore and adding new features to provide the best experience for our users. In the next release we plan to add new features and improvements, including:

Shareable Query Links

We are developing a feature that allows users to generate and share links to specific queries in ReductStore.

This will simplify collaboration by enabling team members to access query results without needing direct access to the ReductStore instance. It will also allow users to download results directly via a link and support integration with external tools and platforms such as Foxglove.

Integration with Grafana

We are also working on a Grafana plugin that enables users to visualize and analyze data stored in ReductStore directly within Grafana dashboards.

This integration will provide a seamless experience with Grafana’s powerful visualization tools, allowing you to:

Build custom dashboards using data from ReductStore.
Monitor your data streams and historical records in real time.
Visualize labels and data output in JSON or CSV formats.

Stay tuned for the first release—coming soon!

I hope you find those new features useful. If you have any questions or feedback, don’t hesitate to use the ReductStore Community forum.

Thanks for using ReductStore!

DEV Community: ReductStore

Persistent Edge Storage for a Legacy IIoT System

No persistent storage, no access to raw data

Reliable delivery to the cloud

Continuous Queries

Replication to a fake target

Building the HTTP-to-MQTT bridge

What we achieved

How to Store and Manage Robotics Data

Challenges in Robotic Data Management

General Strategies for Managing Robotic Data

ReductStore: Built for Robotic Data

ROS Integration with reduct-bridge

Cloud Storage: Native S3 and Azure

Visualize ROS Data in Grafana

Export to MCAP for Foxglove

Zenoh Native API

Query Language and Batching

Replication and Edge to Cloud Replication

Retention Strategies

Example Applications

Comparing ReductStore with Rosbag and MongoDB

Hands-On Example: Storing Robotic Data in ReductStore

Setting Up ReductStore

Store and Query Data

Create a Bucket

Generate Trajectory Data

Calculate Metrics

Write to ReductStore

Query by Label

Main Function

Conclusion

ReductStore v1.19: Open Data Backbone for Robotics and ROS

What's new in 1.19.0?​

Nested Data Model with Attachments​

Native Zenoh API​

ReductBridge for ROS Integration​

What’s Next​

Compatibility and Migration​

ReductStore Core Adopts Apache 2.0 License

What's changing​

Why we're doing this​

What this means for you​

Air-Gapped Drone Data Operations with Delayed Sync and Auditability

Drone-to-Ground Architecture​

Setting Up the Drone Node​

Storing Drone Data with Labels​

Setting Up Selective Replication​

Replicating with context (before and after)​

Querying for Audit Reports​

Why This Setup Works Well for Drones​

Next Steps​

ReductStore v1.18.0 Released with Resilient Deployments and the Multi-entry API

What's new in 1.18.0?​

Resilient Deployments​

Hot standby (active-passive) for write availability​

Read-only replicas for read scaling​

Multi-entry API​

What’s Next​

Native Zenoh API​

Entry attachments (metadata)​

Comparing Data Management Tools for Robotics

Key Criteria for Comparison

Tool Overviews

ReductStore

Foxglove and MCAP

Rerun

Heex

Comparative Analysis Table

Conclusion

Distributed Storage in Mobile Robotics

Edge-to-Cloud Architecture​

How Replication Works​

Cloud ReductStore With S3 Backend​

Setting Up Replication​

Storing Sensor Data​

Querying Sensor Data Using ReductSelect​

Why This Setup Works Well for Robotics​

Next Steps​

The Missing Database for Robotics Is Out

What's new in 1.19.0?

Nested Data Model with Attachments

Native Zenoh API

ReductBridge for ROS Integration

What’s Next

Compatibility and Migration

What's changing

Why we're doing this

What this means for you

Drone-to-Ground Architecture

Setting Up the Drone Node

Storing Drone Data with Labels

Setting Up Selective Replication

Replicating with context (before and after)

Querying for Audit Reports

Why This Setup Works Well for Drones

Next Steps

What's new in 1.18.0?

Resilient Deployments

Hot standby (active-passive) for write availability

Read-only replicas for read scaling

Multi-entry API

What’s Next

Native Zenoh API

Entry attachments (metadata)

Edge-to-Cloud Architecture

How Replication Works

Cloud ReductStore With S3 Backend

Setting Up Replication

Storing Sensor Data

Querying Sensor Data Using ReductSelect

Why This Setup Works Well for Robotics

Next Steps

What Makes It a New Category

Data Handling and Querying

1. MCAP topic filtering

2. CSV/JSON field extraction and filtering

3. Query any type of data

4. Browse petabytes of data

Cloud Integration and Cost Savings

Observability Stack Integration

Foxglove Studio

Grafana

Canonical Observability Stack (COS)

Closing Thoughts

What's new in 1.17.0?

🔗 Query Links for Data Access

☁️ S3-Compatible Storage Backend

What’s Next

📦 Multiple Entries in a Single Request

🔒 Read-Only Mode for ReductStore

What We’ll Build

Ingress layer

Egress layer

NGINX Load Balancer

Quick Start

NGINX Configuration

ReductStore Configuration Notes

Failure Drills

Runbook

References

What's new in 1.16.0?

Querying and Replicating Data with Context

New ReductSelect Extension

New ReductROS Extension

What next?

Shareable Query Links

Integration with Grafana