DEV Community

周明
周明

Posted on

Recovering Corrupted ROS 2 Bags Without a ROS 2 Runtime

#opensource #cpp #robotics #ros2

Robots do not always fail gracefully.

A delivery robot can hit an obstacle. A lawn-mowing robot can lose power. An inspection robot can emergency-stop in the middle of a mission. In many of these cases, the most valuable data is the last few seconds before the incident: camera frames, IMU samples, odometry, localization, control commands, and diagnostics.

For ROS 2 systems, that data is often stored in rosbag2.

But there is a painful failure mode: if the process exits abnormally or the device loses power, the underlying MCAP or SQLite3 file may not be closed cleanly. The data file can be missing a footer, summary, index, WAL checkpoint, or part of the final page. Official tools may refuse to open it, even though many complete messages are still inside.

That is the problem I started working on with LibRobotBagFix.

GitHub: https://github.com/Adiao1973/LibRobotBagFix

What is LibRobotBagFix?

LibRobotBagFix is an open-source C++ SDK and command-line tool for inspecting, repairing, and reading ROS 2 rosbag2 black-box data without requiring a ROS 2 runtime.

The project focuses on commercial robot field diagnostics:

  • autonomous delivery vehicles
  • lawn-mowing robots
  • inspection robots
  • mobile robot fleets
  • robotics support tools
  • accident analysis workflows

The goal is not to replace ROS 2 or implement a full rosbag player. The goal is more focused:

Recover as many complete messages as possible from damaged MCAP or SQLite3 rosbag2 files, then make the result readable again.

Why this matters

In real robot incidents, the most important data is usually near the end of the recording.

Unfortunately, the end of the file is also the part most likely to be damaged during power loss or a crash.

Common symptoms include:

  • ros2 bag info cannot open the bag
  • an MCAP file is missing its summary or footer
  • a SQLite3 .db3 file has uncheckpointed WAL data
  • the last SQLite page is partially written
  • the file contains complete earlier messages, but normal tools cannot reach them

For field teams, there is another practical issue: they may not have a full ROS 2 environment available. They may be using macOS, Windows, a tablet, or a lightweight diagnostic tool.

That is why LibRobotBagFix is designed as a small embeddable SDK with a CLI, not as a ROS 2 node.

Current features

The CLI currently provides two main commands.

Inspect a bag directory or single data file:

./build/robotbagfix inspect path/to/bag_or_file --json
Enter fullscreen mode Exit fullscreen mode

Repair a damaged MCAP file:

./build/robotbagfix repair damaged.mcap -o repaired.mcap --json
Enter fullscreen mode Exit fullscreen mode

Repair a damaged SQLite3 rosbag2 database:

./build/robotbagfix repair damaged.db3 -o repaired.db3 --json
Enter fullscreen mode Exit fullscreen mode

The project currently supports:

  • rosbag2 directory detection
  • MCAP detection and record scanning
  • SQLite3 .db3 detection
  • SQLite WAL sidecar discovery
  • MCAP tail repair
  • SQLite fast-path recovery
  • SQLite page-level salvage fallback
  • C++ SDK message iteration
  • stable C ABI for FFI
  • lightweight CDR helpers
  • optional Qt desktop demo

How MCAP repair works

MCAP has a structured binary container format. A valid file includes magic bytes, a header, a data section, optional summary sections, a footer, and trailing magic bytes.

A common corruption pattern is tail truncation. For example, the file may contain many complete records, but the final summary or footer was never written.

LibRobotBagFix scans the MCAP file sequentially, keeps complete records, discards the partial tail record if one exists, and rebuilds the required trailing structure.

The simplified idea looks like this:

damaged.mcap
  -> scan complete records
  -> stop at the first incomplete record
  -> discard damaged tail bytes
  -> rebuild Data End / Summary / Footer
  -> write repaired.mcap
Enter fullscreen mode Exit fullscreen mode

The source file is never modified in place.

How SQLite3 repair works

The SQLite3 backend is more complex because rosbag2 data is stored inside database tables.

The first strategy is to let SQLite do what SQLite is good at:

  • open the database safely
  • apply WAL recovery when possible
  • run integrity checks
  • export complete rows
  • rebuild a clean .db3

If that fails, LibRobotBagFix falls back to page-level salvage. It scans SQLite pages, tries to recover sqlite_schema, locates rosbag2 tables such as topics and messages, and writes only rows that can be proven complete.

Uncertain data is reported, not silently written into the output database.

Design principles

A few principles guide the project:

No ROS 2 runtime dependency

The SDK should be usable in diagnostic tools, desktop applications, mobile apps, and support workflows without installing a full ROS 2 environment.

Repair by container, not by sensor

At the repair layer, an image, IMU sample, odometry message, and point cloud are all payload bytes inside MCAP or SQLite3.

The repair engine focuses on the container format. Sensor-specific interpretation belongs in a higher-level parser.

Do not keep half-written messages

If a message payload itself is incomplete, LibRobotBagFix does not pretend it is valid. It keeps complete messages and reports discarded bytes, records, pages, or uncertain rows.

Keep the SDK embeddable

The project provides a C++ API and a stable C ABI. That makes it easier to integrate with tools written in Swift, Kotlin/JNI, C#, Python FFI, Qt, or other application layers.

Quick start

Build the default SDK and CLI:

git clone https://github.com/Adiao1973/LibRobotBagFix.git
cd LibRobotBagFix
scripts/build.sh
Enter fullscreen mode Exit fullscreen mode

Check the version:

./build/robotbagfix --version
Enter fullscreen mode Exit fullscreen mode

Run tests:

ctest --test-dir build --output-on-failure
Enter fullscreen mode Exit fullscreen mode

Inspect a file:

./build/robotbagfix inspect path/to/bag_or_file --json
Enter fullscreen mode Exit fullscreen mode

Repair a file:

./build/robotbagfix repair damaged.mcap -o repaired.mcap --json
Enter fullscreen mode Exit fullscreen mode

Read repaired output through the SDK examples:

./build/rbf_read_bag_cpp repaired.db3
./build/rbf_read_bag_c repaired.db3
Enter fullscreen mode Exit fullscreen mode

Who this is for

This project may be useful if you work on:

  • robotics incident analysis
  • autonomous robot field support
  • ROS 2 data tooling
  • fleet diagnostics
  • lightweight robot data viewers
  • C++ SDKs for robotics products
  • mobile or desktop tools that need to read robot black-box data

It is especially relevant when you need to inspect or recover robot data outside a full ROS 2 workstation.

Current boundaries

LibRobotBagFix is still focused. It does not try to do everything.

Current non-goals include:

  • modifying damaged bags in place
  • recovering half-written image or point-cloud payloads
  • implementing a full rosbag player
  • publishing or subscribing to DDS topics
  • making ROS 2 a runtime dependency
  • making Qt or mobile UI frameworks part of the core SDK

What I am looking for

This project is open source, and I would appreciate feedback from people who work with ROS 2 bags in real systems.

Useful feedback includes:

  • damaged bag patterns you have seen in production
  • MCAP or SQLite3 edge cases
  • cross-platform build feedback
  • SDK API suggestions
  • ideas for minimal fixtures and regression tests
  • field diagnostic workflows where this could help

GitHub repo:

https://github.com/Adiao1973/LibRobotBagFix

If you work with ROS 2 robot logs and have ever lost the last seconds of data after a crash or power failure, I would be interested to hear what failure modes you have seen.

Top comments (0)