RoboDoctor: A Local AI Debugging Companion for Robotics

Shreyansh Chauhan — Sun, 04 Oct 2026 13:27:57 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

Debugging robotics software has a particular kind of pain.
You don't just have one terminal.
You might have a ROS2 launch running in one terminal, a controller in another, a sensor node somewhere else, RViz open on the side, and a Python script running in another window.
Then something breaks.
You get a wall of logs, a stack trace, or a ROS2 error, and the real question becomes:
"Okay... but why did it actually break?"
I built RoboDoctor for a friend who works with robotics and spends far too much time chasing these errors across terminals.
RoboDoctor is a local AI debugging companion that sits quietly on the desktop until something goes wrong.
It can monitor multiple RoboDoctor-managed terminal sessions, detect failed commands, associate an error with the correct terminal/session, and surface the problem through a small desktop robot companion.
When I double-click the robot, it takes me directly to the problematic session and starts an investigation.
Instead of simply sending the error message to an LLM, RoboDoctor can gather additional diagnostic evidence through its own restricted diagnostic terminal and give that context to Gemma 3 4B running locally through Ollama.
The goal is simple:
Don't make the developer hunt for the error. Bring the solution to the developer.

Demo

Here you can watch how the whole code works!

Code

The complete project is open source:
GitHub: https://github.com/Shreyansh08-bit/RoboDoctor.git
The repository contains the backend, frontend, desktop companion, diagnostic logic, examples, tests, and startup scripts.

## How I Built It

RoboDoctor is built around a simple idea:

the AI shouldn't be the entire application.

Gemma is the reasoning layer inside a larger debugging system.

The architecture looks roughly like this:

Developer Terminals
↓
Terminal Session Manager
↓
Execution + Error Detection
↓
Diagnostic Blackboard
↓
Deterministic Diagnostics
↓
Gemma 3 4B
↓
Diagnostic Investigation
↓
Diagnosis
↓
RoboDoctor Companion

Multi-terminal debugging

Robotics development rarely happens in one terminal.

RoboDoctor therefore treats every managed terminal as an independent session.

Each session tracks information such as:

command
working directory
stdout/stderr
exit code
execution sequence
ROS2 environment
current issue state

This means an error in Terminal 3 does not accidentally become the diagnosis for Terminal 1.

Automatic error detection

The developer does not have to copy an error and paste it into a chatbot.

When a command fails inside a RoboDoctor-managed terminal, RoboDoctor captures the execution context automatically.

For example:

ros2 run my_robot controller

might return:

Package 'my_robot' not found

RoboDoctor associates that failure with the correct terminal session and marks it as needing attention.

The desktop companion

The floating robot is the entry point to the debugging workflow.

It has different states depending on what is happening:

Idle
Healthy
Needs attention
Investigating
Diagnosis ready

If multiple sessions have problems, RoboDoctor keeps those issues separate so the developer can choose which one to investigate.

Double-clicking the companion takes the developer directly into the relevant debugging workflow.

Diagnostic Blackboard

Between terminal activity and the AI is a lightweight diagnostic state layer.

It keeps track of:

active sessions
executions
errors
evidence
investigation state
diagnoses
confidence

This prevents stale errors or example data from becoming the context for a new failure.

Gemma

The reasoning layer uses Gemma 3 4B, running locally through Ollama.

Instead of sending terminal information to a remote API, the core reasoning loop can run locally:

Developer terminal
↓
RoboDoctor
↓
Local Gemma
↓
Diagnosis

Gemma receives structured debugging context rather than an unstructured dump of the entire machine.

Diagnostic investigation

RoboDoctor can go beyond simply repeating the error message.

When the developer opens an issue, a separate diagnostic session can gather additional read-only evidence.

For example, for a ROS2 problem it can investigate information such as:

installed ROS2 packages
active nodes
topics
services
ROS2 environment
package visibility
Python environment
Git state

That evidence is then provided to Gemma so it can reason about the likely root cause.

The diagnostic terminal is deliberately restricted.

Gemma is not given unrestricted shell access, and RoboDoctor does not automatically modify source code or install packages.

The AI investigates.

The developer stays in control.

Why Does Open Innovation Matter?

For RoboDoctor, using an open-weight model locally isn't just a challenge requirement. It changes what the product can be.

A robotics debugging tool can encounter information that developers may not want to send to a remote service:

source paths
terminal output
ROS2 environments
package names
hardware configuration
network information
internal project details

With Gemma running locally through Ollama, the core debugging context can remain on the developer's machine.

There is also no requirement for a paid cloud AI API for the core reasoning loop.

The architecture becomes:

local terminal → local diagnostics → local model → local diagnosis

That makes a local-first debugging companion much more practical for development environments where privacy, connectivity, or API cost matters.

More importantly, Gemma becomes a component that I can build a complete system around rather than the entire product.

The terminal/session system, diagnostic blackboard, deterministic detectors, investigation loop, safety controls, and desktop companion are all part of the application surrounding the model.

That's what open innovation made interesting for this project:

The model becomes a component you can build around, rather than a service your entire product depends on.

My Agent Session

Prize Categories

Best Use of Gemma

RoboDoctor is entering the Best Use of Gemma category.

Gemma 3 4B is not being used as a generic chatbot. It is the local reasoning engine inside a real debugging workflow.

RoboDoctor automatically captures terminal failures, structures the evidence, investigates the relevant environment through a restricted diagnostic terminal, and provides that evidence to Gemma for root-cause reasoning.

Final Thought

I wanted RoboDoctor to feel less like:

"Paste your error here and I'll explain it."

and more like:

"Something broke. I found it. Let me figure out why. Will look inside your system and give you exact solution"

That's RoboDoctor.

DEV Community: Shreyansh Chauhan