geomap-engine: fusing AR pose + object detections into a 2D map, in Rust

#rust #opensource #showdev #robotics

Every AR toolkit (ARKit, ARCore, RoomPlan, Polycam) bundles SLAM,
object detection, and a UI into one closed, heavyweight package. If you
just want the middle piece — turn a stream of {camera pose, object detections} into a clean, deduplicated 2D map of "here's a chair, here's
a door, here's where they are" — there's nothing small and embeddable
that does just that.

So I built geomap-engine: an open-source Rust crate that does exactly this one job.

What it does, what it doesn't

Feed it a Frame (camera pose + intrinsics + a list of 2D bounding-box
detections) and it maintains a SceneMap: one stable entry per real
object, not one per detection.

It explicitly does not:

run SLAM (bring your own ARKit/ARCore/ORB-SLAM3/whatever)
run object detection (bring your own model)
have any UI

It's the glue layer that sits between those and your app/robot/tool.

How it works

Projection — a 2D bbox + camera pose + intrinsics becomes an estimated 2D world position, via ray-plane intersection with the ground plane (v0.1 assumes a known/fixed camera height).
Association — nearest-neighbor match against existing tracked objects, by label + position proximity.
Fusion — repeated observations merge into one object: running- average position, confidence combined as independent evidence so it grows toward 1 with repeated sightings instead of just averaging.
Map maintenance — a single staleness rule handles pruning one-off noise, stale objects, and objects that moved, all at once.

Try it

use geomap_engine::Engine;
use geomap_engine::proto::{CameraIntrinsics, CameraPose, Detection, Frame};

let mut engine = Engine::new();

let frame = Frame {
    pose: Some(CameraPose {
        timestamp: 0.0,
        x: 0.0, y: 0.0, z: 1.5, // 1.5m camera height above the floor
        qx: 1.0, qy: 0.0, qz: 0.0, qw: 0.0, // looking straight down
    }),
    intrinsics: Some(CameraIntrinsics { fx: 500.0, fy: 500.0, cx: 320.0, cy: 240.0 }),
    detections: vec![Detection {
        label: "chair".into(),
        confidence: 0.8,
        bbox_x: 0.45, bbox_y: 0.45, bbox_w: 0.1, bbox_h: 0.1,
    }],
};

let scene_map = engine.ingest_frame(frame);
println!("{}", scene_map.to_geojson_json().unwrap());

Output is GeoJSON-shaped (FeatureCollection of Point features), so
you can drop it straight into geopandas/matplotlib or any map viewer
without writing a converter.

Why Rust + Protobuf

Rust for portability and eventual FFI into iOS/Android. The Frame/
SceneMap schema is Protobuf, so any front-end — Swift, Kotlin, a
Python test harness — can feed it language-agnostically without caring
what the engine is written in.

Status

v0.1 is implemented and tested — 11 tests, cargo build / cargo test
/ cargo clippy all clean, CI running on every push. It's tested against
both a hand-written fixture and a real camera trajectory from the
TUM RGB-D benchmark (with synthetic detections and realistic detector
noise layered on top), to make sure the association/fusion logic holds
up against actual sensor jitter, not just clean synthetic input.

Pre-1.0, so the API may still shift a bit.

Get it

geomap-engine = https://github.com/ghimirebibek/geomap-engine
Repo (MIT licensed)

Feedback, issues, and PRs welcome — especially from anyone who's fought
with ARKit/ARCore object placement before and has opinions about the
projection/association assumptions.

DEV Community