A short Python script can make StagingBench easier to inspect without hiding missing measurements. This independent test of AI virtual staging tools uses 10 real room photographs, with one recorded run per tool and room cell. Its per-cell score files contain fields named median_residual_px and inlier_ratio, which can be summarized by room.
The task here is deliberately specific: group rows by their recorded room identifier, collect each field when it is present, and print the median of each collection. There is no tool ranking, and no attempt to infer what an image looks like from a numeric entry.
Keep the collections separate
For this example the per-cell score.json values were flattened into one CSV named score-details.csv, with one row per tool and room. An empty CSV cell is an empty string when read with Python's csv module. It is not a numeric zero. In this data, empty measurements were not recorded because alignment failed.
The script therefore checks for an empty string before converting a value to float. A recorded zero survives that check and participates in the median. An absent measurement does not.
Each field has its own collection. If a row contains a residual but lacks an inlier ratio, the available residual can still contribute. This follows the rule of using values wherever they are present, rather than requiring complete pairs.
Run from the CSV directory
The script reads score-details.csv in the current directory. It uses only the standard library and prints room identifiers in sorted order. The output columns are room, median of median_residual_px, and median of inlier_ratio.
import csv
from collections import defaultdict
from statistics import median
rooms = defaultdict(lambda: [[], []])
with open("score-details.csv", newline="") as source:
for row in csv.DictReader(source):
buckets = rooms[row["room"]]
for i, field in enumerate(("median_residual_px", "inlier_ratio")):
if row[field] != "":
buckets[i].append(float(row[field]))
for room, buckets in sorted(rooms.items()):
values = [f"{median(b):.3f}" if b else "NA" for b in buckets]
print(room, *values)
I ran this snippet against the flattened CSV. The first three output lines were:
01 0.580 0.762
02 0.530 0.782
03 0.560 0.591
Here, 01, 02, and 03 are the recorded room identifiers. The residual summary comes before the ratio summary. The formatting fixes decimal precision for readability; it does not imply that the underlying measurements are accurate to every displayed place.
Understand what the medians combine
For a room, the residual result is the median of the available per-row median_residual_px entries. It is therefore a summary of recorded row summaries. It does not reconstruct individual residuals or calculate a pooled median across all matched points.
Similarly, the ratio result is the median of available inlier_ratio entries. It is not calculated by combining match counts or dividing aggregated counts. The script uses the published ratio values directly.
The published method describes SIFT keypoints, a ratio test, and a RANSAC similarity transform. It does not document the exact residual calculation or explicitly identify the inlier-ratio denominator. The residual field's name indicates pixels, but its coordinate space is not specified. These gaps limit interpretation even when the arithmetic is reproducible.
Preserve absence in the output
If a room has no recorded values for a field, the script prints NA in that position. That preserves a distinction between "no available measurement" and a measured zero. It also avoids calling median on an empty collection.
Do not replace blanks with zeros before aggregation. That would introduce observations that were never recorded and could change the result. Dropping an entire room because its collections are empty would hide another useful fact: the room exists in the input but has no available summary for those fields.
This remains a small test of 10 rooms with one run per cell. Grouping by room helps inspect the published measurements, but it does not establish repeatability or explain why values differ. The data and method are described on the StagingBench methodology page.
Top comments (0)