DEV Community

StagingBench Team
StagingBench Team

Posted on AI-assisted

Read editor ratings and prices from results.json without inventing numbers

A small reporting script should preserve the difference between an observed value and a missing one. For editor ratings, that means printing the number of rated rooms alongside any average. For pricing, it means keeping an absent per photo figure empty. Neither a missing rating nor an unknown price should quietly become zero.

The StagingBench dataset makes the underlying records available through results.json. Download that file and place it at data/results.json before running the example. The code below reads that local path using Python's standard library. It does not download anything.

The field layout it assumes is: tool records live under tools, room records appear under room01 and rooms, and an editor rating is a nonempty realism.scores dictionary with a mean next to it. Hidden tools carry hidden: true and are skipped.

import json
from statistics import mean

with open("data/results.json", encoding="utf-8") as source:
    data = json.load(source)

tools = data["tools"]
entries = tools.items() if isinstance(tools, dict) else enumerate(tools)

for key, tool in entries:
    if tool.get("hidden") is True:
        continue

    name = tool.get("id", tool.get("slug", tool.get("name", str(key))))
    rooms = list(tool.get("rooms") or [])
    first = tool.get("room01")
    if isinstance(first, dict):
        rooms.insert(0, dict(first, room="01"))

    seen = set()
    rated = 0
    values = []
    for room in rooms:
        room_id = room.get("room", room.get("id", room.get("room_id")))
        if room_id is not None:
            room_id = str(room_id).removeprefix("room").zfill(2)
            if room_id in seen:
                continue
            seen.add(room_id)

        realism = room.get("realism")
        if not isinstance(realism, dict):
            continue
        scores = realism.get("scores")
        if not isinstance(scores, dict) or not scores:
            continue

        rated += 1
        value = realism.get("mean", scores.get("mean"))
        if isinstance(value, (int, float)) and not isinstance(value, bool):
            values.append(value)

    average = round(mean(values), 2) if rated and len(values) == rated else None
    pricing = tool.get("pricing") or {}
    print(name, rated, average, pricing.get("per_photo_cheapest_usd"))
Enter fullscreen mode Exit fullscreen mode

Run against the current file, the script prints one line per tool: id, rated rooms, mean of the stored means, and the cheapest per photo USD figure. Four lines from that run show the shapes to expect:

reimaginehome 7 3.71 0.63
virtualstaging-ai 1 3.4 1.25
roomagen 0 None 0.4
homedesignsai 0 None None
Enter fullscreen mode Exit fullscreen mode

Six of the 12 tools have no rated rooms and print None for the average. One tool has no verified per photo price and prints None there too. Neither case is a zero.

Start with visibility. The explicit hidden is True check excludes a tool record only when that flag is the Boolean value true in JSON. Apply that filter before building coverage totals. A report intended to describe the published tools should use the same visibility boundary throughout its calculations.

Next, collect both room locations. Reading only the rooms list could miss a separately stored room01. Reading only room01 could discard the remaining room records. The example brings both into one sequence and normalizes common room identifiers before checking for duplicates. This assumes duplicate identifiers describe the same observation.

That assumption deserves inspection when adapting the script. Records without identifiers cannot be deduplicated reliably by their room name here. Likewise, conflicting copies of the same identified room would require an explicit decision about which record is authoritative. The example keeps the first identified record encountered rather than silently counting both.

The rating test is deliberately separate from room existence. A room counts as rated only when realism is a dictionary containing a nonempty scores dictionary. An image record without that structure does not establish an editor judgment. This keeps the count tied to the evidence being summarized.

The average uses the stored mean field. It does not substitute an automatic measurement or manufacture a value from another part of the record. If a counted rating lacks a numeric mean, the example prints None for the aggregate. That prevents an average over an undisclosed subset from appearing beside the full rated count.

Pricing follows the same principle. A missing pricing object, missing key, or JSON null produces Python None. HomeDesignsAI has no verified per photo value, and reporting that absence preserves the distinction between an unknown price and free access.

Finally, read the count before interpreting the average. Virtualstaging-ai has 1 rating, so its average describes only that room. The ratings come from one editor and were not blind, so even a larger count would not make them a ranking. More observations broaden coverage, but different room sets still limit comparisons. The script should expose those limits through counts and missing values, leaving the interpretation attached to the evidence.

Top comments (0)