AutoLensAI composites photographs of real vehicles into virtual studios for dealership listings. The setting changes. The vehicle's bodywork, equipment, cabin, and visible condition have to survive that change.
The windows make this harder than the outline suggests.
A side window can show a headrest, a reflection of a building behind the photographer, and another car visible through the far side of the cabin. Move the vehicle into a studio and pieces of the parking lot can come with it. Remove too much and the headrest goes too.
In Negative Space Is a Label, I wrote about teaching a matting model to reject shadows and floor reflections beside a vehicle. A dark patch touching a tire can still be background. The work in this follow-up concerns side glass, where the unwanted surroundings can overlap the vehicle detail I need to preserve.
Getting the outer edge right leaves that problem unresolved.
1. The problem moves inside the outline
A vehicle matte tells the compositor how much of the source image to retain at each location. Around the car, that helps separate a bumper from the lot behind it or preserve the fine edge of a mirror.
Inside a window, retaining the photographed color can retain several things at once.
The glass itself belongs to the vehicle. So do the seats, headrests, pillars, and other cabin surfaces visible behind it. A building reflected across those surfaces belongs to the surroundings. So does the parked car seen through an unobstructed view out the opposite window.
Those contributions don't necessarily occupy separate pixels. A reflection can cross a headrest. Calling that pixel “vehicle” keeps the reflection with it. Calling it “background” can remove part of the headrest.
Treating the whole window as opaque foreground preserves the old scene. Treating it as an empty opening can put the new backdrop where upholstery should be. Adjusting one transparency value across the pane cannot distinguish the seat from the view beyond it.
A glass mask can trace the window perfectly while the correction inside it damages the cabin. The mask locates the work. It does not establish which contributions can be separated.
2. A test passed with the correction removed
In a local glass-authoring experiment recorded on September 8, 2026, the correction was replaced with a function that returned the original image. A check was supposed to reject that result because no glass treatment had occurred.
The check passed anyway.
The operation assembling the image introduced tiny floating-point differences even when its inputs were identical. The check detected those differences and treated them as evidence that the correction had done something.
A regression test with synthetic floating-point data exposed the size of the error: 31 of 988 numeric elements differed, with a maximum absolute difference of 1.1920929e-07. Those are individual numeric values, not 31 altered vehicle pixels and not a measurement of reflection-removal accuracy.
The control had failed for a reason that would be almost impossible to establish by looking at the image. It was accepting arithmetic noise as intervention.
After the fix, unchanged inputs stayed bit-identical. The recorded local caller rejected the source-copy control and wrote no after image. The focused test run finished with 22 tests passing. That record establishes the local test behavior on that date, not coverage of the production system.
This exposed two different obligations. Protected pixels must remain unchanged, but a difference elsewhere does not establish useful work. A test that asks only whether two images differ cannot tell a repaired reflection from rounding error or damaged upholstery.
3. A cleaner window is not necessarily a more accurate one
For a dealership image, I care about whether the result still represents the vehicle being sold. A window that looks cleaner can fail that requirement.
An erased headrest might disappear into a dark cabin without drawing attention. An opening that has been made too transparent might show an attractive studio background where the car should block it.
A brighter interior might look better while giving the wrong impression of the tint. Darkness alone is no reason to erase a seat, just as brightness alone is no reason to erase a reflection crossing one.
Moving an image toward a cleaner reference does not, by itself, measure the glass's physical properties. It does not prove how much light the pane transmitted, or that the removed color came entirely from reflection.
The same applies to detail hidden by a reflection. An inferred seat can be visually convincing without matching the seat in that vehicle. Where the capture does not establish the underlying detail, generating a plausible replacement changes the nature of the photograph.
I am using this system to present an existing car. Preserving uncertainty is preferable to presenting an inferred feature as something the camera recorded.
4. The glass step has no permission to repair the rest of the car
In Part 1, the boundary refiner could work only within an uncertain strip at the edge of the car. The glass step follows the same restriction: it can modify a limited region rather than the whole photograph.
At the glass-processing step, pixels outside the permitted region are copied from the input unchanged. That includes the painted door, window trim, pillars, and other protected areas. The restriction applies to this operation, rather than making a claim that every pixel in a resized or encoded final image is numerically identical to the capture.
The architecture separates proposing a change, limiting where it can be written, and deciding whether the result is acceptable. The correction method remains inside one box.
Architecture
flowchart TD
source[Source photograph] --> proposal[Proposed glass correction]
source --> copy[Original protected pixels]
proposal --> write[Restricted write]
copy --> write
region[Permitted edit region] --> write
write --> check[Protected pixel check]
source -.-> check
check -->|Fail| reject[Reject candidate]
check -->|Pass| review[Cabin and reflection review]
source -.-> review
review -->|Unsupported| reject
review -->|Supported| accept[Accept correction]A change to a protected pillar fails the copy check even when the window looks better. Passing that check sends the candidate to image review; it does not approve the correction.
The following is an independent example using NumPy, Python's array library. It demonstrates restricted copying, not the reflection-removal method. Both image arrays must already refer to the same pixel locations and color representation.
import numpy as np
def apply_patch(
source: np.ndarray,
candidate: np.ndarray,
editable: np.ndarray,
) -> np.ndarray:
"""Demonstrate restricted copying, not glass correction."""
if source.ndim != 3 or source.shape[-1] != 3 or source.size == 0:
raise ValueError("source must be a nonempty H x W x 3 RGB array")
if candidate.shape != source.shape:
raise ValueError("source and candidate shapes must match")
if source.dtype != np.float32 or candidate.dtype != np.float32:
raise TypeError("both images must use float32")
if editable.shape != source.shape[:2] or editable.dtype != np.bool_:
raise ValueError("editable must be an H x W boolean mask")
if not np.isfinite(source).all() or not np.isfinite(candidate).all():
raise ValueError("image values must be finite")
result = source.copy()
result[editable] = candidate[editable]
return result
The example has no blending, resizing, color conversion, or transparency handling. It copies the source and replaces only the permitted RGB samples. Shape validation prevents an incompatible array from being accepted, but matching shapes do not prove that two photographs are aligned.
The boundary itself can be wrong, though. A region drawn too tightly prevents useful work near the edge. A region drawn too loosely can admit trim or bodywork into an area intended for glass. Restricting edits does not make region selection infallible.
The consequence of a tight boundary is often a visible remnant of the original scene. That can be inspected and traced back to the region that blocked the correction. Allowing broader edits to hide it would make the system harder to judge, because the apparent improvement could come from changing the car.
I would rather correct the region that was wrong than give a later stage more freedom to conceal the mistake.
5. Four tests, and what they cannot establish
These tests use a synthetic four-by-four RGB array, not a captured vehicle or a production training sample. They check an unchanged candidate, a restricted edit, damage outside the permitted region, and damage inside it.
The byte comparison is deliberate. The requirement is an exact copy, not an image that is numerically close enough to the original.
import unittest
class CopyRuleTests(unittest.TestCase):
def setUp(self):
self.source = np.full((4, 4, 3), 0.25, dtype=np.float32)
self.source[0, 0, 0] = -0.0
self.editable = np.zeros((4, 4), dtype=bool)
self.editable[1:3, 1:3] = True
def check_protected(self, result):
self.assertEqual(
result[~self.editable].tobytes(),
self.source[~self.editable].tobytes(),
)
def test_unchanged_candidate_preserves_every_byte(self):
result = apply_patch(self.source, self.source.copy(), self.editable)
self.assertEqual(result.tobytes(), self.source.tobytes())
def test_only_permitted_pixels_change(self):
before = self.source.tobytes()
candidate = np.full_like(self.source, 0.75)
result = apply_patch(self.source, candidate, self.editable)
self.check_protected(result)
np.testing.assert_array_equal(result[self.editable], candidate[self.editable])
self.assertEqual(self.source.tobytes(), before)
def test_one_protected_change_is_detected(self):
result = self.source.copy()
result[0, 0, 1] = 0.5
with self.assertRaises(AssertionError):
self.check_protected(result)
def test_inside_damage_passes_the_copy_check(self):
candidate = self.source.copy()
candidate[1, 1] = 0.0
result = apply_patch(self.source, candidate, self.editable)
self.check_protected(result)
np.testing.assert_array_equal(result[1, 1], candidate[1, 1])
self.assertFalse(np.array_equal(result[1, 1], self.source[1, 1]))
if __name__ == "__main__":
unittest.main()
Save both Python blocks, in order, as glass_copy_example.py. Install NumPy with python -m pip install numpy, then run python glass_copy_example.py. The example runs four tests.
The last test is the important limit. It darkens a permitted sample and still passes the protected-pixel check. That is expected: the check knows where a change occurred, not whether the changed sample belongs to a headrest. The test makes that limitation explicit rather than treating a passing preservation check as proof of image quality.
The September 8 review had a separate visual failure. The replacement scene read as a bright inserted opening beside the retained glass and cabin. A wider correction removed more of the original scene, but the preview remained rejected for photographic delivery. Fixing the no-change control did not make that image acceptable.
That is why image review still needs the source photograph. It has to establish whether the same cabin structures remain, whether they still block the view behind them, and whether old surroundings have been removed without deleting vehicle detail. Neither an accurate glass outline nor a count of changed pixels answers those questions.
6. When the evidence runs out
Part 1 separated vehicle, confirmed background, and uncertainty. That last category remains important here, but the reason for uncertainty has changed.
At the outer edge, the question was often where the vehicle ended. Inside the window, I can know where the pane is and still lack enough evidence to separate what the photograph contains.
The practical response is to limit the correction, preserve the parts that are established, and leave unresolved areas visible as unfinished work. A rule about where changes are allowed helps enforce that response. It does not solve the missing-information problem.
A remaining reflection may make the image less polished. A missing headrest makes it inaccurate. The finished listing still has to show the car I photographed.
🎧 Listen to the audiobook — Spotify · Google Play · All platforms
🎬 Watch the visual overviews on YouTube
📖 Read the full 13-part series
Top comments (0)