supervision 0.31.0 shipped on October 16, and its release notes open with "supervision now sees depth." A depth map is a single-channel image. So is a thermal frame, and so is an MRI slice like the ones in my TumorLytix project. The next two patch releases, 0.31.1 on October 20 and 0.31.2 on October 24, taught the mask, halo, crop and icon annotators to draw on grayscale scenes.
On October 25 I opened a pull request for one annotator that still crashed on those frames. It was merged on October 26. This post is the short version of that side quest, from the first ValueError to a test pattern I plan to reuse.
The short version
A colour is three numbers. A pixel in a grayscale image holds one number. ComparisonAnnotator tried to put the three numbers into every pixel under a mask, and NumPy refused. The rest of the library already had a rule for this case, keep the blue value, and a helper that applies it. The fix routes the annotator's three colours through that helper and adds eight tests that check grayscale output against the BGR output, channel for channel.
The concept, animated
Figure 1. The grids are the colour layer NumPy allocates with the scene's shape, the outline is the boolean mask the annotator writes under, and the chips are the three values of one colour in OpenCV's B, G, R order. The three states: a BGR scene, a grayscale scene before the fix, and after it.
ComparisonAnnotator highlights where two sets of detections on the same frame differ: one colour for regions only the first set covers, one for regions only the second covers, and one for the overlap. Its docstring suggests it for comparing two models, or a ground truth with a prediction.
Under the hood it builds an empty colour layer shaped like the scene and writes each colour into the pixels under its mask. On a BGR frame every pixel has three slots, and a colour's three values fit. On a one-channel frame every pixel has one slot. NumPy cannot place three values into it, and the first colour written is the overlap, so that was the line that failed. In my repro two 41 by 41 boxes overlapped in a 21 by 21 square, which is where the 441 in the message comes from.
The same thing happened for (H, W) arrays, (H, W, 1) arrays and a Pillow "L" image, with opencv-python installed and with the NumPy fallback that supervision uses when OpenCV is absent.
How I got there
I was exploring supervision's annotators with one question: would they draw on the grayscale frames that medical and thermal pipelines produce? I wrote a probe script that runs every concrete annotator on four kinds of scene: (H, W) arrays, (H, W, 1) arrays, BGRA images and float images.
On develop at commit b651cca, most of the grayscale work had already landed. MaskAnnotator, HaloAnnotator, PolygonAnnotator, ColorAnnotator, CropAnnotator and IconAnnotator all handled single-channel scenes. ComparisonAnnotator raised a ValueError on all three single-channel inputs.
Before writing any code, I checked that nobody else was on it. A search for "ComparisonAnnotator" found only merged PRs, none about grayscale, and no open PR or issue. Mohammad Hijjawi, who wrote the grayscale fixes in #2755 and #2788, had listed the annotators that still failed and left them for separate PRs: BackgroundOverlayAnnotator, HeatMapAnnotator and RichLabelAnnotator. ComparisonAnnotator was on neither list, so I took it and left those three alone.
The fix, in one idea
Figure 2. After #2803 every colour passes through _color_for_canvas. On a three-channel scene the tuple goes through unchanged. On a single-channel scene only the blue value reaches the pixel.
The repo already had the answer in a helper. _color_for_canvas, added in #2755, keeps only the first value of the BGR tuple when the canvas is grayscale. That value is the blue component, and it is what cv2.rectangle and cv2.fillPoly write on single-channel images. The fix routes all three of the annotator's colours through it. On a three-channel scene the helper returns the tuple unchanged, so BGR frames take exactly the path they took before. The change to core.py is 8 lines added and 4 removed, plus a docstring update and a changelog line.
Why blue instead of a luminance mix, which would look more natural? Because the rest of the repo already uses blue, so a frame annotated with BoxAnnotator, MaskAnnotator and ComparisonAnnotator gets one gray value per colour. The convention has a cost, and I put it in the PR description: the default red and green both have a blue component of 0, so on a grayscale frame both "only" regions come out the same gray. If you need to tell them apart, pass colours with different blue components. Changing the convention itself would be a decision for the whole library.
Testing layouts against each other
Hard-coded pixel values would tie the test to today's colours and blend math. So the new test class annotates a BGR scene and its grayscale copy with the same detections, then asserts that the grayscale result equals the first channel of the BGR result. That one assertion runs for box, mask and oriented-box detections on (H, W) and (H, W, 1) scenes, which makes six cases. A seventh checks the legend labels the same way. An eighth checks that a Pillow "L" image comes back as an "L" image with the expected pixels.
All 8 fail on develop and pass with the change, on the NumPy fallback and on opencv-python 5.0.0. I ran both backends because OpenCV is optional now and CI tests both. On the fallback, the full suite went from 5,931 passed to 5,939 passed. One unrelated test timed out on my laptop, on both branches.
What the fix changed for users
If your frames come from an MRI scanner, a thermal camera or a depth sensor, you can now compare two sets of detections on them without converting to BGR first. ComparisonAnnotator accepts (H, W) arrays, (H, W, 1) arrays and Pillow "L" images, and it hands back the same layout it received. Its gray values match the boxes and masks that other annotators draw on the same frame. BGR users see no change. The fix is on develop, with a changelog entry under Unreleased.
BGRA frames are still an open question. They need a maintainer decision: strip the alpha channel and reattach it in the input decorator, or reject BGRA outright. I plan to raise it as an issue.
Thank you, Borda and the supervision maintainers
Borda (Jirka Borovec) approved and merged #2803 on October 26, the day after I opened it. Thank you. For a first-time contributor the path was clear: a CLA bot, a maintainer approving the first CI runs, and an AGENTS.md that asks for a changelog line linking the PR number. That last rule meant one small follow-up commit once #2803 existed.
Thanks also to the maintainers who shipped six supervision releases between October 4 and October 24, and to Mohammad Hijjawi, whose #2755 gave me the helper and whose list of remaining failures told me which fixes were already spoken for.
What I took away
When a library accepts several image layouts, test the layouts against each other. The grayscale result should equal one channel of the BGR result, and that single assertion stays true for any colour, opacity or geometry. A hard-coded pixel test covers one case and breaks the day a default colour changes.
The parity test also makes review cheap. A reviewer can read one assertion and know what all eight cases prove, which matters when a maintainer is reading dozens of outside PRs a month. It is the first test I will write the next time I touch an image pipeline, including my own.
I'm open to ML engineering roles, computer vision included. You can find my work at shreyaboyane.dev, on GitHub as Sonica-B, and on LinkedIn.
Top comments (0)