From Manual Colony Measurements to an Open-Source Robotic Imaging Pipeline

RoTSL — Sun, 26 Jul 2026 21:34:46 +0000

How I replaced a repetitive laboratory workflow with an open-source robot, a computer vision pipeline, and a validated deep learning model.

For decades, measuring fungal colony growth has been a surprisingly manual process. A researcher walks to an incubator, removes a stack of Petri dishes, photographs each one under consistent lighting, opens every image in Fiji/ImageJ, draws a region of interest around the colony, records a diameter, and repeats the whole process the following day. Then the next day. Sometimes for two weeks straight. There isn't anything technically difficult about this workflow.
It's just repetitive. The repetition becomes the experiment.

While working at The Sainsbury Laboratory, I found myself wondering why this process still looked much the same as it did years ago. Cheap single-board computers exist. Affordable motion control hardware exists. Deep learning models capable of segmenting biological images now run comfortably on consumer hardware. Open-source software has reached the point where almost every building block already exists.
What didn't exist was a complete pipeline that connected those pieces together. So I decided to build one.

Each project can be used independently. Together they automate almost the entire workflow from image acquisition through to quantitative biological measurements.
Instead of moving plates out of the incubator every few hours, the robot performs scheduled imaging automatically. Instead of manually outlining colonies, a lightweight U-Net model segments them. Instead of trusting the model because "it looks right", every predicted diameter is compared against manually reviewed reference measurements.By the end of the project, I had developed three connected open-source projects with separate responsibilities:

Project	Purpose
Automator	A Raspberry Pi–controlled robotic platform for scheduled imaging and tracking of QR-labelled Petri dishes.
`metrics-petri`	A production Python package that segments colonies and calculates calibrated growth, morphology, cracking, and texture measurements.
`petrimodel`	The training and diameter-validation repository used to develop, inspect, and evaluate the SmallUNet model distributed with `metrics-petri`.

Each project can be used independently, but they are designed to form a connected workflow. Automator produces consistently named time-series images; metrics-petri converts those images into calibrated measurements, overlays and charts; and petrimodel records how the segmentation model was trained and validated.
The present documentation describes Automator as an automated imaging and tracking system, while metrics-petri describes its scope as growth, morphology, cracking and texture analysis rather than simply “sixteen morphometric measurements.”

That last part mattered to me.

Automation is useful only if the measurements remain trustworthy.

Rather than simply reporting segmentation accuracy, I wanted to know whether the numbers biologists actually use in papers still agreed with manual measurements.The validation ended up being one of the most satisfying parts of the project. Across 605 model-versus-manual measurement pairs—121 original images and 484 augmented derivatives—the pipeline produced the following overall agreement statistics:

Metric	Value
Measurement pairs	605
Original images	121
Augmented derivatives	484
Bias	0.039 mm
RMSE	1.866 mm
R²	0.9956
Lin's Concordance Correlation Coefficient	0.9978
MAE	1.184 mm

These values are reproduced in the current generated analysis report. Bias is defined as model minus manual measurement.
Those aren't segmentation metrics. They're agreement statistics between the automated measurements and a manually reviewed reference workflow, which is ultimately what matters if the software is going to replace manual measurements in practice.
The architecture eventually settled into something surprisingly clean.

+------------------------------------+
| Petri dishes inside incubator      |
+------------------+-----------------+
                   |
                   v
+------------------------------------+
| Automator                          |
+------------------+-----------------+
                   |
                   v
+------------------------------------+
| High-resolution images             |
+------------------+-----------------+
                   |
                   v
+------------------------------------+
| metrics-petri                      |
+------------------+-----------------+
                   |
                   v
+------------------------------------+
| SmallUNet segmentation             |
+------------------+-----------------+
                   |
                   v
+------------------------------------+
| 16 calibrated morphometric metrics |
+------------------+-----------------+
                   |
                   v
+------------------------------------+
| Saved tables, provenance, overlays |
| and growth charts                  |
+------------------+-----------------+
                   |
                   v
+------------------------------------+
| petrimodel training and validation |
+------------------+-----------------+
                   |
                   v
+------------------------------------+
| Model provenance and agreement     |
| analysis                           |
+------------------------------------+

petrimodel is not a stage that ordinary metrics-petri analyses pass through at runtime. It is the companion development and validation repository behind the packaged checkpoint.

One thing I deliberately avoided was building one enormous application.Large laboratory software has a habit of becoming difficult to maintain because everything depends on everything else. If imaging, analysis, model training and validation all live inside one repository, every change risks breaking something unrelated. Instead, I split the pipeline into three repositories with well-defined responsibilities.

Automator knows how to move hardware and capture images.
metrics-petri knows how to analyse images.
petrimodel knows how to train and evaluate models. The only thing they really share is a filename convention and a common understanding that every image contains a standard 90 mm Petri dish.That design decision turned out to simplify far more than I expected.

Most computer-vision pipelines require an external calibration object somewhere in the image. In this workflow, the Petri dish itself provides that reference.
The software detects the dish rim and calculates the physical scale from its pixel diameter and the configured outside dish diameter. The default is 90 mm, matching the dishes used to train and validate the current model, but metrics-petri now accepts other sizes through the --dish-size-mm option.
For a configured dish diameter D and detected image diameter d, the calibration is:

millimetres per pixel = D / d

This removes the need to place a ruler or calibration grid in each photograph while allowing area, perimeter, diameter, displacement and filament-length estimates to be reported in physical units.
The current CLI explicitly supports, for example, --dish-size-mm 60; 90 mm is the default rather than the only permitted size.

The workflow I wanted

Before writing the controller, I wrote down what an ideal experiment should look like. A researcher would prepare and label the Petri dishes, scan the workspace, select the detected plates and configure each plate’s name, imaging interval and end time through the Automator interface.
The robot would then execute the scheduled run: move to each selected plate, enter the capture position, acquire and save the image, reset safely and repeat when the next capture became due. The web interface would provide live status, normal pause and resume controls, image previews and methods for retrieving the captured data.

A segmentation model isolates each colony. Sixteen morphometric measurements are calculated automatically. Time-series plots appear. Overlay images allow every prediction to be inspected visually.The measurements are exported as CSV and JSON files ready for statistical analysis.
That was the goal.
The rest of this article explains how I built each part of that pipeline, what worked, what didn't, and why I made some design decisions that might initially seem a little unusual.

Building Automator

The robot came first. That might sound backwards. Most image analysis projects begin with a dataset and a neural network, then worry about how new images will arrive later.
I had the opposite problem.

I knew I could always train a better segmentation model in the future. What I couldn't buy was consistent image acquisition. Every experiment depended on somebody remembering to photograph plates at the same time every day, under roughly the same lighting, from roughly the same height, while trying not to disturb colonies growing inside the incubator. "Roughly" isn't something computer vision likes. If images are captured under changing illumination, with different camera heights or slightly different viewing angles, the segmentation model ends up learning those inconsistencies instead of the biology.So before thinking about neural networks, I wanted a robot that could produce the same photograph every single time.

Why not buy a commercial system?

Commercial colony imaging systems certainly exist. They're also expensive, closed source, and usually tied to proprietary hardware or consumables. I wasn't trying to compete with those systems. I wanted something that another research group could reproduce using readily available parts, a 3D printer, and open-source software.That decision shaped almost every engineering choice that followed. The entire machine is built from off-the-shelf components.

Component	Choice
Controller	Raspberry Pi 4B (8 GB)
Motion controller	MKS Robin Nano V3.1
Motion firmware	Klipper
API server	Moonraker
High-resolution camera	Raspberry Pi Camera Module 3
Detection camera	USB webcam
Motion	Three NEMA17 stepper motors
Lid handling	MG995 servo + vacuum suction cup
Lighting	5 V LED strip
Chassis	PETG printed components + aluminium extrusion

None of these components are exotic. In fact, several came from the desktop 3D printing ecosystem.That wasn't accidental.

Reusing the 3D printing ecosystem

Building reliable motion systems from scratch is difficult.
Fortunately, thousands of people have already solved exactly that problem.Modern desktop 3D printers routinely achieve sub-millimetre positioning accuracy over hundreds of thousands of movements. Instead of reinventing that work, I borrowed it.

Automator uses Klipper as its motion firmware.
Klipper normally controls printers.
Here it controls a laboratory robot.

+---------+     +-------+     +------------+     +---------+     +-------+
| Browser | --> | Flask | --> | Moonraker  | --> | Klipper | --> | STM32 |
+---------+     +-------+     +------------+     +---------+     +---+---+
                                                                     |
                 +-------------------+-------------------+-------------+-------------------+-------------------+
                 |                   |                   |                                 |                   |
                 v                   v                   v                                 v                   v
           +-----------+       +-----------+       +-----------+                     +---------+       +----------+
           | X Motor   |       | Y Motor   |       | Z Motor   |                     | Servo   |       | Vacuum   |
           +-----------+       +-----------+       +-----------+                     +---------+       +----------+

Klipper handles everything timing sensitive.

The Raspberry Pi doesn't generate step pulses directly. Instead it sends high-level movement commands to Moonraker, which forwards compiled motion plans to the STM32 microcontroller. That separation has a few advantages.
First, the Raspberry Pi remains free to handle cameras, QR decoding, scheduling and the web interface without worrying about real-time motor control.
Second, every mechanical parameter lives inside a version-controlled configuration file. Changing acceleration or travel limits doesn't require recompiling firmware.
It requires editing a text file. For a research project where hardware evolves constantly, that's a much more pleasant workflow.

Two cameras, two jobs

Early prototypes used a single camera. That quickly became awkward. The camera needed two completely different viewpoints.
One job required a wide field of view for finding Petri dishes and reading QR codes.
The other needed a close-up, distortion-free image suitable for quantitative measurements.
Trying to satisfy both with one camera meant compromising both.
Eventually I split the responsibilities.

+----------------+
| USB Camera     |
+-------+--------+
        |
        +--------------------+--------------------+
        |                    |                    |
        v                    v                    v
+-----------------+   +-----------------+   +-------------------+
| Plate detection |   | QR decoding     |   | Workspace preview |
+-----------------+   +-----------------+   +-------------------+


+----------------+
| Pi Camera      |
+-------+--------+
        |
        v
+------------------------+
| High-resolution stills |
+-----------+------------+
            |
            v
+----------------------------------+
| Colony analysis / metrics-petri  |
+----------------------------------+

The USB webcam remains active almost continuously.It watches the workspace, detects plates, reads QR codes and provides the live preview shown in the browser.The Pi Camera Module 3 stays idle until the robot reaches imaging position.Only then does it switch on the LED lighting, capture the still image and immediately switch everything off again.Besides reducing power consumption, this also keeps colonies in darkness between imaging cycles.

Opening a Petri dish without human hands

This was probably the most mechanically challenging part of the project. Photographing colonies through closed lids isn't ideal. Condensation creates reflections. Plastic scratches confuse segmentation. Lighting becomes inconsistent. Removing the lid solves all of those problems. Unfortunately, removing the lid usually requires a person. I wanted the robot to do it instead. After experimenting with several gripper designs, I settled on a vacuum suction cup.
The imaging sequence became:

+------------------+
| Move above plate |
+--------+---------+
         |
         v
+------------------+
| Lower Z axis     |
+--------+---------+
         |
         v
+------------------+
| Enable vacuum    |
+--------+---------+
         |
         v
+------------------+
| Lift lid         |
+--------+---------+
         |
         v
+------------------+
| Rotate lid aside |
+--------+---------+
         |
         v
+------------------+
| Capture image    |
+--------+---------+
         |
         v
+------------------+
| Replace lid      |
+--------+---------+
         |
         v
+------------------+
| Release vacuum   |
+--------+---------+
         |
         v
+------------------+
| Return home      |
+------------------+

It sounds simple. It took far longer than I expected.
The vacuum had to hold firmly enough to lift the lid without deforming it. The servo needed enough torque to rotate the lid reliably. Travel heights had to avoid collisions with neighbouring dishes. Even something as mundane as tubing length affected repeatability because longer tubes increased the time needed to establish vacuum pressure. Those details rarely appear in papers, but they're where most of the engineering time went.

QR codes as metadata

I wanted every image to explain itself. Instead of creating spreadsheets that later needed matching against photographs, each Petri dish carries a QR code before the experiment even begins.

The encoded string contains:

experiment date
strain
growth medium
treatment
viewing orientation

Once scanned, that metadata becomes part of every captured filename.

YYYYMMDD_P001_D03_MAGNAPORTHE_PCBM_CONTROL_TOP.JPG

This looks like a filename. It's actually an like an API between repositories. metrics-petri can recover experiment information without requiring another database. That meant the robot didn't need to know anything about image analysis. It simply writes well-structured filenames.

Safety first

Laboratory robots shouldn't assume software always behaves correctly. Automator therefore includes several hardware safety features.

A latching emergency stop physically disconnects power.
Door sensors pause imaging if the enclosure opens.
Axis end stops protect against over-travel.
Motion can only resume after the enclosure is closed.

The important point is that these protections exist independently of Python. If the Raspberry Pi crashes, the emergency stop still works. Software safety is useful. Hardware safety is essential. Anyone reproducing it remains responsible for electrical protection, mechanical guarding, external emergency-stop hardware, risk assessment and compliance with local laboratory regulations.

A browser interface instead of a conventional desktop application

I decided early not to build a conventional desktop GUI. The Raspberry Pi already hosted the controller, camera services and imaging schedule, so Automator exposes its operational interface through a browser.
The current system supports two normal access modes. A user can connect a MacBook directly over Ethernet and open the interface at
http://192.168.50.2:8080/ or http://automator.local:8080/.

Alternatively, a monitor, keyboard and mouse can be attached directly to the Raspberry Pi, which boots into a full-screen local kiosk.
The interface guides users through workspace scanning, plate detection, per-plate configuration, QR-label generation, scheduled imaging, status monitoring and image retrieval. The two access methods should not be used simultaneously because doing so can introduce latency.
Captured images can be viewed or retrieved through the interface or exported to USB. The current storage policy retains images for 21 days and automatically keeps the data directory below 5 GiB.(These access and retention rules are now explicitly documented.)

The first image

The first successful automated image wasn't particularly exciting. It looked almost identical to a photograph I could have taken myself. That was exactly what I wanted. The point wasn't to produce a spectacular image. The point was to produce the same image every single time. Once I had consistent image acquisition, everything else became easier.

From Images to Measurements: Building `metrics-petri`

Once I had a robot capable of producing consistent images, the next question was obvious. Now what? A folder containing hundreds of JPEGs isn't especially useful on its own. Someone still has to extract biological measurements from every image.
That usually meant opening Fiji/ImageJ, calibrating the image, outlining the colony, measuring its area or diameter, saving the results, and repeating the process for every photograph.
That was exactly the bottleneck I wanted to remove. Rather than writing another GUI application that automated mouse clicks, I wanted something that behaved like any other Python tool.

Install it.
Point it at a folder.
Receive measurements. That became metrics-petri.

Unlike Automator, which is tightly coupled to hardware, metrics-petri is intentionally hardware agnostic. It doesn't care where the images came from. They might have been captured using Automator. They might have been photographed by hand.As long as the image contains a Petri dish, the pipeline is the same.

Installing the package

I wanted installation to be almost trivial.

pip install metrics-petri

That command installs the complete analysis pipeline, including the trained segmentation model. One design decision I'm particularly happy with was bundling the production checkpoint directly inside the Python wheel.Many computer vision packages require downloading a model after installation. That sounds reasonable until somebody tries to run the software on a machine without internet access or discovers the download link has disappeared two years later.

Instead, the wheel already contains the production checkpoint. The package simply works after installation. If users prefer another checkpoint, metrics-petri can also load one supplied through an environment variable, a command-line argument, or download one automatically from Hugging Face as a fallback.
The search order looks like this:

UNET_MODEL
      │
      ▼
Local model file
      │
      ▼
Bundled checkpoint
      │
      ▼
Hugging Face

That gives flexibility without making the default installation fragile.

One command

Most users only need a single command.

metrics-petri analyse ./experiment_images

The software scans the folder, processes every image, and writes a ZIP archive containing

CSV results
JSON results
overlay images
growth charts
metadata
publication-ready figures There isn't a complicated project structure. There aren't dozens of configuration files. The defaults are designed for the imaging workflow produced by Automator.

Five stages

Internally, the analysis pipeline is surprisingly simple.

+-------------+     +-------------+     +------------------------+
| Input image | --> | Detect dish | --> | Calibrate mm per pixel |
+-------------+     +-------------+     +-----------+------------+
                                                    |
                                                    v
+-------------+     +--------------------+     +-----------+
| ZIP archive | <-- | Charts and CSV     | <-- | SmallUNet |
+-------------+     +--------------------+     +-----+-----+
                                                    |
                                                    v
                                      +------------------------+
                                      | Binary colony mask     |
                                      +-----------+------------+
                                                  |
                                                  v
                                      +---------------------------+
                                      | Morphometric measurements |
                                      +---------------------------+

Every image passes through exactly these five stages. Each stage exists for a reason.

Stage 1 — Finding the Petri dish

Before measuring anything, the software needs to understand scale.Computer vision models naturally measure everything in pixels. Biologists rarely care about pixels. They care about millimetres.The solution was surprisingly elegant.Every experiment already uses identical 90 mm Petri dishes.Instead of placing a ruler beside every colony, the software detects the dish itself using a circular Hough transform.

Once the dish diameter is known,

mm per pixel = 90 mm/ detected dish diameter

Every downstream measurement inherits that calibration automatically.

Area.
Perimeter.
Diameter.
Hyphal length.

Everything ends up expressed in physical units rather than image coordinates.That means measurements remain comparable even if different cameras produce different image resolutions.

Stage 2 — Preparing the image

The segmentation model expects a fixed input size. Every photograph therefore passes through a preprocessing stage before inference.

    Raw image

       ↓

   Detect dish

       ↓

     Resize

       ↓

   256 × 256

       ↓

   SmallUNet

Resizing dramatically reduces computation while keeping enough spatial information for accurate segmentation.Once inference completes, the predicted mask is mapped back onto the original image resolution before measurements are computed.That means users get accurate measurements without sacrificing image detail.

Stage 3 — Segmenting the colony

The heart of the package is a deliberately lightweight U-Net variant called SmallUNet. There are now hundreds of segmentation architectures available.Many achieve slightly better benchmark scores than U-Net. Most are also considerably larger. I wasn't interested in chasing another decimal place of IoU if it meant shipping a 400 MB model.

Instead I wanted something that

installs quickly
runs on ordinary laptops
works on Apple Silicon
works on CPUs
is easy to redistribute

SmallUNet ended up being around 23 MB, small enough to bundle directly inside the package.The architecture follows the familiar encoder-decoder pattern.


   Image

     ↓

  Encoder

     ↓

 Bottleneck

     ↓

  Decoder

     ↓

  Sigmoid

     ↓

 Binary mask

Nothing particularly revolutionary.Sometimes the best architecture is the one people already understand.

Stage 4 — Measuring biology instead of pixels

Once segmentation is complete, the interesting work begins. Rather than returning only colony area, metrics-petri calculates sixteen morphometric measurements.These were chosen because they capture different aspects of fungal growth.

Metric	Unit	Description
Area	mm²	Colony size
Diameter	mm	Equivalent circular diameter
Perimeter	mm	Colony perimeter
Eccentricity	—	Shape elongation
Edge roughness	—	Boundary irregularity
Centre displacement	mm	Offset from dish centre
Texture standard deviation	—	Intensity variation
Entropy	bits	Image complexity
Crack area	mm²	Physical crack size
Crack coverage	%	Crack percentage
Crack count	—	Number of crack regions
Frangi hyphal length	mm	Filament estimate
Meijering hyphal length	mm	Alternative estimate
Hybrid hyphal length	mm	Combined estimate
Relative growth rate	day⁻¹	Requires metadata
Absolute growth rate	mm²/day	Requires metadata

Some of these are familiar. Area and diameter appear in almost every fungal growth paper. Others are less common. Edge roughness, for example, captures how irregular colony boundaries become during development. Crack coverage quantifies visible fissures rather than simply reporting that they exist.
Hyphal length estimates use vessel enhancement filters originally developed for medical imaging. Collectively they describe colony morphology far better than a single diameter measurement ever could.

Seeing what the model saw

One thing I dislike about many AI tools is that they quietly produce numbers without showing how they arrived there. metrics-petri deliberately produces overlay images.

Each output includes

the original photograph
detected dish boundary
predicted colony mask

Raw image + Detected dish + Predicted mask = Overlay

If something looks wrong, users can inspect the overlay immediately instead of discovering problems weeks later during statistical analysis. Transparency was far more valuable to me than squeezing a little more processing speed from the pipeline.

Metadata without spreadsheets

Growth measurements become much more useful once time enters the picture.Instead of manually constructing spreadsheets, the package includes a metadata builder.

metrics-petri-metadata

The application generates a simple metadata file describing

experiment
user
dates
imaging days
day codes

Once those dates are available, the package automatically computes relative and absolute growth rates. Because Automator already embeds day codes into filenames, the two projects fit together naturally.
No database.
No proprietary file format.
Just sensible filenames.

More than a command-line tool

Although most users will probably interact with the command line, metrics-petri also ships with several supporting utilities.

Tool	Purpose
metrics-petri	Complete analysis pipeline
metrics-petri-metadata	Metadata editor
metrics-petri-crop	Crop individual dishes from larger photographs
doctor	Environment diagnostics

The doctor command turned out to be especially useful during development. Rather than letting users discover missing dependencies halfway through an experiment, it checks Python, NumPy, Torch, available hardware acceleration, and model availability before analysis begins.A small feature. A surprisingly large reduction in support emails.

Packaging for reproducibility

One lesson I learned during this project is that machine learning software becomes much easier to reproduce when models travel with the code.

Repositories disappear.
Download links break.
Cloud storage changes.
Package indexes remain.

By shipping the trained checkpoint inside the package, the version installed from PyPI remains tied to exactly the model it was validated with. Years from now someone should still be able to install the same version and reproduce the same measurements without hunting for archived checkpoints. That was worth the extra megabytes. Of course, none of this matters unless the measurements actually agree with human measurements. Building the analysis pipeline solved the automation problem.It didn't answer the much more important question.
Can those measurements be trusted?
That question became the motivation for the third project: petrimodel, where the segmentation model was trained, evaluated and compared against a manually reviewed reference pipeline.

Training the Model: Why `petrimodel` Exists

By the time Automator and metrics-petri were working together, I could automatically photograph colonies and calculate measurements from them.That still left one uncomfortable question.

How do I know those measurements are actually correct?
Deep learning papers often stop after reporting segmentation metrics such as Dice score or Intersection over Union.Those metrics are useful for comparing neural networks.They are much less useful for convincing a biologist that a reported colony diameter is trustworthy.

Nobody publishes a paper saying,

"Our Dice score was excellent."

They publish biological measurements.So I wanted to validate the measurements themselves rather than simply the segmentation masks.That decision eventually became the third project in the pipeline: petrimodel. Unlike the other two repositories, petrimodel isn't intended for everyday users. It's the engineering and research repository behind the production model. It contains everything needed to reproduce the model development process, including

the annotated dataset
training scripts
augmentation pipeline
checkpoint selection
validation software
agreement statistics
manual annotation tools

In other words, it answers the question:

Where did the model inside metrics-petri actually come from?

Building a dataset

Computer vision projects live or die by their datasets.The model can only learn from what it sees. For this project I assembled a dataset of fungal colony images captured during laboratory experiments.
The complete training dataset contains:

Dataset component	Count
Original images	121
Augmented images	484
Total image/mask pairs	605

Every image has a corresponding LabelMe polygon annotation describing the visible colony.One detail I deliberately mention is that these are rough annotations. That wording wasn't accidental. The initial polygons are starting points. Later stages of the validation process include manual review and correction.Rather than pretending every annotation is perfect, I wanted the workflow itself to acknowledge that human judgement still matters.

Why SmallUNet?

Machine learning evolves incredibly quickly.Every few months another segmentation architecture appears claiming slightly higher benchmark scores. For this project I wasn't interested in chasing leaderboards. I cared about something much simpler.
Could another researcher install the package, run it on an ordinary laptop, and obtain reliable measurements?

That meant the model needed to be

lightweight
reproducible
portable
easy to distribute

SmallUNet turned out to be a good balance. It follows the familiar encoder-decoder structure introduced by U-Net while remaining compact enough to ship inside the Python package.

+-------------------------+
| 256 x 256 RGB image     |
+------------+------------+
             |
             v
+-------------------------+
| Encoder                 |
| 16 channels             |
+------------+------------+
             |
             v
+-------------------------+
| Encoder                 |
| 32 channels             |
+------------+------------+
             |
             v
+-------------------------+
| Encoder                 |
| 64 channels             |
+------------+------------+
             |
             v
+-------------------------+
| Bottleneck              |
| 128 channels            |
+------------+------------+
             |
             v
+-------------------------+
| Decoder                 |
+------------+------------+
             |
             v
+-------------------------+
| Skip connections        |
+------------+------------+
             |
             v
+-------------------------+
| 1 x 1 convolution       |
+------------+------------+
             |
             v
+-------------------------+
| Binary colony mask      |
+-------------------------+

There's nothing especially fashionable about the architecture. That was part of the appeal. Researchers already understand U-Net. Documentation becomes easier. Maintenance becomes easier.
Future contributors spend their time improving the biology instead of deciphering an exotic network architecture.

Training wasn't just optimisation

The objective wasn't simply to minimise segmentation error. Ultimately the model exists to measure colonies. That led me to experiment with a composite loss function rather than relying on binary cross entropy alone.

Thw report combines three terms:

Binary Cross Entropy
Dice loss
Area consistency loss

Conceptually it looks like this.

Loss = Binary Cross Entropy + Dice Loss + λ × Area Loss

The area component deserves particular attention.Two masks can achieve similar Dice scores while producing noticeably different colony areas. Since downstream biological measurements depend directly on area, encouraging area consistency during training made practical sense.

Searching for the best checkpoint

Rather than choosing the first acceptable model, I trained several versions using different weights for the area-consistency term.
The report/repo evaluates four values.

Area weight (λ)	Outcome
0.1	Candidate checkpoint
0.3	Candidate checkpoint
0.5	Candidate checkpoint
0.7	Selected production checkpoint

According to the reported results, the λ = 0.7 model produced the strongest validation performance and therefore became the checkpoint distributed with metrics-petri. That checkpoint is the one users install automatically from PyPI. An important consequence of this approach is reproducibility. When somebody installs version 3.0.0 of the package, they're using exactly the same checkpoint evaluated in the repo.

Measuring biology instead of segmentation

This is where the project began to differ from many computer vision papers. Instead of asking,

"How well does the predicted mask overlap the annotation?"

I started asking,

"If a biologist measured this colony manually, would they reach the same conclusion?"

Those are related questions. They're not identical.

A segmentation mask can look visually convincing while still producing biased measurements. Likewise, a mask that differs slightly along the boundary may have almost no practical effect on colony diameter.Because the end goal was biological measurement, validation also needed to happen in biological units.

Building a manual reference workflow

To create that reference, I built a separate desktop application specifically for reviewing segmentation outputs.

The interface displays

the original image
the automatically generated mask
detected dish calibration
editable polygons

Instead of accepting the model output blindly, every colony can be inspected and corrected manually. That corrected polygon becomes the reference measurement.

+-----------+     +------------------+     +---------------+
| Raw image | --> | Model prediction | --> | Manual review |
+-----------+     +------------------+     +-------+-------+
                                                   |
                                                   v
+--------------------+     +----------------------+     +-------------------+
| Agreement analysis | <-- | Diameter calculation | <-- | Reference polygon |
+--------------------+     +----------------------+     +-------------------+

The software also supports multipart polygons. That turned out to matter more often than I expected because some colonies develop separated lobes or visible cracking during growth.A single closed polygon isn't always sufficient.

Calibration matters

One design decision appears repeatedly throughout the report. Everything assumes a 90 mm Petri dish. At first glance that might seem unnecessarily restrictive.

Why not support every possible dish size?
The answer is consistency.

Because every experiment uses the same physical standard, calibration becomes automatic.The detected dish diameter immediately provides

millimetres per pixel

No rulers.
No calibration grids.
No additional workflow.

Every image calibrates itself. That same assumption appears in

Automator
metrics-petri
petrimodel

Keeping that convention consistent across all three repositories removed an entire category of user error.

Cross-checking with Fiji/ImageJ

I didn't want the validation to depend on only one measurement workflow. The report therefore includes an independent comparison using Fiji/ImageJ. Twenty-five representative images were measured manually in Fiji after calibrating the Petri dish diameter to 90 mm. Equivalent colony diameters calculated from Fiji were then compared against measurements generated by the validation pipeline. This comparison isn't intended to prove Fiji is wrong or the model is right. It's a sanity check.
If both workflows agree closely, confidence in the overall measurement process increases.

Agreement instead of accuracy

One thing I learned while reading the statistical literature is that agreement matters more than correlation.Two methods can correlate almost perfectly while consistently disagreeing by several millimetres. That's why it reports several complementary statistics.
Among them are

bias
mean absolute error
root mean squared error
Bland–Altman limits of agreement
Lin's Concordance Correlation Coefficient

Together they answer a more useful question.

Can the automated workflow replace the manual one?

Rather than

Do these two sets of numbers increase together?

That distinction became one of the central ideas behind the project.Ultimately the purpose of automation isn't producing beautiful segmentation masks. It's producing measurements that researchers can trust. In the next section I'll look at the validation results in detail, explain what those statistics actually mean, and discuss the engineering lessons I took away after building the complete imaging pipeline from robot to reproducible biological measurements.

Do the Numbers Hold Up? Validating an Automated Measurement Pipeline

Building a robot is satisfying.
Training a segmentation model is satisfying.

Neither of those guarantees the final measurements are scientifically useful.

That became the central question behind the validation work. Could an entirely automated workflow produce measurements that were interchangeable with a careful manual workflow?

Notice the wording.

Not similar.

Not highly correlated.

Interchangeable.

That distinction shaped almost every decision in the validation study. Instead of stopping at machine learning metrics like Dice score or IoU, I wanted to evaluate what researchers actually report in papers: physical measurements. If someone measured the same colony by hand, would they reach the same biological conclusion?

A Three-Part Validation Strategy

Rather than relying on a single comparison, I designed the validation as three complementary studies.

+----------------------+     +-----------------------+     +------------------+
| Fiji / ImageJ        |     | Manual polygon review |     | Model prediction |
+----------+-----------+     +-----------+-----------+     +---------+--------+
           |                             |                           |
           +-----------------------------+---------------------------+
                                         |
                                         v
                              +----------------------+
                              | Agreement analysis   |
                              +----------+-----------+
                                         |
              +--------------------------+--------------------------+
              |                          |                          |
              v                          v                          v
        +-----------+              +-----------+              +-------------+
        | Bias      |              | RMSE      |              | Lin's CCC   |
        +-----------+              +-----------+              +-------------+
                                         |
                                         v
                               +-------------------+
                               | Bland-Altman      |
                               +-------------------+

Each comparison answers a different question.

Comparison	Question
Fiji vs Manual	Is the manual measurement pipeline internally consistent?
Model vs Manual	Does the neural network reproduce manual measurements?
Group and per-image analysis	Does performance remain stable across experiments?

Looking at only one comparison would have left important questions unanswered.

Why Correlation Isn't Enough

It's surprisingly easy to obtain a very high correlation coefficient. Imagine every automated measurement is exactly 2 mm larger than the manual measurement.Correlation would still be almost perfect. From a biological perspective, the software would clearly be wrong.
That is why the report follows measurement-validation literature rather than relying solely on machine learning benchmarks. The goal is to determine whether two methods can reasonably replace one another. Correlation alone cannot answer that.

The Statistics I Chose

The report exhibits several complementary statistics.Each captures a different aspect of measurement quality.

Statistic	Why it matters
Bias	Average difference between methods
MAE	Typical absolute error
RMSE	Larger errors receive greater weight
Bland–Altman analysis	Detects systematic disagreement
Lin's Concordance Correlation Coefficient	Combines precision and accuracy
R²	Overall agreement trend

No single number tells the whole story.Together they provide a much clearer picture of method agreement.

Part 1 — Fiji vs Manual Measurements

Before trusting the model, I wanted to know whether two independent manual workflows already agreed with one another. For this comparison, a representative subset of 25 images was measured using Fiji/ImageJ.Each image was calibrated manually using the known 90 mm Petri dish diameter before colony measurements were extracted.Those measurements were then compared against the measurements produced by the polygon-review workflow developed for petrimodel.

The agreement was reassuring.

Metric	Value
Images	25
Mean bias	−0.036 mm
Mean absolute error	0.333 mm
RMSE	0.422 mm

The average difference between the two manual workflows was only a few hundredths of a millimetre.That doesn't mean the measurements were identical.It means they differed by an amount small enough to establish confidence in the manual reference pipeline itself. Without that step, later comparisons against the neural network would have rested on much weaker foundations.

Part 2 — The Full Validation

Once the manual workflow had been established, I compared the production segmentation model against manually reviewed measurements across the full dataset. This comparison involved 605 image pairs.That's considerably larger than many proof-of-concept studies and provides a much better indication of how the software behaves under realistic laboratory conditions. The overall agreement statistics reported in the report are shown below.

Metric	Result
Images	605
Mean bias	0.039 mm
RMSE	1.866 mm
R²	0.9956
Lin's CCC	0.9978

Those numbers were particularly encouraging because they measure agreement in physical units rather than segmentation overlap. From the perspective of a biologist, colony diameter is far more meaningful than pixel-level IoU.

Visualising Agreement

One of my favourite ways to examine measurement agreement is the Bland–Altman plot. Unlike a scatter plot, it makes disagreement immediately obvious.

+-----------------+     +------------------+     +------------+     +---------------+
| Manual diameter | --> | Average diameter | --> | Difference | --> | Bias analysis |
+-----------------+     +------------------+     +------------+     +---------------+

Instead of asking

"Do larger colonies remain larger?"

it asks

"How far apart are these measurements?"

The reported limits of agreement were approximately

−3.62 mm to +3.70 mm

Importantly, the average bias remained close to zero. That suggests the automated measurements do not consistently overestimate or underestimate colony size.

Looking Beyond Averages

Summary statistics are useful. They can also hide interesting behaviour. For that reason the report also examines agreement

across experimental groups
across individual images
across the complete dataset

This helps identify whether particular growth conditions or colony morphologies systematically challenge the segmentation model. One of the reassuring observations was that the agreement remained stable across these different levels of analysis rather than depending on only a handful of easy examples.

Why Validation Was Harder Than Training

Training the model was relatively straightforward.

Collect images.
Annotate colonies. -Optimise the network.

Validation required much more thought.The difficult part wasn't calculating statistics. It was deciding which statistics actually answered the scientific question.Many computer vision papers stop after reporting Dice score because that's the standard benchmark. My end users aren't comparing segmentation architectures.
They're measuring fungal growth.
That changed the evaluation criteria completely.I found myself reading clinical measurement-validation literature more often than machine learning papers. Concepts like Bland–Altman analysis and Lin's Concordance Correlation Coefficient became much more relevant than another percentage point of IoU. Looking back, I think that shift in perspective improved the project more than changing the neural network architecture ever could.

Engineering Lessons

Working on all three repositories simultaneously taught me several lessons that I probably wouldn't have appreciated if I had focused only on software.

1. Hardware consistency simplifies software

The more repeatable image acquisition became, the simpler the segmentation problem became.Many computer vision challenges originate from inconsistent data rather than inadequate models. Automating the imaging process reduced that variability before the neural network ever saw an image.

2. Constraints can be useful

Supporting only 90 mm Petri dishes initially felt restrictive.In practice it simplified calibration, reduced configuration, and made measurements directly comparable across experiments.Sometimes deliberately limiting flexibility produces a much better user experience.

3. Reproducibility begins with packaging

Machine learning projects often publish code while leaving trained models scattered across cloud storage providers. Bundling the production checkpoint directly inside the Python package ensures that every installation corresponds to a validated model.That small packaging decision makes the software much easier to reproduce years later.

4. Transparency matters

I deliberately chose to generate overlay images alongside numerical outputs. Researchers should be able to inspect what the software measured rather than treating it as a black box.
If a segmentation looks wrong, it should be obvious immediately.Trust grows much more easily when users can see the evidence.

Looking Ahead

Although I'm pleased with where the pipeline ended up, there are still plenty of directions worth exploring. Some ideas are already obvious.

Supporting additional Petri dish sizes without sacrificing automatic calibration.
Running segmentation directly during image acquisition.
Extending the analysis beyond fungal colonies to bacterial or plant pathology experiments.
Exploring newer lightweight segmentation architectures as they mature.

The important point is that the overall architecture doesn't need to change.The robot captures consistent images.The analysis package extracts calibrated measurements. The training repository improves the model.Each component can evolve independently while preserving the workflow. That modularity was intentional from the beginning, and after living with the system for a while, I think it turned out to be one of the best design decisions in the entire project.

Closing Thoughts

Looking back, what I'm happiest with isn't the motion system, the segmentation model, or even the validation statistics.

It's that the entire workflow is open.
Anyone can inspect the hardware.
Anyone can read the code.
Anyone can reproduce the measurements. Scientific software shouldn't be a black box. If another laboratory can take these repositories, adapt them to their own experiments, improve them, and contribute those improvements back to the community, then the project has achieved something much more valuable than simply automating my own workflow. That's exactly what open source should enable.

Getting Started

All three projects are open source and can be used independently, although they are designed to work together.

Project	Description	Link
Automator	Robotic imaging platform	https://github.com/rotsl/Automator
Documentation	Complete hardware and software documentation	https://rotsl.github.io/Automator/
metrics-petri	Colony segmentation and morphometric analysis	https://github.com/rotsl/metrics-petri
PyPI	Installable analysis package	https://pypi.org/project/metrics-petri/
petrimodel repo	Training data and validation resources	https://github.com/rotsl/petrimodel

Installing the Software

Installing the analysis pipeline is intentionally simple.

python -m venv .venv

source .venv/bin/activate

pip install metrics-petri

Verify the installation.

metrics-petri doctor

Typical output looks similar to

Python ✔️

Torch ✔️

NumPy ✔️

Model ✔️

Accelerator: Apple MPS

Ready.

Running an Analysis

Analysing a folder of images requires only a single command.

metrics-petri analyse ./experiment

Outputs are organised into

analysis.zip
└── analysis/
    ├── analysis_full.csv
    ├── analysis_full.json
    ├── metadata.csv
    ├── overlays/
    └── charts/

The overlays are particularly useful because they make every prediction inspectable.Instead of blindly trusting numerical outputs, I can immediately verify whether the segmentation behaved as expected.

Working with Metadata

For longitudinal experiments I generate metadata before analysis.

metrics-petri-metadata

This creates

image_metadata.csv

Once dates are available, metrics-petri automatically calculates

Relative Growth Rate
Absolute Growth Rate without any additional scripting.

Example Python Usage

Although the command line is the primary interface, the package is designed to fit naturally into Python workflows.

from pathlib import Path

from metrics_petri import analyse_folder

results = analyse_folder(

    Path("images"),

    output="analysis.zip"

)

print(results.summary())

That makes it straightforward to integrate the package into larger data-processing pipelines.

Reproducing the Workflow

The complete workflow can be summarised in just a few steps.

+----------------------+     +------------------+     +----------------+
| Prepare Petri dishes | --> | Attach QR labels | --> | Load Automator |
+----------------------+     +------------------+     +--------+-------+
                                                               |
                                                               v
+-------------------+     +-----------------+     +-------------------+
| CSV + JSON        | <-- | metrics-petri   | <-- | Automatic imaging |
+---------+---------+     +-----------------+     +-------------------+
          |
          v
+---------------------------------+
| R / Python statistical analysis |
+----------------+----------------+
                 |
                 v
+-------------------------+
| Publication             |
+-------------------------+

The important point is that the only manual step is preparing the biological experiment itself.

Everything after imaging can be reproduced automatically.

Why I Released Everything

One decision I made early was that the software, hardware, and training pipeline should all be open. Many laboratory automation projects publish a paper while leaving the implementation inaccessible. Others release code without datasets. Some release datasets without trained models. I wanted anyone reading the report to be able to reproduce the complete workflow.

That meant releasing

hardware designs
controller software
Python package
training pipeline
datasets
validation methodology

Separating these into three repositories also made long-term maintenance much easier. Someone interested only in image analysis doesn't need to clone a robotics project. Someone building laboratory hardware doesn't need to understand neural network training. Each repository has a single responsibility.

What I'd Do Differently

No engineering project is ever really finished. Looking back, there are several things I'd like to explore in future versions.

Additional hardware

interchangeable dish holders
automatic focus
environmental sensors
barcode support

Computer vision

instance segmentation
uncertainty estimation
confidence maps
real-time inference

Analysis

additional fungal phenotypes
multispectral imaging
bacterial colony support
richer temporal modelling
moving from macroscopy to microscopy

The important thing is that none of those ideas require rewriting the architecture.Each repository can evolve independently.That modularity was one of the original goals, and I think it has proven worthwhile.

Final Thoughts

The most rewarding part wasn't getting a robot to move accurately or training another segmentation network. It was seeing three independent open-source projects fit together into a workflow that feels coherent. Each project solves a different problem. Together they remove a surprising amount of repetitive laboratory work while keeping every stage transparent and reproducible.I don't expect this pipeline to be the final answer for automated fungal phenotyping. I hope it becomes a foundation that other researchers can build upon. If someone replaces the robot, swaps in a different segmentation model, or adapts the analysis pipeline for another organism, then the architecture has done exactly what I hoped it would do. Open science isn't just about making code available. It's about making the entire engineering process understandable, reproducible, and useful to the next person.If this work saves another researcher a few hours every week, or inspires someone to build a better version, then publishing it openly has already been worthwhile.

References

How to cite

If you use this software, hardware designs, or derivative works in academic publications, preprints, theses, posters, or internal reports, please cite relevant repository details of which can be found below.

(Automator)10.5281/zenodo.21609180
(metrics-petri)10.5281/zenodo.21628852
(petri-model)10.5281/zenodo.21628765

DEV Community: Automation in Biology Community