What I Learned Evaluating AI Image, Video and Audio Models

Tony Sterlin Cantave — Wed, 04 Nov 2026 20:04:28 +0000

Part of my week is spent as a freelance AI evaluator. I review output from AI models that generate images, video, audio and user interfaces, and score each one against a structured rubric. It is detailed work, and it has changed how I think about building software.

The first lesson: following instructions matters more than looking good. An image can be beautiful, but if the prompt asked for three people and the model drew four, it failed the task. A plainer result that does exactly what was asked is the better answer. That rule applies to almost everything I build now. Do what the user asked first, then make it pretty.

The second lesson: false positives are worse than misses. When you flag a problem that is not really there, you send the people training the model in the wrong direction. I would rather be precise and miss a small issue than be loud and wrong. In my own projects, that means I test carefully before I call something broken or fixed.

The third lesson: short, clear feedback beats long explanations. A tight verdict that names the exact issue is more useful than a paragraph of opinion. I bring the same habit to code reviews and client conversations.

Working on the evaluation side also makes it clear how much human judgment still sits behind these systems. Models improve because people check their work, carefully and consistently. For anyone moving into AI or software, I recommend trying evaluation work. You learn quickly what good output looks like, and you start to see the gap between a demo and something people can rely on.

Originally published at tonycantave.com by Tony Sterlin Cantave, a Connecticut software developer and AI model evaluator.

What Logistics and Manufacturing Taught Me About Building Software

Tony Sterlin Cantave — Wed, 04 Nov 2026 20:02:47 +0000

People sometimes assume that moving into software and AI means starting over. For me it felt more like changing tools. I spent more than 15 years in logistics and manufacturing systems: running regional delivery routes for time sensitive shipments, managing production with ERP and SAP integration, working as a cleanroom equipment technician, handling fleet operations and shipping. Today I write Python and JavaScript, evaluate AI output for training, and build the screen management software behind Local Flyer Ads. Almost every habit I rely on as a developer started on a loading dock, a production floor or a delivery route.

Here are the lessons that carried over most directly.

Every system is a flow, and the bottleneck decides the speed

In a production plant you learn quickly that the line moves only as fast as its slowest station. Adding people to a station that is already keeping up changes nothing. Software works the same way. When a script or an app feels slow, the instinct is to optimize whatever code you happen to be looking at. The better move is to measure first and find the one step that is actually holding everything up. Sometimes it is a database query. Sometimes it is a network call that runs once per item when it could run once per batch. Production management trained me to look for the constraint before touching anything else.

Write the procedure before you trust the process

Cleanroom work runs on written procedures. You do not improvise when you are servicing equipment in a controlled environment, because small mistakes are expensive and hard to undo. That habit made me a better programmer. Before I build something, I write down in plain language what it should do, what inputs it expects and what should happen when something goes wrong. That short document becomes the checklist I test against. It also makes the code easier for someone else to pick up later, including future me.

The handoff is where things break

On a delivery route, the drive is rarely the problem. The trouble shows up at handoffs: a package that was scanned at one stop but not the next, a label that does not match the manifest, a pickup window that changed without anyone updating the schedule. Software fails at handoffs too. Most bugs I see live at the boundary between two pieces: one function returns a date as text and the next expects a date object, or an API changes a field name and the code downstream keeps running with empty values. Now I put my checks at the boundaries. Validate what comes in, confirm what goes out, and log both.

Data entry is a design problem

ERP and SAP integration taught me that bad data rarely comes from bad people. It comes from forms that make the wrong thing easy. If a field accepts any text, someone will type a date three different ways. If a dropdown is missing the right option, people will pick the closest wrong one. When I design screens and tools now, including the scheduling side of the Local Flyer Ads software, I try to make the correct input the easiest input. Clear choices, sensible defaults and validation that explains what it wants are worth more than any cleanup script.

Rubrics beat gut feelings

In logistics, performance is measured: on time rates, accuracy, turnaround. You cannot manage what you only feel. My AI evaluation work through Handshake AI is built on the same idea. I score image, video, audio and interface output against written rubrics rather than personal taste, because a rubric makes judgments consistent and explainable. I bring that into my own projects too. Before I call a feature done, I decide what done means and check it against that list.

Reliability is the whole job

A time sensitive delivery route does not get partial credit. The shipment arrives on time and intact, or the route failed. That standard stays with you. In software it shows up as caring about the unglamorous parts: error handling, retries, clear logs and alerts that tell you something broke before a customer does. For a screen network running inside local shops, a display that silently stops updating is the equivalent of a missed delivery, so a lot of my effort goes into making failures visible and recoverable.

If you are making a similar move

If you work in operations, logistics or manufacturing and you are curious about software, you are not starting from zero. You already understand flows, constraints, procedures and accountability. What you need to add is the language to express them in code. Pick one real problem from your own work, something repetitive or error prone, and automate a small piece of it with Python. Certifications helped me build structure around the fundamentals; CompTIA A+, Network+, Security+, Cloud+ and CySA+ each filled in a layer I had only seen from the outside. I am continuing my computer science degree for the same reason.

The tools changed. The way of thinking did not.

Originally published at tonycantave.com by Tony Sterlin Cantave, a Connecticut software developer and founder of Local Flyer Ads.

DEV Community: Tony Sterlin Cantave