We build computer-vision systems for a living: shelves in stores, cattle on farms, parts on a conveyor. On almost every one, the thing that eats the lots of time is getting clean data out of the field.
Usually it goes like this. You tell the client "just photograph your shelves," and you get a pile of images in WhatsApp and email, half of them blurry and dark, no idea which photo is which SKU or which animal. Then someone on your side copies them around by hand.
Тraining the model is rather easy now. What decides whether you get a working system or a demo is the clear data, and it's grunt work and it's less fun than training models.
What we learned collecting field data:
Capture camera metadata at the source. Intrinsics (fx, fy, cx, cy, focal length) and EXIF should be saved with every photo. If you ever want to measure anything from the image, you need this at capture time and you cannot recover it later.
Assume there is no signal on the mobile device. Save the capture on the device first, then upload with a resumable protocol, because a warehouse basement or a field will constantly drop your connection. Resume from the last unsent file.
Guide the shot. Show the person a reference angle, and run a cheap on-device check for blur and exposure before the photo is saved. A two-second "retake this" prompt beats finding blurry captures later.
Keep the collection scenario in config - what you collect and in what order should change with an edit to a config file.
Bundle each capture as one unit. When someone submits, the form data and the photos (with their metadata) get packed into one record tied to the project. So an image never floats around on its own, and no one has to work out later which photo belongs to which cow or which shelf.
We'd rebuilt some version of this for every project, so we finally made it a real thing and open-sourced it. A Flutter app for offline field capture, plus a Django admin to define projects and review what comes back. The scenario config lives in Git.
Repo: https://github.com/epoch8/data-collector
It's early: no background upload yet, but it already beats the messenger-and-spreadsheet loop, though.
Top comments (0)