Detecting Objects in Images from Any Text Prompt (Not Fixed Classes)

Eyasu Asnake — Thu, 15 Jan 2026 19:40:24 +0000

Most object detection systems assume a fixed label set: train a model on COCO, Open Images, or a custom dataset, and you’re limited to whatever classes you trained for.

I’ve been exploring a different approach: prompt-based object detection, where the input is:

an image
a free-form natural language prompt

and the output is localized detections matching the prompt. Even when the concept isn’t a single predefined object class.

I built a small web tool to experiment with this idea.

What it can do

The tool supports complex, compositional prompts, not just object names.

These prompts combine attributes, relations, text, and world knowledge. Things that don’t map cleanly to standard detector classes.

What it’s not good at

This approach is not designed for:

very small objects
obscure, barely visible objects
dense real-time detection out of the box

It performs better on concepts that require reasoning and world knowledge, rather than pixel-level precision on tiny targets.

Why I built it

The main motivation so far has been:

creating training data for highly specific detectors

Instead of manually labeling or training a new detector for every niche concept, this can be used to:

bootstrap datasets
explore whether a concept is learnable
validate prompts before committing to full training pipelines

Try it

I’ve made the tool publicly available as a demo:

👉 Detect Anything – Free AI Object Detection Online
https://www.useful-ai-tools.com/tools/detect-anything

No login required. Images are processed transiently and not stored.

(Please don’t abuse it. Inference is relatively expensive.)

Looking for feedback

I’m especially interested in:

good real-world use cases people see for this
stress-testing and failure modes
where this approach breaks down compared to task-specific detectors