Building a Small Agentic Workflow for Photo-Based Problem Solving
I have been experimenting with a small agentic workflow for a task that starts with an image: take a photo of a problem, extract what matters, decide how to approach it, and produce a step-by-step explanation.
The project is not a large agent framework. It is closer to a practical pipeline with a few specialized steps. That made it a useful way to think about where "agentic" behavior is actually helpful, and where a simpler deterministic flow is enough.
👉 Download Now from the App Store: https://apps.apple.com/us/app/ai-snapsolve-homework-solver/id6763911277
App Store Search: AI SnapSolve
Why a Workflow Instead of One Prompt?
The first version of many AI ideas is a single prompt.
For photo-based problem solving, that quickly feels too loose. A photo can contain handwriting, printed instructions, diagrams, symbols, tables, and irrelevant background noise. Before asking a model to solve anything, the system needs to understand what it is looking at.
So I found it more useful to split the task into smaller responsibilities:
- read the image
- extract the problem
- classify the subject
- choose a solving route
- generate an explanation
- check whether the output is coherent
None of those steps needs to be dramatic. But separating them makes the system easier to reason about.
The Image Agent
The first stage is an image-reading step.
Its job is not to solve the problem. Its job is to create a clean enough representation of the question for the next stages.
That means paying attention to:
- printed text
- handwriting
- equations and symbols
- diagram labels
- page layout
- multi-part instructions
- missing or ambiguous visual context
This stage is easy to underestimate. If the system drops a negative sign or misses a diagram label, a later reasoning step can be confident and still wrong.
The Routing Step
After the problem is extracted, the workflow needs to decide what kind of problem it is.
This is where a lightweight agentic step becomes useful. The system can look at the extracted content and ask:
- Is this algebra, geometry, physics, chemistry, or something else?
- Does it require symbolic manipulation?
- Are units important?
- Is there a diagram dependency?
- Does the problem need multiple images for context?
The routing step does not solve the problem directly. It decides which solving strategy is likely to fit.
In practice, this can be much more useful than sending every input through the same generic path.
Multiple Solvers as Parallel Reasoning
One experiment I liked was using more than one solver.
Instead of treating a single model output as final, the workflow can ask several solving paths to approach the same extracted problem. Each path may emphasize something different:
- a formula-based explanation
- a conceptual explanation
- a verification step
- a method closer to what might be taught in class
The point is not to create noise. The point is to make reasoning inspectable.
If the outputs agree, confidence goes up. If they disagree, that disagreement is useful information too.
The Comparison Layer
The comparison step is where the workflow starts to feel agentic rather than just sequential.
It can ask:
- Do the solutions reach the same answer?
- Are the assumptions compatible?
- Did one path skip an important step?
- Is the explanation understandable enough for a human to review?
This is not a guarantee of correctness. But it is a practical guardrail.
For learning-oriented tasks, I think this matters more than making the system sound confident.
Multi-Image Context
Another lesson: a single image is often not enough.
A problem may span multiple pages. A table may be separate from the question. A diagram may be on one photo and the instructions on another.
Supporting multiple images changes the workflow. The image-reading step has to merge context, not just process files independently. The routing step has to know that page two may depend on page one.
This is a small detail in the UI, but a big detail in the reasoning flow.
What I Would Keep Simple
Not every part needs an agent.
Some steps are better as ordinary rules, validators, or structured prompts. For example:
- checking whether an extracted question is empty
- preserving image order
- limiting output format
- requiring step-by-step structure
- detecting whether multiple solver outputs disagree
The more I worked on this, the more I liked a hybrid approach: agentic where judgment is useful, deterministic where reliability is more important.
What I Learned
The main lesson is that agentic workflows are most useful when they reduce ambiguity between stages.
For photo-based problem solving, the workflow is not just:
image in, answer out
It is more like:
image in, context extracted, task classified, route selected, explanations generated, reasoning compared
That extra structure makes the system easier to debug and easier for a user to inspect.
Final Thought
I do not think every AI app needs a complex agent architecture.
But for tasks that start with messy real-world input, a small agentic workflow can help. It gives each stage a clearer job, makes errors easier to locate, and turns a single answer into something closer to a reviewable reasoning process.
That is the part I find most interesting: not making the AI more theatrical, but making the workflow a little more legible.
Top comments (0)