Lessons From Using LLMs in a Photo-Based Study App

Lessons From Using LLMs in a Photo-Based Study App

Building with LLMs gets more interesting when the input is not a clean prompt.

For a small study app experiment, I wanted the starting point to be a photo of a homework problem. That sounds simple from the outside: take a picture, send it to AI, get an explanation. In practice, the useful work happens in the middle.

The LLM is important, but it is only one part of the system. The product has to read the image, preserve the context, choose the right reasoning style, and present the answer in a way that helps the student understand the next step.

👉 Download Now from the App Store: https://apps.apple.com/us/app/ai-snapsolve-homework-solver/id6763911277

App Store Search: AI SnapSolve

Lesson 1: The Prompt Starts Before The Prompt

When the user types a question, the prompt is already mostly formed.

With a photo-based workflow, the prompt has to be constructed. A homework image may contain handwriting, printed text, diagrams, tables, irrelevant margins, and multiple questions on the same page. If that visual input is flattened into messy text, the LLM starts from a weak foundation.

So the first lesson was that preprocessing matters.

The system needs to extract the useful parts of the photo while keeping enough structure for reasoning:

• math notation
• labels and units
• diagram context
• question order
• multi-part relationships
• confidence around unclear text

That context becomes part of the prompt, even if the user never sees it.

Lesson 2: Routing Beats One Giant Prompt

It is tempting to solve everything with one large prompt.

That works sometimes, but the answers tend to become generic. A geometry question, an algebra equation, a chemistry reaction, and a physics word problem all need different kinds of reasoning.

I found it more useful to add a routing step before the main explanation. The system first tries to identify what kind of problem it is seeing:

• subject area
• likely concept
• whether a diagram matters
• whether the answer should be numeric or symbolic
• whether multiple images are part of one context

This does not need to be over-engineered. Even a lightweight classification step can improve the final explanation because the LLM is asked to reason in a more specific frame.

Lesson 3: LLMs Are Better When They Show Their Work Carefully

For a study tool, the answer is not the whole product.

If the LLM jumps straight to the result, the student may still be stuck. The explanation needs to slow down at the places where confusion usually happens. That means the model output should make the reasoning visible without becoming a long essay.

The format I kept returning to was:

• identify the concept
• explain the first step
• show the intermediate reasoning
• give the final result
• mention a common mistake when useful

This is less about forcing a chain-of-thought style and more about making the explanation readable. The student should be able to follow the path without decoding a wall of generated text.

Lesson 4: Multiple Answers Can Be Useful, But Only If Curated

One experiment was to run more than one solving path.

This can help in two ways. If different approaches agree, the system gains confidence. If they disagree, the disagreement becomes a signal to inspect the problem again.

It can also help learning. A quadratic equation might be solved by factoring or by using the formula. A physics problem might be organized from known values or from a diagram. Seeing different paths can make a concept feel less rigid.

The catch is that raw multiple answers can overwhelm the user. The interface needs to summarize differences, keep explanations concise, and avoid presenting every model response as equally trustworthy.

Lesson 5: Multi-Image Context Changes The Problem

Real assignments rarely fit perfectly into one image.

A student might upload a worksheet page, then a diagram, then a follow-up question. If each image is treated as a separate task, the LLM loses important context.

The better approach is to merge images into a shared representation before reasoning. This lets the system keep variables consistent, understand earlier givens, and avoid solving part B as if part A never existed.

For LLM workflows, this is a reminder that context assembly is a product feature, not just a backend detail.

Lesson 6: Confidence Should Be Visible

One of the harder parts is handling uncertainty.

LLMs can sound confident even when the image extraction was poor. That is risky in a study app. If handwriting is unclear or the diagram is ambiguous, the tool should not quietly pretend everything is fine.

Some useful guardrails:

• flag low-confidence OCR regions
• ask for a clearer photo when needed
• show assumptions before solving
• compare multiple solution paths
• keep the final answer tied to visible steps

This does not remove mistakes, but it makes them easier to notice.

Lesson 7: Shorter Is Often Better

LLMs are very good at producing a lot of explanation.

That is not always what students need. A simple problem can become more confusing if the response is too long. The right answer length depends on the difficulty of the task and the likely learning gap.

For easy questions, the output should be brief. For harder ones, the explanation should expand around the important steps, not around every possible detail.

This is one of the most practical design lessons: generated text needs editing pressure.

Closing Thought

Using LLMs in a photo-based study app taught me that the model call is only the middle of the workflow.

The system has to turn a messy visual input into a structured problem, choose the right reasoning style, check the result, and explain it in a way that supports learning.

That is the interesting part to me. Not just using an LLM to answer a question, but designing the path around the LLM so the answer becomes easier to understand.