The AI Recognized the Food. That Didn't Mean the Calories Were Right.

SidneyOkine — Sat, 07 Nov 2026 22:31:26 +0000

What building an AI-assisted meal scanner taught us about uncertainty, user control, and the difference between a working model and a useful product.

A photograph of a plate of rice and stew doesn't tell you how much oil went into the stew.

It doesn't tell you whether the rice weighs 150 or 250 grams. It doesn't tell you about the ingredients underneath the sauce, either.

A computer vision model might recognise the food perfectly.

And still get the calorie estimate wrong.

That's an interesting problem when you're building a feature that's supposed to make nutrition tracking easier.

At TheSidneySpace, we work on LTG Lifestyle, a mobile wellness application that includes AI-assisted meal scanning.

Users can photograph a meal, receive an estimate of its nutritional content, review the results, and adjust them before saving.

The feature sounds straightforward when described in a sentence.

But it raises a question that's relevant to almost anyone building AI-powered products:

What should an application do when its model produces a plausible answer that might not be correct?

Recognition isn't measurement

There's an important difference between identifying an object and understanding its properties.

A model recognising a banana is one thing.

Estimating the calories in a mixed meal is another.

Consider a bowl containing rice, vegetables, and chicken.

The model might correctly identify all three.

But the nutritional calculation also depends on quantities, preparation methods, ingredients, and information that may not be visible in the image.

Two meals that look similar can have meaningfully different nutritional values.

The difficulty isn't necessarily that the model has failed.

The difficulty is that the image doesn't contain enough information to determine an exact answer.

That's a limitation of the input, not something that can always be solved by choosing a more capable model.

And it has implications for how the product should behave.

A precise-looking number can be misleading

Imagine the scanner returns:

642 kcal

It looks convincing.

The interface has produced a specific number, presumably after analysing the photograph. The user might reasonably assume that number is accurate.

But what does 642 actually mean?

Is it based on an estimated serving size? How much of the meal was recognised correctly? Were cooking oils accounted for? How reliable were the ingredient assumptions?

Displaying additional decimal places wouldn't answer any of those questions.

This is where interface design becomes part of the reliability problem.

Developers often think about reliability in terms of availability, error rates, response time, and whether a service returns valid data.

Those are important.

But an AI-powered feature can return a technically valid response and still communicate something misleading.

A successful API request doesn't necessarily produce a trustworthy user experience.

We need to treat those as separate concerns.

Don't confuse a generated result with a confirmed record

One useful way to think about AI-assisted data entry is to separate the model's output from the information the user has accepted.

For a meal scanner, that suggests a few distinct states:

Processing: The image is being analysed.
Draft: The model has returned an estimated result.
Review: The user can inspect and correct the result.
Saved: The user has accepted the information.
Failed: The scan couldn't produce a usable result.

These distinctions matter.

A generated estimate should not automatically become part of someone's nutrition history simply because an API request succeeded.

The user may need to change the food identification, adjust the quantity, or discard the result entirely.

Separating those states creates room for that decision.

It also encourages cleaner boundaries between AI-generated information and user-confirmed data.

For developers, that's worth considering at the data-model level, not just as a button added to the interface later.

The edit screen is part of the AI feature

It's tempting to spend most of the development effort improving the model request.

You experiment with prompts, response formats, image handling, and different approaches to extracting structured data.

Then you build a result screen.

But if the result can be wrong, correcting it shouldn't feel like an afterthought.

In LTG, users can review and adjust meal estimates before logging them.

That interaction is important because the person using the app often knows something the model doesn't.

They know whether the bowl was particularly large.

They know if the meal contained an ingredient the camera couldn't see.

They know whether the food was prepared differently from what the model assumed.

AI can make an initial estimate easier to obtain.

It shouldn't make that estimate unnecessarily difficult to correct.

This principle extends well beyond nutrition.

Document extraction, receipt scanning, transcription, AI-generated forms, and automated categorisation all face versions of the same problem.

The initial output may be useful without being final.

Design for that distinction from the beginning.

Failure doesn't always look like an error message

When integrating an AI service, there are obvious failures to account for.

The network request times out.

The provider is unavailable.

The response doesn't match the expected schema.

The image cannot be processed.

Those are relatively familiar engineering problems.

A harder category involves responses that appear valid but are not useful.

Perhaps the system identifies the wrong food.

Perhaps it returns incomplete nutritional information.

Perhaps the result contains a quantity that makes little sense for the meal.

The JSON parses correctly. The request returns successfully. Nothing crashes.

But something is still wrong.

That's why the success of the transport layer, the validity of the response structure, and the usefulness of the result should be evaluated separately.

A successful HTTP response isn't the end of validation.

What should happen when the scan fails?

Suppose someone has taken a photograph of their lunch.

The upload begins, the processing indicator appears, and then the connection drops.

What now?

An application could show a generic error and send the user back to the beginning.

Or it could preserve the context, explain the problem, and offer a sensible next step.

For this type of workflow, useful fallback behaviour might include retrying the request, allowing manual meal entry, or preserving a draft so the user doesn't have to reconstruct the entire interaction.

The right choice depends on the product's architecture and constraints.

But the larger point is straightforward.

An AI feature should not require AI to be available for the entire application to remain usable.

The model is there to reduce effort.

When it isn't available, it shouldn't create unnecessary additional effort.

Observability needs to go beyond API success

If you're trying to understand whether an AI-assisted feature is working well, request success rates only tell part of the story.

Consider the difference between these events:

A scan was started.
The scan service returned a response.
The result was presented to the user.
The user edited the result.
The user saved the meal.
The user abandoned the flow.

Each event answers a different question.

A high number of successful scan requests might look encouraging.

But what if many users leave after seeing the estimate?

What if the results are frequently edited?

What if users complete the scan but rarely save the meal?

None of these observations automatically proves that the model is inaccurate.

A correction might reflect personal preference, a portion adjustment, or additional information the user knows.

But these events provide useful starting points for investigation.

They help shift the conversation from "Is the API working?" to "Is the feature helping people accomplish what they came to do?"

When recording analytics, it's also worth being deliberate about what information is actually necessary. You can often understand a workflow's completion and failure patterns without collecting identifiable meal images or detailed personal nutrition records in your analytics platform.

The model isn't the whole product

There's a tendency to evaluate AI features primarily by the sophistication of the model behind them.

Which provider are you using?

How accurate is it?

How quickly does it respond?

How much does each request cost?

These are legitimate engineering questions.

But they don't describe the entire experience.

A less sophisticated model with understandable results, practical corrections, and reliable fallback behaviour may provide a better experience than an impressive model surrounded by confusing interactions.

The product has to account for everything that happens after the inference is complete.

It has to help the user understand the result, make a decision, and continue with whatever they were trying to accomplish.

That's where a great deal of the engineering and design work belongs.

The question worth asking before shipping

We've found it useful to think about AI-assisted meal scanning as an information-entry problem rather than simply an image-recognition problem.

The photograph is an input.

The model produces an estimate.

The interface helps someone review that estimate.

The application records the information they've chosen to accept.

Each stage has a different responsibility.

And each deserves attention.

Because the objective isn't to demonstrate that a model can recognise someone's lunch.

It's to help someone keep a useful record of what they've eaten without making the process unnecessarily complicated.

An AI feature doesn't become trustworthy just because it produces an answer. It becomes more trustworthy when the product makes room for the answer to be wrong.

That's a principle worth carrying into the next AI feature you build.

DEV Community: TheSidneySpace