LLaVA vs Moondream: Testing Local AI Alt Text on an 8 GB Mac

I built a local alt text generator using LLaVA and Ollama. It worked, but generating a single description could take around two minutes on my Mac.

I wondered if a smaller vision model could do the same job faster.

So I tried Moondream.

In my initial test, Moondream took 70 seconds to generate alt text, compared with 140 seconds for LLaVA. Its description was also more detailed. But when I tested Moondream with different prompts, I discovered something unexpected: the model sometimes returned absolutely nothing, even when the request succeeded.

And the timing results turned out to be more complicated than they first appeared.

Here's what I found while comparing the two models, experimenting with prompts, and trying to make local AI work on modest hardware.

First, see the models in action

I tested both models using the same photo of a surprised-looking cat.

LLaVA

LLaVA generating alt text locally. Recorded time: approximately 140 seconds.

Moondream

Moondream generating alt text locally. Recorded time: approximately 70 seconds.

These are the timings from my initial runs, not a controlled benchmark. Later tests suggested that other factors may have affected the results, so I'll come back to that.

Why build an alt text generator?

Adding an image to a website takes seconds. Writing meaningful alt text for every image? That's the part many of us put off.

Alt text provides a textual alternative to meaningful images, allowing screen reader users to understand visual content. Without it, important information can be missed entirely.

I wanted to build a tool that could generate a useful first draft automatically. My requirements were simple:

• It had to run locally on my laptop.
• It had to be free to use, without paid API calls.
• It had to generate concise, useful descriptions.
• It had to be simple enough to use without worrying about the underlying model.

I'd already built the first version using LLaVA and Ollama. The next step was to see whether a smaller model could offer a better experience on my 8 GB Mac.

Meet the contenders

I compared two models through Ollama.

LLaVA was my starting point. The version I used belongs to the 7-billion-parameter model class. It could generate descriptions, but performance on my laptop was a concern.

Moondream was the smaller alternative, with approximately 1.86 billion parameters in the version referenced by this comparison. I wanted to find out whether it could produce useful descriptions with less computational overhead.

The basic application stayed the same: a browser frontend, a Node.js and Express backend, and Ollama running locally.

The backend sends the image and prompt to Ollama's /api/chat endpoint. Switching models meant changing one constant in server.js:

const MODEL = "llava";

// Switch to:
const MODEL = "moondream";

Simple enough. But choosing a model turned out to be only part of the story.

The initial results: Moondream was twice as fast

Here's what I observed in my first comparison.

Measurement	LLaVA	Moondream
Parameters	~7B	~1.86B
Approximate download size	~4.7 GB	~1.7 GB
Time to generate alt text	140 seconds	70 seconds
Output length	54 characters	106 characters
Response to original prompt	Text generated	Empty response
Response to simplified prompt	Not needed in this test	Text generated

Download sizes are approximate. Confirm the exact model versions and sizes installed through Ollama.

Moondream was twice as fast in this initial run. It also produced a longer description.

But there's an important caveat: this was one image, one initial timing per model, and a machine under memory pressure. Later prompt tests revealed that Moondream could respond much faster when called directly.

So rather than treating these numbers as a definitive speed comparison, I looked at what the models actually generated and how they responded to different prompts.

What did the models actually write?

Here's what each model returned for the cat photo.

LLaVA:

Cat with surprised expression, paws raised in the air.

Moondream:

iced blue wall with a gray cat with wide open eyes and mouth, ears perked up, and paws raised in surprise.

The difference is immediately noticeable.

LLaVA focused on the subject and its expression. Moondream included more details about the cat's appearance and the background.

Shorter isn't necessarily worse

LLaVA's description is concise and captures the main action. For a simple photo, that might be all a screen reader user needs.

Moondream's description is more detailed, which could be useful if the background or appearance of the subject matters.

However, additional details also create more opportunities for mistakes. The phrase "iced blue wall" sounds awkward, and every claim about colour or appearance needs to match the actual image.

I also noticed inconsistent descriptions of the cat across my prompt experiments: Moondream described its fur as grey in one response and orange in another. That inconsistency is a reminder that a fluent description isn't necessarily an accurate one.

My takeaway: LLaVA was more restrained, while Moondream offered more detail. Neither model was ready to produce descriptions that I would automatically publish without review.

One image isn't enough to determine which model is more accurate, but it did reveal an interesting trade-off.

The unexpected discovery: prompt sensitivity

This was the most interesting part of the experiment.

My original prompt contained several instructions. I wanted the model to generate one sentence, stay under 125 characters, avoid starting with "Image of," and include any visible text.

LLaVA returned a description. Moondream returned an empty string.

The HTTP request succeeded with status 200, so there wasn't an obvious request failure.

Here's the relevant part of the response:

{
  "model": "moondream",
  "message": {
    "role": "assistant",
    "content": ""
  },
  "done": true,
  "done_reason": "stop",
  "eval_count": 1
}

The model returned no usable text. The response indicated that generation had stopped after one token.

At first, I thought something might be wrong with my backend. But the problem persisted when I tested the model directly.

I tried six different prompts

Instead of guessing, I ran six prompt variants against the same image, removing the generation options for these tests.

Test	Prompt	Result	Time
A	Original multiline prompt with four rules	Empty response	11.4 s
B	Same instructions on one line	Empty response	2.9 s
C	Same rules, without quotation marks	Empty response	4.9 s
D	"Describe this image in one sentence."	Text generated	4.9 s
E	"What is in this image? Answer in one sentence."	Text generated	4.3 s
F	"Write alt text for this image."	Text generated	8.2 s

These results changed how I thought about the problem.

Removing line breaks didn't fix it. Removing quotation marks didn't fix it either. But the shorter prompts consistently produced text in these tests.

I haven't isolated exactly which instruction caused the empty response, so I can't claim to know the underlying reason. The additional instructions were the common difference in the failing prompts.

What I can say is that Moondream was sensitive to how I phrased the request.

A simpler prompt fixed the empty response, but introduced another problem

My working prompt became:

const PROMPT = "Describe this image in one sentence.";

That was enough to get a response.

But there was another catch: a response doesn't mean the model followed every instruction.

When I tried the simpler prompt, "Write alt text for this image," Moondream returned three sentences, with 65 generated tokens. It also started with "The image features"—exactly the kind of opener I wanted to avoid.

In other words, the model could generate text quickly but wasn't always reliable about formatting.

This distinction matters. Prompting can influence the output, but it isn't a guarantee that the model will follow every constraint.

So, which model is actually faster?

This is where I had to be careful with my initial conclusion.

The first comparison showed 140 seconds for LLaVA and 70 seconds for Moondream. But during the direct prompt experiments, Moondream's successful requests took roughly 3–11 seconds.

That discrepancy suggests the original 70-second measurement may include time spent outside the model's actual text generation. Possible contributors include model loading, image processing, machine load, or other application overhead.

I haven't measured those components separately, so I can't say which one explains the difference.

The models also differ significantly in size. On an 8 GB Mac, a larger model can compete with macOS, the browser, and Node.js for memory. When the system is under memory pressure, performance can deteriorate.

To make a fairer comparison, I'd need to run both models repeatedly under the same conditions, close unnecessary applications, verify which model is active, and record Ollama's timing metrics separately.

For example:

const secs = (ns) =>
  ((ns || 0) / 1e9).toFixed(1);

console.log(
  `load: ${secs(data.load_duration)}s | ` +
  `prompt: ${secs(data.prompt_eval_duration)}s | ` +
  `generation: ${secs(data.eval_duration)}s`
);

This helps distinguish the time spent loading the model from the time spent processing the prompt and generating the response.

Until I run those tests, I wouldn't claim that Moondream is consistently twice as fast as LLaVA. The initial results are observations from my machine, not a reliable performance benchmark.

Which model would I choose?

For my alt text generator on an 8 GB laptop, I'd start with Moondream.

It is the smaller model, and my prompt experiments showed that it could produce responses in seconds when given a short prompt. That makes it a promising choice for an interactive tool.

However, I'd keep a few things in mind:

• For a lightweight local tool: I'd start with Moondream and test it on a wider range of images.
• For concise descriptions: Both models produced a usable description of the test photo, but neither should be assumed to be consistently better.
• For detailed scenes: Moondream may provide more visual detail, but each additional claim needs verification.
• For screenshots and charts: I haven't tested either model on these, so I'd evaluate both before choosing.
• For production use: I'd validate the output, handle empty responses, and allow users to edit the generated text.

The key is to choose based on the actual task rather than model size or a single timing result.

What I'd improve next

This experiment gave me a useful starting point, but there's more I'd like to test.

1. Run a proper performance comparison.

I'd test each model several times on the same images, measure cold starts separately, and compare subsequent requests under controlled conditions.

2. Build a more representative image set.

I'd start with 20 images covering photographs, product images, screenshots containing text, charts, and decorative images.

3. Score output quality.

I'd compare accuracy, completeness, concision, and incorrect details, ideally with another person reviewing the descriptions.

4. Explore prompt sensitivity further.

I'd test which individual instructions cause Moondream to return empty responses, and whether prompt wording affects the consistency of its descriptions.

5. Try other small vision models.

I'd like to add gemma3:4b and a small Qwen vision model to see whether either offers a better balance of speed, quality, and memory usage.

These tests would help me make a more confident decision about which model to use.

What I learned

I started this experiment because LLaVA felt slow on my laptop. I ended up learning something more interesting than which model generated a description faster.

Model choice, prompt design, hardware constraints, and application behaviour all influence the final experience.

Moondream showed promise because of its smaller size and its ability to generate descriptions with short prompts. But it was also sensitive to prompt wording, sometimes returned empty responses, and didn't always follow formatting instructions.

The initial timing comparison also showed me why it's important to measure performance carefully before drawing conclusions.

For now, Moondream is my starting point for this project, not a proven winner. I'd want a broader evaluation before making claims about its accuracy or performance relative to LLaVA.

Try it yourself

The project runs locally using Node.js, Express, and Ollama. You can switch models by changing the MODEL constant in server.js and try the same experiment on your machine.

Download the models:

ollama pull llava
ollama pull moondream

Then follow the setup instructions in the repository.

Link to Github

I'd love to hear from other developers experimenting with local vision models.

Have you encountered prompt-sensitive behaviour like this? Which small vision model has given you the best results on modest hardware?

I'd be particularly interested in seeing comparisons that measure not just speed, but also whether the generated descriptions are accurate, concise, and genuinely useful for accessibility.