DEV Community: Ryan Hsu

I Ran Five Small Multimodal Models on a Jetson. The Fastest One Was Not the Best Baseline.

Ryan Hsu — Thu, 18 Jun 2026 02:45:16 +0000

I have been building WearEdge Pro, a wearable industrial edge AI runtime. Think of a frontline operator wearing a smart-glasses device, capturing a first-person image of a machine, and getting back a structured action card from a local Jetson box.

The key phrase is "structured action card." This is not a chat demo. In a factory setting, an answer needs an audit trail, a mode boundary, a human-confirmation gate, and a way to hand off to maintenance, quality, EHS, or work-instruction workflows.

I recently tested five compact multimodal models on the same Jetson path:

Gemma 4 E2B
Qwen2.5-VL-3B
SmolVLM2-2.2B
InternVL3-2B
Qwen2.5-Omni-3B

The goal was not to crown a universal benchmark champion. I wanted to know which model was the best current baseline for an industrial edge Agent runtime.

The Harness

Every model was exposed through a local OpenAI-compatible llama.cpp endpoint on the Jetson. Each model got the same five prompts and images:

maintenance
quality inspection
changeover
work instruction
hazard review

The main run used 560 image tokens, which matches the current WearEdge gateway budget. Qwen2.5-VL also got a 1024-image-token pass because grounding can improve with more visual tokens.

The Results

Model	Completion	Avg latency	Takeaway
Gemma 4 E2B	5/5	37.51s raw	Best product baseline
Qwen2.5-VL-3B	5/5	39.72s	Best OCR challenger
SmolVLM2-2.2B	5/5	12.84s	Fastest, but weak grounding
InternVL3-2B	5/5 only after ctx4096	80.35s	Too slow/risky for baseline
Qwen2.5-Omni-3B	5/5	50.09s	Interesting future audio/video branch

SmolVLM2 was the speed star. But the answers were often too generic for real operator guidance. In changeover and work-instruction tasks, it returned fields that looked more like placeholders than grounded industrial guidance.

Qwen2.5-VL was the most impressive challenger. It nailed a changeover OCR task with LABELER-FL1 and SKU-C500, where Gemma had a machine-label typo. It also produced useful IQC defect scores. If I were building a pure OCR or visual inspection assistant, I would take Qwen very seriously.

InternVL3 reminded me that token speed is not the whole story. At 2048 context it failed three of five tasks with context errors. At 4096 context it finished, but the latency was high and one raw IQC answer had unsafe release-style wording.

Qwen2.5-Omni ran cleanly, but its strongest value is probably a future audio/video workflow rather than this current image+text industrial baseline.

Why Gemma Still Won

Gemma 4 E2B did not win every micro-test. It stayed the baseline because it fit the product runtime:

local Jetson deployment
structured multimodal prompts
long-context workflow design
function-calling-oriented architecture
deterministic guards
human confirmation
action cards
audit logs

In an industrial setting, "fast and fluent" is not enough. The model has to behave inside a system that can say: this came from this image, this route, this required field, this action boundary, and this audit record.

That is why Gemma remained the WearEdge baseline, while Qwen2.5-VL became the serious A/B challenger for OCR-heavy branches.

Lesson Learned

Edge AI model selection is not just a leaderboard exercise. The right question is:

Can this model run locally, understand the evidence, obey the workflow boundary, and produce an action that the system can audit?

For WearEdge Pro today, the answer is Gemma 4 E2B as the baseline, Qwen2.5-VL as the next challenger, and a clear path to keep testing without pretending every benchmark cell means the same thing.

Public artifact link: Benchmark results and public discussion: https://www.hackster.io/ryanon2008/wearedge-pro-jetson-edge-ai-agent-50ec35

I Ran Five Small Multimodal Models on a Jetson. The Fastest One Was Not the Best Baseline.

Ryan Hsu — Wed, 17 Jun 2026 11:55:59 +0000

I recently tested five compact multimodal models on the same Jetson path:

Gemma 4 E2B
Qwen2.5-VL-3B
SmolVLM2-2.2B
InternVL3-2B
Qwen2.5-Omni-3B

The goal was not to crown a universal benchmark champion. I wanted to know which model was the best current baseline for an industrial edge Agent runtime.

The Harness

Every model was exposed through a local OpenAI-compatible llama.cpp endpoint on the Jetson. Each model got the same five prompts and images:

maintenance
quality inspection
changeover
work instruction
hazard review

The main run used 560 image tokens, which matches the current WearEdge gateway budget. Qwen2.5-VL also got a 1024-image-token pass because grounding can improve with more visual tokens.

The Results

Model	Completion	Avg latency	Takeaway
Gemma 4 E2B	5/5	37.51s raw	Best product baseline
Qwen2.5-VL-3B	5/5	39.72s	Best OCR challenger
SmolVLM2-2.2B	5/5	12.84s	Fastest, but weak grounding
InternVL3-2B	5/5 only after ctx4096	80.35s	Too slow/risky for baseline
Qwen2.5-Omni-3B	5/5	50.09s	Interesting future audio/video branch

Qwen2.5-Omni ran cleanly, but its strongest value is probably a future audio/video workflow rather than this current image+text industrial baseline.

Why Gemma Still Won

Gemma 4 E2B did not win every micro-test. It stayed the baseline because it fit the product runtime:

local Jetson deployment
structured multimodal prompts
long-context workflow design
function-calling-oriented architecture
deterministic guards
human confirmation
action cards
audit logs

That is why Gemma remained the WearEdge baseline, while Qwen2.5-VL became the serious A/B challenger for OCR-heavy branches.

Lesson Learned

Edge AI model selection is not just a leaderboard exercise. The right question is:

Can this model run locally, understand the evidence, obey the workflow boundary, and produce an action that the system can audit?

For WearEdge Pro today, the answer is Gemma 4 E2B as the baseline, Qwen2.5-VL as the next challenger, and a clear path to keep testing without pretending every benchmark cell means the same thing.

Public artifact link: Benchmark results and public discussion: https://www.hackster.io/ryanon2008/wearedge-pro-jetson-edge-ai-agent-50ec35

WearEdge Pro: An OPEA Manufacturing Five-Agent Suite for Frontline Operators

Ryan Hsu — Thu, 28 May 2026 07:49:53 +0000

Manufacturing operators often see early warning signs before enterprise systems
do: an unusual gearbox sound, a quality defect, a label changeover mismatch, a
work-instruction question, missing PPE, or a blocked walkway. These observations
are valuable, but they often stay trapped in verbal handoffs.

WearEdge Pro packages that frontline evidence into an OPEA-aligned
Manufacturing Agent Suite. The submitted competition artifact is not an
Android-only application. It is a Docker-runnable Web/API package with a
browser demo console, five agent routes, Qdrant-backed RAG, official OPEA TEI
embedding profile, guardrails, and evaluation evidence.

The five agent routes are:

Agent Workflow Target
maintenance Predictive maintenance from M400 evidence maintenance_work_order
iqc Incoming and in-process quality checks qms_quality_event
changeover SKU setup and first-piece verification changeover_checklist
wi Released work-instruction guidance wi_reference
hazard PPE, moving-parts, and walkway observations ehs_case
The architecture follows an OPEA-style path:

M400 / API evidence
-> Gateway
-> Manufacturing Megaservice
-> route registry
-> Dataprep
-> RAG / Retriever
-> Qdrant Vector DB
-> OPEA-compatible embedding service or official OPEA TEI profile
-> LLM adapter or deterministic demo path
-> deterministic evaluator
-> guardrails
-> bounded action card
The most important design decision is route isolation. Maintenance must not
issue safety clearance. Hazard observations must not invent final root cause.
Quality must not release a lot. Changeover must not grant restart permission.
Work-instruction guidance must stay tied to released source evidence.

For OPEA evidence, the repository includes:

Docker Compose base profile with Qdrant and the Manufacturing Gateway;
OPEA-compatible /v1/embeddings profile;
official OPEA TEI profile using Hugging Face TEI, opea/embedding:latest,
TEI_EMBEDDING_ENDPOINT, and OPEA_TEI_EMBEDDING;
OpenAI/OPEA-compatible LLM adapter boundary;
GenAIEval-compatible route evaluation package;
upstream OPEA RFC, comments, and a CI-green GenAIExamples PR.
The evaluation package includes 15 cases across the five routes and verifies:

action-card contract;
integration target correctness;
channel correctness;
risk-level correctness;
human gate correctness;
guardrail pass;
RAG source match;
route isolation.
The hardware evidence was captured on Google Cloud C3 c3-standard-4: a
single-node, 4-vCPU, 16-GiB-RAM, no-GPU Intel Xeon host exposing AVX-512 and AMX
flags. On that class of host, WearEdge validated the deterministic five-agent
route benchmark, Docker/Qdrant E2E, OPEA-compatible embedding profile E2E, and
official OPEA TEI profile E2E.

The public repository is here:

https://github.com/davidmillerak2026-sys/wearedge-opea-manufacturing
The upstream OPEA PR is here:

https://github.com/opea-project/GenAIExamples/pull/2462
WearEdge is still a prototype, not a certified safety or release controller.
The important point is the platform pattern: one OPEA-aligned manufacturing
suite can convert frontline evidence into bounded, auditable action cards
across maintenance, quality, changeover, work instructions, and safety.