It sees, speaks and makes decisions. All in a 4B model?

DRVEXO_LLM — Tue, 10 Nov 2026 04:16:21 +0000

A picture, a spoken question and a task can all be part of an interactive application. DRVEXO-4B Unified brings image understanding, speech input, text conversation, speech generation, tool-call candidates and explicit candidate scoring into a complete, independently loadable package.

It uses a 4B language backbone, with additional vision, speech, codec and candidate-scoring components.

Download: Hugging Face · ModelScope

Try it: Hugging Face Space · ModelScope Studio

Evaluate each capability on its own terms

The reported tasks are IFEval (541 strict prompt-level examples), MATH-500 (500 examples), a stratified 3,000-question MMLU-Pro subset, MMMU validation (900), ChartQA validation (1,920) and a 231-example public JevBench subset.

The MMLU-Pro subset score is 2,338/3,000, or 77.93%, using a long-thinking setting. It is a subset result, rather than a full MMLU-Pro score. See the model card for the evaluation scope and results.

Measure text and speech separately

On November 7, 2026, the complete service on an NVIDIA DGX Spark / GB10 delivered the following aggregate text output rates:

Text concurrency	Aggregate output
1	24.02 tokens/s
8	185.17 tokens/s
32	601.55 tokens/s
128	1,378.94 tokens/s

These are batch text measurements with a fixed short-input/output budget and thinking disabled. Separate speech streaming tests reported 307–324 ms to the first audio chunk, 2.45–2.58× real-time generation and 24 kHz output. These results do not establish 128 simultaneous voice sessions or physical playback latency.

See: describe the visible scene

The prompt asks the model to describe clothing, equipment, pose and visible background objects without guessing a name, date or mission. The recorded Chinese answer describes the orange spacesuit, the person facing the camera, a US flag and a spacecraft model in the background.

This is a concrete image-question example. Applications can send a picture and a question together, and include the full submitted conversation history for follow-up questions. Image source: NASA, public domain, distributed by scikit-image.

Speak: speech input, streaming output and reference voices

DRVEXO supports Chinese and English speech input and transcription, text answers and generated speech. Incremental text and audio chunks let an application start presentation while more output arrives.

Greeting sample · approximately 2.6 seconds

Play or download the greeting

West Lake introduction · approximately 19.6 seconds

Play or download the longer sample

These two files are actual default-voice examples from a historical reference baseline. For the current service and custom voices, use the live demo with an authorized reference recording. The model card recommends 1–30 seconds of mono speech; no separate training per speaker is required.

Use tools: one question, three read-only calls

The example asks for the status of demonstration order EX-20261107-01 and the available EXO-17 inventory in Hangzhou and Shanghai, with no changes to the order or stock.

The model proposes three tool calls: order status and one inventory query per warehouse. After receiving application results, it reports the order as shipped, with 18 available units in Hangzhou and 7 in Shanghai.

This replay uses synthetic local business data. It does not connect to a real order system or modify an order. The animation timing is presentation timing, rather than measured inference latency.

Structure fields with JSON Schema

A separate example asks for two EXO-17 units and one EXO-23 unit for a customer in Hangzhou, with a 14:00–16:00 delivery window. With an object-and-array schema supplied in advance, the model outputs nested customer, items and delivery fields.

The example output passes its schema validation. The requested time window is preserved; the relative date “tomorrow” is not resolved to a calendar date. An application still needs its own validation and submission workflow.

Decide: score explicit alternatives

The customer is available for a call after 18:00, wants an email summary now, and has not authorized an order or address change.

Among four supplied alternatives, the model selects the plan to send a summary now and call after 18:00 without modifying the order, with a candidate probability of 99.42% in this example.

The response has executable: false. No email is sent and no call or device action occurs. Candidate probability is not an execution-success rate; the application handles authorization and execution.

Try the application, then load the model

The demo exposes text and image conversation, microphone and camera inputs, streaming speech, reference voices, candidate scoring and editable system prompts. It shares a self-hosted service and queues requests when busy.

The current API accepts at most one image or one audio input per request. A camera input and image understanding do not establish continuous video understanding or real-time video calling.

Both model repositories provide the complete package, model code, configuration, tokenizer, documentation and examples. Follow the model card for the local Python entry point and the pinned DRVEXO vLLM-Omni source revision for serving.

from pathlib import Path
from drvexo_unified import UnifiedModel

model = UnifiedModel(Path("./drvexo-4b-unified"), device="cuda")
print(model.chat("Hello. Please introduce yourself.", max_new_tokens=256)["text"])

Serve the model and connect an application

The model card documents a DRVEXO-adapted vLLM-Omni source revision. Use that pinned source and its environment requirements rather than assuming a standard release includes this integration.

Endpoint	Purpose
`POST /v1/chat/completions`	Text, image/audio input, JSON Schema and tool calls
`POST /v1/audio/speech`	Reference-voice synthesis
`POST /v1/drvexo/speech`	Default-voice WAV synthesis
`POST /v1/drvexo/speech/stream`	Chunked speech for supplied text
`POST /v1/drvexo/chat/stream`	Incremental text and sentence-level speech
`POST /v1/drvexo/decision`	Explicit instruction, state and candidate scoring

Standard text chat uses SSE. DRVEXO speech streams use NDJSON events for text, audio, completion and errors. Each audio chunk is an independent base64-encoded WAV; a text-complete event does not mean all speech has finished.

The application handles microphone/VAD, playback, tool execution and device control. For embodied applications such as exoskeleton interaction, the model package is a starting point for integration; product behavior must be validated on the intended hardware and in the intended scenario.

Get the model: Hugging Face · ModelScope

Open the demo: Hugging Face Space · ModelScope Studio

The repositories list Apache-2.0; see the license files for the applicable terms. Reproducible feedback with inputs, expected behavior and actual output is welcome.

DEV Community: DRVEXO_LLM