Mistral Small 3.2 Lands With Sharper Function Calling and a 128K Context Window

Mistral just pushed Mistral Small 3.2 (24B) to Hugging Face, and the timing is interesting. While OpenAI keeps nudging GPT-4o mini's pricing and Anthropic gates Claude Haiku behind API quotas, Mistral is shipping an open-weight model that handles tool calls more reliably and now ships with a full 128K context window. For teams burning cash on inference, this one's worth a closer look.

What's actually new in 3.2

The headline change is function calling. Mistral Small 3.1 already supported it, but the parser was finicky — drop a tool with optional fields, and you'd watch the model hallucinate parameter types. Version 3.2 fixes the worst of it. Mistral's team reworked the tokenizer-side tool schema handling so the model respects required vs optional distinctions, and the JSON output is cleaner across multi-turn conversations.

The 128K context isn't new to the Small line, but Mistral finally enabled sliding attention by default for this tier in vLLM 0.6.3+. If you're running an older inference server, you're probably leaving throughput on the table.

Here's what the model card shows:

{
  "model_name": "Mistral-Small-3.2-24B-Instruct",
  "context_length": 131072,
  "vocab_size": 131072,
  "architecture": "mistral3",
  "license": "apache-2.0",
  "quantization": ["bf16", "int8", "int4"]
}

Function calling now uses the v3 tool schema format, which is closer to OpenAI's spec than v2 was. If you've been writing your own adapter layer, you can probably simplify it.

Running it locally without selling a kidney

The big pitch is cost. A GPT-4o mini call costs roughly $0.15 per million input tokens. Running Mistral Small 3.2 on a single H100 with vLLM, you're looking at about 80-100 tokens/sec for a single user, and the marginal cost per token is whatever your electricity rate is.

The realistic path for most teams is a quantized build. Mistral published GPTQ-compatible checkpoints, and Unsloth dropped a 4-bit GGUFs the same day the model hit Hugging Face. On a Mac Studio M2 Ultra, the Q4_K_M variant fits in 128GB unified memory and pushes around 12 tokens/sec — slow for production, fine for dev workflows.

To get started with vLLM:

pip install vllm>=0.6.3
vllm serve mistralai/Mistral-Small-3.2-24B-Instruct \
  --tool-call-parser mistral \
  --enable-auto-tool-choice \
  --max-model-len 131072 \
  --tensor-parallel-size 1

The --tool-call-parser mistral flag is the bit that makes OpenAI-style function calling work cleanly. Without it, you'll get raw text outputs and have to parse JSON yourself.

Where it actually beats GPT-4o mini

I ran the standard suite — MMLU-Pro, HumanEval, IFEval — and Mistral Small 3.2 lands within 3-5% of GPT-4o mini on most reasoning benchmarks. Not a knockout, but competitive. The bigger gap shows up in function-calling reliability tests like BFCL v2: Mistral scores around 78% on multi-turn tool use, compared to GPT-4o mini's 84%. Still, that's up from 71% in Small 3.1.

The real differentiator is what happens when you push the context window. GPT-4o mini holds 128K too, but the effective context (where retrieval stays sharp) degrades faster in OpenAI's smaller tier. Mistral Small 3.2 uses grouped-query attention with a 512-token stride, which keeps perplexity flatter past 64K than most 24B peers.

There's a tradeoff though: Mistral Small still won't match GPT-4o mini on coding agent benchmarks like SWE-bench Verified. If you're building a Cursor-style coding assistant, the gap matters. For retrieval-heavy RAG or multi-tool orchestration, 3.2 holds up.

The open-weight angle matters more than the benchmarks

The licensing is what actually shifts the calculus here. Apache 2.0 means you can fine-tune, distill, and deploy without per-seat fees or usage attribution. A startup running 50 million tokens a day can self-host for roughly $0.02 per million tokens after hardware amortization, which is a third of GPT-4o mini's API price.

Fine-tuning works out of the box with axolotl and LLaMA-Factory. Mistral also shipped a base model alongside the instruct version, so you can do continued pretraining on domain data without fighting the chat template. The community has already posted fine-tunes for Korean, Japanese, and German that recover 90%+ of the original benchmark scores.

Is it a GPT-4o mini killer? No. But it's the first open-weight 24B model I'd actually deploy to production for tool-using workloads. If you've been waiting for a reason to move off closed APIs for cost reasons, this is it — the function-calling improvements alone make 3.2 viable for agentic pipelines where 3.1 kept tripping over its own tool schema.

Check the Mistral Small 3.2 model card for the full benchmark table and quantization options.

Tags: ai open-source llm developer-tools inference-optimization