ExLlamaSharp v1.4.0-beta: what shipped

ExLlamaSharp v1.4.0-beta is out.

Local LLM server for Windows with NVIDIA GPUs — OpenAI-compatible /v1, Blazor admin, and EXL3 inference.

Release notes

## ExLlamaSharp 1.4.0-beta

Multi-GPU (highlight)

• Real pipeline and tensor parallelism via ExLlamaV3 worker (Settings → Multi-GPU)
• Configurable GpuSplitGb and GpuMemoryUtilization
• CUDA device remap so cuda:0 is the highest-VRAM card in the visible set
• Admin UI shows combined VRAM across visible GPUs (Dashboard / Models / Setup fit badges)
• Vision tower is skipped under multi-GPU (text-only load); use single-GPU mode for VLM
• On Windows, prefer pipeline over tensor for JSONL worker stability

API / reliability

• Oversized prompts now fail fast with prompt_too_long (no more silent hang → client 408)
• Worker submit errors are routed onto the stream channel correctly

Other

• Smoke helper tools/exl3_worker/smoke_multigpu.py
• Docs: troubleshooting for TP timeout and VLM + multi-GPU

Install

Download ExLlamaSharp-Setup-win-x64.exe below (Windows x64). Default UI: http://127.0.0.1:14563

Full notes: https://github.com/Kortexio/ExLlamaSharp/releases/tag/v1.4.0-beta

Repo: https://github.com/Kortexio/ExLlamaSharp