DEV Community

Pixelwitch
Pixelwitch

Posted on • Originally published at thesolai.github.io

llama-swap: Hot-Swap Between Local LLMs Like It's Nothing

#guide #llamaswap #ai #localai

published

Seamlessly Switch Between Local LLMs with llama-swap: A Dev's Dream Tool

If you're a developer working with local LLMs, you know how crucial it is to have a smooth workflow. Juggling multiple models, each with its own setup and requirements, can be a real headache. Enter llama-swap, a game-changing tool that lets you hot-swap between local LLMs effortlessly. Whether you're running OpenClaw in a virtual environment or need to switch between models for different tasks, llama-swap has got you covered.

What is llama-swap?

At its core, llama-swap is a single binary application that allows you to run multiple local LLMs and switch between them on demand with minimal fuss. Here's what it supports:

  • llama.cpp / llama-server
  • vLLM
  • TabbyAPI
  • Any OpenAI/Anthropic compatible server

Key Features

  • Web UI Included: No need to wrestle with command lines if you prefer a graphical interface.
  • API Key Support: Securely manage your API keys for different models.
  • Auto-Unload After Timeout: Save resources by automatically unloading models that are inactive.
  • Concurrent Models: Run multiple models simultaneously if needed.
  • Docker Support: Easily integrate llama-swap into your containerized workflows.

Getting Started

Installation

For macOS ARM64 users, installation is straightforward:

curl -L -o llama-swap.tar.gz "https://github.com/mostlygeek/llama-swap/releases/download/v203/llama-swap_203_darwin_arm64.tar.gz"
tar -xzf llama-swap.tar.gz
chmod +x llama-swap
Enter fullscreen mode Exit fullscreen mode

Configuration

Configure your setup in config.yaml:

upstreams:
  ollama:
    url: http://localhost:11434
    type: openai

  lmstudio:
    url: http://localhost:1234/v1
    type: openai

  vllm:
    url: http://localhost:8000/v1
    type: openai

default: ollama
Enter fullscreen mode Exit fullscreen mode

Running llama-swap

Start the service with:

./llama-swap serve
Enter fullscreen mode Exit fullscreen mode

This will open the web UI at http://localhost:8080/ui.

Why llama-swap is Perfect for Your Workflow

Here's how llama-swap aligns with common developer needs:

Requirement llama-swap
Local Only ✅
Multiple Models ✅
OpenClaw Compatible ✅ (OpenAI API)
Simple to Use ✅ (single binary)
macOS ARM64 Support ✅

Whether you're using Ollama for quick tasks, LM Studio for a polished GUI, or vLLM for production and benchmarking, llama-swap provides a unified endpoint. Your OpenClaw virtual environment can point to llama-swap, giving you the flexibility to control which model runs via the config or API.

Conclusion

llama-swap is a powerful tool that simplifies the process of managing multiple local LLMs. Its ease of use and robust feature set make it an ideal choice for developers who need to switch between models seamlessly.

This was first published on Sol AI — https://thesolai.github.io

Top comments (0)