DEV Community

Cover image for How to Use GLM-5 for Free with Ollama?
Preecha
Preecha

Posted on

How to Use GLM-5 for Free with Ollama?

#llm #ai #opensource

GLM-5 from Z.ai is a frontier-level open-source model you can now run through Ollama — complex reasoning, software engineering, and long-horizon agentic workflows, entirely on your own hardware.

Try Apidog today

💡 Download Apidog for free to complement your setup. It lets you visually design, test, and debug requests against Ollama's local OpenAI-compatible endpoint, streamlining experimentation with GLM-5 from your first interaction.

What Makes GLM-5 Stand Out

The essentials you need to know:

  • License: MIT. Weights are freely available on Hugging Face and ModelScope.
  • Architecture: Mixture-of-Experts (MoE) — 744B total parameters, only 40B active per token. High intelligence at controlled inference cost.
  • Pre-training: 28.5 trillion tokens; strong multilingual support, excelling in English and Chinese.
  • Context: up to ~198K tokens in the Ollama implementation via DeepSeek Sparse Attention (DSA), which reduces computational overhead without sacrificing long-sequence performance.

Image

Benchmarks back it up: 92.7% on AIME 2026 I, 86.0% on GPQA-Diamond, and 77.8% on SWE-bench Verified — competitive in coding, mathematical reasoning, and agentic tasks such as multi-step planning and tool use.

Image

It also generates structured documents (PRDs, spreadsheets, reports), works with agent frameworks, and transitions smoothly from simple chat to sophisticated engineering workflows.

Why Pair GLM-5 with Ollama

Ollama manages model downloads, quantization, and serving on macOS, Linux, and Windows, and exposes an OpenAI-compatible REST API at http://localhost:11434/v1. Anything built for OpenAI endpoints works with GLM-5 out of the box.

You avoid cloud costs, rate limits, and sending data to third parties. Model switching is trivial, and the glm-5:cloud tag gives you an optimized variant tailored for local execution that balances capability and resource demands.

Prerequisites

Component Requirement
OS macOS (Apple Silicon preferred), Linux, or Windows with WSL2
GPU NVIDIA 24 GB+ VRAM for comfortable performance at higher context lengths; Apple Silicon with 32 GB+ unified memory also performs well
RAM 32 GB minimum; 64 GB+ improves stability during long contexts
Storage 50 GB+ free SSD for model files and the Ollama runtime
Internet Required for the initial ollama pull

CPU-only setups work but yield slower token generation. Mid-range GPUs often reach usable speeds by limiting context or using lower quantization where available — test incrementally after setup.

Step 1: Install Ollama

Download the installer for your platform from the official Ollama website. On macOS or Linux, run the installation command from the site in a terminal; on Windows, execute the downloaded .exe.

Verify the installation:

ollama --version
Enter fullscreen mode Exit fullscreen mode

If the server doesn't launch automatically, start it in the background:

ollama serve
Enter fullscreen mode Exit fullscreen mode

Step 2: Pull and Run GLM-5

Download the model:

ollama pull glm-5:cloud
Enter fullscreen mode Exit fullscreen mode

Monitor progress in the terminal — download time depends on your connection. Then start an interactive session:

ollama run glm-5:cloud
Enter fullscreen mode Exit fullscreen mode

Type prompts, observe responses, and exit with /bye when finished.

Step 3: Call the API from Your Code

The CLI suits quick testing; for programmatic access, use the REST API. A minimal chat completion with curl:

curl http://localhost:11434/api/chat -d '{
  "model": "glm-5:cloud",
  "messages": [
    { "role": "user", "content": "Explain the advantages of Mixture-of-Experts architectures in large language models." }
  ],
  "stream": false
}'
Enter fullscreen mode Exit fullscreen mode

Ollama returns a JSON response containing the assistant's message. Set "stream": true for real-time token output.

Since the endpoint is OpenAI-compatible, existing OpenAI SDK code adapts effortlessly — just point base_url at Ollama:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    [REDACTED CREDENTIAL]  # Placeholder; no real key required
)

response = client.chat.completions.create(
    model="glm-5:cloud",
    messages=[
        {"role": "system", "content": "You are an expert software architect."},
        {"role": "user", "content": "Design a scalable microservices system for an e-commerce platform handling 1M daily users."}
    ],
    temperature=0.7,
    max_tokens=2048
)

print(response.choices[0].message.content)
Enter fullscreen mode Exit fullscreen mode

The official ollama Python library works as well.

Step 4: Test Visually with Apidog

Visual API testing accelerates development and debugging — Apidog provides an intuitive interface for crafting requests, managing environments, and generating client code.

Image

Download Apidog for free from the official site, create a new project, and configure:

  • Base URL: http://localhost:11434/v1
  • Endpoint: POST /chat/completions
  • Headers: Content-Type: application/json (no Authorization header needed for local Ollama)

Build the request body visually: define the messages array, set the model to "glm-5:cloud", and adjust temperature, top_p, or max_tokens. Send the request and inspect the full JSON response, including token usage and timing.

Apidog also lets you:

  • Save reusable environments for different models or contexts.
  • Generate SDK code in Python, JavaScript, or other languages.
  • Create automated test suites validating GLM-5 outputs against expected schemas.
  • Mock responses for frontend development while the backend runs locally.

This turns raw API experimentation into a structured, collaborative process — particularly valuable when testing complex multi-turn conversations or tool-calling scenarios with visual debugging tools.

Advanced Configurations and Optimizations

Customize behavior with a Modelfile:

FROM glm-5:cloud
SYSTEM You are a precise engineering assistant focused on long-term planning and code quality.
PARAMETER temperature 0.6
PARAMETER num_ctx 131072
Enter fullscreen mode Exit fullscreen mode

Build and run the custom model:

ollama create my-glm5 -f Modelfile
ollama run my-glm5
Enter fullscreen mode Exit fullscreen mode

Adjust context length carefully: larger windows consume more memory but enable analysis of extensive codebases or documents. Monitor VRAM usage with nvidia-smi.

For agentic workflows, launch compatible tools directly:

ollama launch openclaw --model glm-5:cloud
Enter fullscreen mode Exit fullscreen mode

Similar commands support Claude Code, Codex, and other frameworks, letting GLM-5 power desktop agents or coding assistants locally.

Image

Experiment with system prompts to steer the model toward specific domains such as frontend architecture or cybersecurity analysis. Track tokens-per-second — it typically improves with GPU acceleration and optimized context management.

Troubleshooting Common Issues

  • ollama pull fails — verify your internet connection and disk space, restart the Ollama service, and retry.
  • Memory errors during inference — insufficient VRAM or an overly ambitious context size. Reduce num_ctx or close other GPU-intensive applications. On Apple Silicon, ensure sufficient unified memory allocation.
  • Slow responses — confirm GPU offloading; check Ollama logs to verify layers load onto the accelerator.
  • Unexpected API response formats — confirm the model tag matches exactly and the request body follows the expected schema. Apidog isolates these issues quickly by displaying raw requests and responses side by side.

Community forums and official documentation provide additional solutions as the ecosystem evolves.

If you need to run GLM-5 without Ollama or with a different inference backend, running GLM-5 locally for free via alternative setups covers the options that give you full control over the serving layer.

If you already have GLM-5 running locally via Ollama and want to evaluate the updated release without spinning up new infrastructure, GLM-5.2's free access tiers and self-hosted options cover both the API quota route and the weights you can pull directly.

Conclusion: Take Control of Advanced AI Today

Running GLM-5 locally through Ollama removes barriers to high-quality AI assistance — state-of-the-art reasoning and coding performance with complete data sovereignty and zero usage costs.

Start with the installation steps above, integrate Apidog to refine your API interactions, and explore custom configurations that match your workflows. Small adjustments — optimized prompts, context management, tool integrations — frequently yield substantial improvements in output quality and efficiency.

GLM-5's capabilities plus Ollama's simplicity let you experiment freely and build production-grade solutions entirely on your own infrastructure. Begin your local deployment now.

Top comments (0)