DEV Community

Cover image for Jan open source ChatGPT alternative: offline local AI
Span Observer
Span Observer

Posted on Originally published at span.observer

Jan open source ChatGPT alternative: offline local AI

#llamacpp #llm #localai #mcp

Jan open source ChatGPT alternative: the janhq/jan repository calls itself an « Open-source ChatGPT replacement », a desktop app that downloads and runs language models on your own machine « with full control and privacy ». It is released under the Apache 2.0 licence by Menlo Research. The README lists Windows, macOS and Linux, and the documentation speaks of full offline support for the desktop app.

For an SRE or a developer, the appeal goes beyond chat. Jan bundles an OpenAI-compatible server on localhost:1337, an MCP host and a command-line tool. That makes it a local inference endpoint for your scripts and agents. What does the repository actually say, how do you install it, and what can you measure on a tool that runs on a workstation?

A note on honesty: span.observer has not installed or run Jan. The commands and settings come from the README, from the official documentation on jan.ai and from the repository’s docs/ files, read on 31 October 2026. The screenshots below reproduce their text. We could not read the star count from the source that day, so we do not quote it.

Jan open source ChatGPT alternative: what the repository says

  • Licence: the repository’s LICENSE file is the Apache License 2.0, with the line « Copyright 2025 Menlo Research ». The README confirms « Apache 2.0 ».
  • Version: the main branch carries version 0.8.8 in src-tauri/tauri.conf.json. The repository’s changelog dates 0.8.8 to 28 October 2026. It is a security release: chat links open in your browser, and the local server checks the origin of browser requests.
  • Engine: the README thanks llama.cpp, Tauri and Scalar. The documentation says the llama.cpp engine is built into the app and updates with it. On Apple Silicon Macs, MLX models are supported too.
  • Listed features: local models from Hugging Face (Llama, Gemma, Qwen, GPT-oss), connections to cloud models (OpenAI, Anthropic, Mistral, Groq, MiniMax), custom assistants, an OpenAI-compatible API and MCP integration.

One more point: « offline » applies to local models. The documentation states that requests routed to a remote provider are not offline and do not stay on your computer.

A tour of the repository and its documentation

The first screenshot reproduces, word for word, the README’s « Features » section and its licence. It shows the scope of the product in six lines.

Excerpt from the janhq/jan README: the six listed features (local models, cloud integration, assistants, OpenAI-compatible API on localhost:1337, MCP, privacy) followed by the Apache 2.0 licence

Source : github.com

The documentation then describes the local server. The defaults are host 127.0.0.1, port 1337 and prefix /v1. An optional API key goes in the Authorization: Bearer header. The second screenshot gathers the sample request and these settings.

Jan documentation: a curl request to http://127.0.0.1:1337/v1/chat/completions with an Authorization Bearer header, then the list of local server settings (host, port, prefix, API key, trusted hosts, CORS, verbose logs)

Source : jan.ai

Step-by-step installation

Step 1: download the app

The README offers one installer per platform (jan.exe for Windows 10 and later, jan.dmg for macOS, jan.deb and jan.AppImage for Linux) from jan.ai or the GitHub Releases page. Windows is also available through the Microsoft Store.

On Linux, the documentation recommends Flatpak:

flatpak install flathub ai.jan.Jan
flatpak run ai.jan.Jan
Enter fullscreen mode Exit fullscreen mode

For a .deb package, the documentation gives sudo apt-get install ./jan-linux-amd64-{version}.deb, where {version} stands for the number you downloaded. For the AppImage, run chmod +x on the file, then execute it directly.

Step 2: download a model

Open the Hub in the sidebar, search for a model, click Download, then pick it in a conversation’s model selector. For macOS, the README gives a memory guide: 8 GB of RAM for 3-billion-parameter models, 16 GB for 7 billion, 32 GB for 13 billion.

Step 3: start the OpenAI-compatible server

  1. Go to Settings > Local API Server.
  2. Choose the Default Model, a llama.cpp or MLX model you have already downloaded.
  3. Click Start Server, then wait until the status shows the server is running.
  4. Test it with the curl request in the screenshot above, replacing YOUR_MODEL_ID.

According to the documentation, you read the list of available models with GET /v1/models. If no key is configured, omit the Authorization header.

Step 4 (optional): the command line

The documentation says the jan CLI is installed automatically the first time you launch the app. It serves a model without the interface, with jan serve, or starts an agent wired to a local model, with jan launch. Note that the CLI exposes the model on localhost:6767/v1, not on the app server’s port 1337. The third screenshot shows the CLI help and the layout of the data folder.

Jan documentation: help output of the jan command (serve, launch, threads, models, examples jan serve and jan models list), then the data folder layout with logs/app.log, llamacpp/models and threads

Source : jan.ai

Limits and open questions

  • Hardware: the documentation says the bundled llama.cpp engine does not load models on Intel Macs since 0.8.5; MLX also requires Apple Silicon.
  • Network exposure: choosing host 0.0.0.0 makes the server reachable from the local network. The documentation then recommends an API key and advises against leaving it empty. Trusted Hosts add a further layer of control.
  • Tool execution: the « Execute Tools on Server » option, disabled by default, runs MCP tool calls received by the API on the server side. Leave it off unless the client needs it.
  • Models and MCP: the documentation warns that not every model handles tool calls well. MCP-specific risks are covered in our guide to securing an MCP server.
  • Telemetry: the privacy page describes usage statistics subject to your choice at first launch, changeable later in settings, sent to PostHog (EU hosting). It states that conversation content is not tracked. Read it again before an enterprise rollout.
  • Pace of change: three releases in three weeks (0.8.6, 0.8.7, 0.8.8, going by the changelog dates). Test each update before rolling it out widely.

Monitoring Jan on a workstation or test server

Jan is a desktop app, not a service designed for Kubernetes. The project documents neither Prometheus metrics nor OpenTelemetry export for the app, so we assume none. What the documentation does provide:

  • Logs: logs/app.log in the data folder, with the [APP], [SERVER] and [SPECS] categories. « Verbose Server Logs » are enabled by default and detail the requests the local server receives.
  • Availability: a synthetic check on GET /v1/models confirms the server answers and the expected model is present. A 401 means a missing or wrong key, a 404 an unknown model ID, and « Connection Refused » a stopped server or wrong port. The documentation lists these three cases.
  • Resources: memory and CPU or GPU use of the process can be measured with host tools. That is where you see a model too large for the machine. The « Request Timeout » setting bounds the wait for a model response.

For client-side calls, instrument the script that queries the endpoint: one span per request, with the model name and the duration. The approach is detailed in monitoring a local LLM with Ollama, and it applies to any local OpenAI-compatible server. To choose the model itself, see the operational criteria for an LLM.

And Dynatrace?

Jan documents no Dynatrace integration, and we do not invent one. OpenTelemetry instrumentation of the client that calls the local server remains possible with any OTLP-compatible backend, Dynatrace SaaS included; check the setup in the Dynatrace documentation.

Best practices

  • Keep the host at 127.0.0.1 while no other machine needs the API, and set an API key as soon as you go beyond that.
  • Leave « Execute Tools on Server » off unless you have a specific need, and limit connected MCP servers. See also guardrails for AI agents in production.
  • Keep Jan up to date: version 0.8.8 fixes issues reported through GitHub Security Advisories.
  • Watch the app.log file after every model or version change.
  • If you need an engine for production, compare it with vLLM and its metrics.

Frequently asked questions

Does Jan really work offline?

Yes for local models, once downloaded: the quickstart describes local models as « private, offline, no API key ». Cloud models and requests routed to a remote provider need a connection.

Can Jan replace the OpenAI API for my scripts?

The documentation presents it as a drop-in replacement for private applications. Point your client at http://127.0.0.1:1337/v1. span.observer has not tested the exact compatibility of each parameter, so check the ones your code depends on.

Which licence, and can I use it at work?

The code is under Apache 2.0, a permissive licence. Each downloaded model has its own licence: read it before any professional use.

Conclusion

Jan is a simple way into local AI. Its OpenAI-compatible server and CLI make it useful for trials, agent prototypes and environments without a cloud. Monitoring remains yours to build around it: logs, synthetic checks and host measurements. The project moves fast, so reread the documentation before each rollout.

span.observer is an independent community blog, not affiliated with Jan, Menlo Research or Dynatrace LLC. Trademarks belong to their owners. Tools change quickly: check commands and options in their official documentation.

Originally published on span.observer. French version: https://span.observer/fr/jan-alternative-chatgpt-open-source/.

Top comments (0)