Dify with Claude and GPT: every field of the OpenAI-API-compatible plugin, explained

Disclosure: I run APIVAI, the gateway used in the examples below. The setup is the same for any endpoint that speaks the OpenAI or Anthropic format.

Dify is an open-source platform for building LLM apps: chatbots, agents, workflows and knowledge-base (RAG) apps. It reaches models through model provider plugins, and with a third-party gateway there are enough fields that it's easy to fill one in wrong. This post goes through the actual fields of Dify's official openai_api_compatible plugin (version 0.0.68) and the anthropic plugin.

Which provider plugin?

Plugin	Address field	Models	Best for
OpenAI-API-compatible	API Base URL: https://api.apivai.com/v1	Claude and GPT	One provider for every model; you type the model IDs
Anthropic	API URL: https://api.apivai.com	Claude only	Claude only, in the native format with prompt caching

Start with OpenAI-API-compatible: you type each model ID yourself, so a new model works as soon as the gateway lists it.

Providers live on the Model Provider page: under Integrations in current Dify Cloud, under Settings in the account menu in older self-hosted versions. Each provider applies to the whole workspace, and only owners and admins can change it.

Step 1: Install the plugin

Open Model Provider, find OpenAI-API-compatible in the list of providers you can install (or on the Dify Marketplace) and click Install.

Step 2: Add a model

On the OpenAI-API-compatible card, click Add Model and fill in:

Field	Value
Model Type	LLM
Model Name	claude-sonnet-4-6 (the exact model ID)
Model display name	optional, e.g. "Claude Sonnet 4.6"
API Key	your key
API Base URL	https://api.apivai.com/v1
model name for API endpoint	leave empty
Completion mode	Chat
Model context size	no larger than the model's context window
Upper bound for max tokens	8192 or more
Function Call Type	Tool Call
Stream function calling	Support
API Type	Chat Completions API (/chat/completions)

Leave the rest at their defaults. Older versions of the plugin call the address field API endpoint URL; the value is the same.

The fields people most often misread:

• Model Name is what Dify actually sends to the API, so it must be the exact ID. The display name is only for the interface.
• Model context size is the limit Dify uses when it fits history and retrieved knowledge into the prompt. A value below the model's real window is fine and keeps long chats cheaper.
• Upper bound for max tokens only sets how high Max Tokens can go in your apps. Thinking is on by default and counts as output, so leave room above 4096.
• Function Call Type: Tool Call is what lets Agent apps use native tool calling.

When you click Save, Dify sends a test request and only saves if it passes. Repeat Add Model for each model you want, with the same key and address, for example gpt-5.5 and claude-haiku-4-5.

Take model IDs from the API:

curl https://api.apivai.com/v1/models -H "Authorization: Bearer your-key"

Step 3: Use the models

• In an app's model selector, pick your model under OpenAI-API-compatible and set Max Tokens to 4096 or more.
• In a workflow, each LLM node picks its own model, so cheap and strong models can sit in the same flow.
• Under Default Models on the Model Provider page, set the System Reasoning Model as the workspace default.

Alternative: the Anthropic plugin for Claude

1. Install the Anthropic plugin and open its setup.
2. Set API Key to your key and API URL to https://api.apivai.com. Do not add /v1; the plugin's Anthropic client appends /v1/messages itself. Leave Enable request metadata on Disabled.
3. Some IDs in the plugin's built-in model list match the gateway's (such as claude-sonnet-4-6); others have different or dated names. If a model is missing or returns "model not found", click Add Model on the Anthropic card and type the exact ID.

Which model for which job

• Question Classifier nodes, extraction, high-volume Q&A: claude-haiku-4-5 or gpt-6-luna
• Default for chatbots and agents: claude-sonnet-4-6
• Agents with many tools, long documents: claude-sonnet-5-5
• Only the hardest reasoning nodes: claude-opus-5-5

A routing workflow (a cheap model classifies first, then hands the hard cases to a strong one) is the most common way to keep Dify costs down.

Common errors

• Credentials validation fails on save / 401: the key is incomplete or deleted. Also check that API Base URL has /v1 exactly once.
• 402 / insufficient_quota: the key's budget is used up, or what's left is too little for one request to an expensive model (Opus, Sonnet 4.6 hit this first). Add budget to the key or move that node to a cheaper model. A key stops working with a little still on it; that remainder isn't lost, and deleting the key returns it to your account balance.
• 404: OpenAI-API-compatible needs https://api.apivai.com/v1, without /chat/completions; the Anthropic plugin needs https://api.apivai.com without /v1.
• Model not found: Model Name holds a display name instead of the ID, or you used a dated name from the Anthropic plugin's built-in list.
• Empty or cut-off answers: Max Tokens in the app is too low (thinking uses output tokens); raise it to 4096 or more, and make sure Upper bound for max tokens is high enough.
• The agent never calls tools: Function Call Type isn't set to Tool Call.
• Knowledge base errors: a knowledge base needs an embedding model, and APIVAI doesn't provide embeddings. Set up a separate embedding provider for it; the LLM that writes the answer can still be Claude or GPT.
• 429: each key allows 60 requests per minute and 5 at once (extra ones wait in a queue for a while). For busy apps, give each app its own key.

What it costs

At the time of writing, most Claude models on APIVAI are 56–62% below Anthropic's list prices and GPT models up to 86% below OpenAI's; for example, Claude Sonnet 4.6 is $1.26 / $6.32 per million input / output tokens against the official $3 / $15. It's prepaid and billed per token, with no subscription. The full table is at https://apivai.com/pricing.

Questions are welcome in the comments.