DEV Community

Cover image for How to Use Ace Data Cloud as an OpenAI-Compatible Model Provider in AnythingLLM
Germey
Germey

Posted on Originally published at platform.acedata.cloud

How to Use Ace Data Cloud as an OpenAI-Compatible Model Provider in AnythingLLM

#ai #devtools #tutorial #programming

If your documents live in AnythingLLM but you want the conversation model to run through an OpenAI-compatible endpoint, the main thing to get right is the Base URL.

AnythingLLM is a practical tool for document Q&A because it wraps documents, users, vector storage, and agent tools into isolated Workspaces. For personal setups, it can run as a desktop app; for team setups, it can be deployed with Docker. In both cases, the useful pattern is the same: keep document indexing local, then point the chat model at a provider that speaks the OpenAI-compatible API shape.

This guide shows how to configure Ace Data Cloud as the conversational model provider in AnythingLLM while keeping the default local AnythingLLM Embedder for document vectorization.

What you are wiring together

The source workflow has two independent parts:

  1. LLM Preference: the chat/completion model used to answer questions.
  2. Embedding Preference: the model or local embedder used to vectorize uploaded documents.

For the LLM side, AnythingLLM provides OpenAI (Generic). That is the provider you want because it supports a custom Base URL. The regular OpenAI option is fixed to api.openai.com and is not the right place to enter a custom endpoint.

For the embedding side, the recommended default is AnythingLLM Embedder. It runs locally, requires no API key, and stores the original text plus vectors in the local LanceDB. The network is used only when the conversation model is called.

The important values from the Ace Data Cloud document are:

Field Value
Base URL https://api.acedata.cloud/v1
Alternative Base URL https://api.acedata.cloud/openai
Chat request path https://api.acedata.cloud/v1/chat/completions
Model list path GET https://api.acedata.cloud/v1/models
Auth header for direct API test Authorization: Bearer {token}

Configure the conversational model

Open AnythingLLM and go to:

Settings → LLM Preference
Enter fullscreen mode Exit fullscreen mode

Choose:

Provider: OpenAI (Generic)
Enter fullscreen mode Exit fullscreen mode

Then fill in:

Base URL: https://api.acedata.cloud/v1
API Key: your Ace Data Cloud token
Selected Model: use the current model ID loaded from /models, or enter one manually
Model context window: set it according to the selected model
Enter fullscreen mode Exit fullscreen mode

The /v1 suffix matters. AnythingLLM builds the final chat completions URL from the Base URL. If the Base URL is correct, the actual request becomes:

https://api.acedata.cloud/v1/chat/completions
Enter fullscreen mode Exit fullscreen mode

The documentation also notes that this Base URL works:

https://api.acedata.cloud/openai
Enter fullscreen mode Exit fullscreen mode

which maps to:

https://api.acedata.cloud/openai/chat/completions
Enter fullscreen mode Exit fullscreen mode

But these two common variants are wrong:

https://api.acedata.cloud/openai/v1
https://api.acedata.cloud
Enter fullscreen mode Exit fullscreen mode

The first points to a non-existent /v1 under /openai; the second omits /v1, causing AnythingLLM to call /chat/completions at the wrong root path.

Keep embeddings local for a simple RAG setup

Next, go to:

Settings → Embedding Preference
Enter fullscreen mode Exit fullscreen mode

Keep the provider as:

AnythingLLM Embedder
Enter fullscreen mode Exit fullscreen mode

This is the built-in local embedder. It works without adding another key, and AnythingLLM stores the vectors in local LanceDB. That separation is useful: uploaded PDF, Word, or Markdown files are split into paragraphs locally; only the retrieved excerpts plus your question are sent to the conversational model when you ask something.

If you want a custom endpoint for embeddings later, the document mentions that Generic OpenAI Embedder has a Base URL field and can point to an Ace Data Cloud embedding model such as text-embedding-3-large. But for a first reliable setup, local embeddings plus a remote conversation model is the cleanest path.

Verify the endpoint with curl

If AnythingLLM shows a connection problem, test the API directly before changing UI settings at random. Replace {token} and MODEL_ID with your real values:

curl -X POST 'https://api.acedata.cloud/v1/chat/completions' \
  -H 'Authorization: Bearer {token}' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "MODEL_ID",
    "messages": [{"role": "user", "content": "ping"}]
  }'
Enter fullscreen mode Exit fullscreen mode

A successful response returns an OpenAI-compatible chat.completion object. That tells you the token and endpoint are usable independently of AnythingLLM.

If you see HTTP 403 used_up, the token is valid but the related application balance is insufficient. If you see 401 Unauthorized, check that the API Key field contains the Ace Data Cloud token without the Bearer prefix and without extra spaces.

Create a Workspace and ask document-grounded questions

After the model configuration is working, create a new Workspace and upload documents: PDF, Word, or Markdown are typical choices.

AnythingLLM will:

  1. split documents by paragraph,
  2. create vectors with the local AnythingLLM Embedder,
  3. store the text and vectors in local LanceDB,
  4. retrieve relevant segments when you ask a question,
  5. send the retrieved context plus your question to the configured conversational model.

At the top of a Workspace, AnythingLLM lets you switch between Query mode and Chat mode. Query mode answers based only on documents, which is usually the safer choice for serious document Q&A. Chat mode can mix in general model knowledge.

A good first test is to upload a small Markdown file and ask something that can only be answered from the file. If the answer cites the uploaded source, your retrieval path and model path are both working.

Docker deployment notes

For team or multi-user setups, the AnythingLLM Docker path can use environment variables to preset the same Generic OpenAI integration. The document lists these names:

GENERIC_OPEN_AI_BASE_PATH
GENERIC_OPEN_AI_API_KEY
GENERIC_OPEN_AI_MODEL_PREF
Enter fullscreen mode Exit fullscreen mode

That is useful when you do not want every user to configure the provider manually in the UI. The same Base URL rule still applies: use https://api.acedata.cloud/v1 for the Generic OpenAI base path.

Troubleshooting checklist

When something fails, I would check in this order:

  • Is the LLM provider set to OpenAI (Generic) rather than regular OpenAI?
  • Does the Base URL end with /v1?
  • Did you paste the token without the Bearer prefix in the AnythingLLM API Key field?
  • Does the direct curl request return a chat.completion object?
  • Is AnythingLLM Embedder selected before uploading documents?
  • After changing embedding settings, did you re-upload documents so vectors are generated?

This setup is intentionally simple: AnythingLLM handles Workspaces, local document storage, and local vectors; Ace Data Cloud handles the OpenAI-compatible chat completions call. Once that boundary is clear, most integration bugs become easy to isolate.

Full source document: https://platform.acedata.cloud/documents/development_anythingllm

Top comments (0)