Count tokens, chunk for RAG and repair LLM JSON with one HTTP call (no tiktoken install)

#llm #ai #api #rag

Three chores show up in every LLM app: counting tokens before you hit the context limit, splitting documents into chunks for embeddings, and fixing the "almost JSON" a model sends back. LLMKit handles all three over HTTP, so it works the same from Python, Node, n8n, Zapier or a Cloudflare Worker.

Count tokens

curl -G "https://llmkit-token-counter-rag-chunker-json-repair.p.rapidapi.com/tokens" \
  --data-urlencode "text=Hello world, this is a token counter." \
  -d encoding=o200k_base \
  -H "X-RapidAPI-Key: YOUR_RAPIDAPI_KEY" \
  -H "X-RapidAPI-Host: llmkit-token-counter-rag-chunker-json-repair.p.rapidapi.com"

{"tokens": 9, "words": 7, "characters": 37, "encoding": "o200k_base", "models": "GPT-4o, GPT-4.1, o-series"}

Chunk a document for RAG

POST /chunk with your text and chunk_size, overlap and mode (tokens, sentences or paragraphs). Each chunk comes back with its own token count, ready to embed.

Repair broken JSON from a model

POST /extract-json pulls the JSON out of a chatty reply (code fences, trailing commas, single quotes, missing brackets) and returns it parsed:

{"json": {"name": "Jane", "age": 30, "tags": ["a", "b"]}, "repaired": true, "valid": true}

Also included

/truncate: cut a prompt to fit a token budget, keeping the start or the end.
/clean: strip zero-width characters and collapse whitespace.
/to-markdown: turn any URL or HTML into clean Markdown for an LLM.

Use it as an MCP server

Point Claude Desktop, Cursor or any MCP client at https://llmkit-token-counter-rag-chunker-json-repair.p.rapidapi.com/mcp, sending your X-RapidAPI-Key and X-RapidAPI-Host headers, and every endpoint above becomes a tool your agent can call.

There's a free plan to try it. LLMKit on RapidAPI

DEV Community