DEV Community: Simon

Git push to your own VPS: what Rubuz does, according to its docs (and what I'd check first)

Simon — Wed, 04 Nov 2026 01:27:54 +0000

Disclosure: Links to rubuz.com in this post are affiliate links. If you sign up through them I may earn a commission, at no extra cost to you. It was also written with the help of AI and reviewed by me. I haven't run Rubuz myself yet: everything below comes from Rubuz's public docs and product pages as of November 2026, with links, and I've flagged where those pages disagree with each other.

The trade-off it targets

If you ship side projects or small production apps, you've probably sat on one side of this fence:

A managed PaaS (Heroku-style) is easy until the bill grows with every app, database and preview environment.
A plain VPS is cheap, but then you are the deploy pipeline: a shell script that pulls and restarts, certbot cron jobs, apt upgrade on a Sunday, and a backup script you hope still runs.

Rubuz pitches itself as the middle: you keep renting the server (Hetzner, DigitalOcean, Vultr, AWS, etc.), and Rubuz acts as a control plane that connects over SSH and handles deploys, databases, SSL, backups and updates. The same idea exists in open-source form (Coolify, Dokku, CapRover, Kamal), so it's worth knowing what Rubuz specifically does and where it differs.

How it works, step by step (per the quickstart)

The quickstart is three steps:

Provision a clean VPS running Ubuntu 26.04 LTS or Debian 13 (or newer). Examples given: Hetzner CPX11 / CAX11 (ARM64), a DigitalOcean Droplet, Vultr High Frequency, AWS Lightsail or EC2.
Connect it in the dashboard: Servers → Connect Server, enter the IP, then either run a single setup command or let Rubuz connect with a temporary SSH credential. It sets up a rootless container runtime and the server turns green.
Deploy a stack: pick an app from the catalog or connect a GitHub repo, give it a domain (or a free rubuz.site preview subdomain), click Deploy Stack.

Server requirements from Connecting your first VPS:

Requirement	Value
OS	Debian 13+ or Ubuntu 26.04 LTS+, 64-bit
RAM	1 GB minimum, 2 GB+ recommended for app + database stacks
Disk	15 GB SSD/NVMe
Server state	Fresh, no cPanel / Plesk / CyberPanel
Open ports (cloud firewall)	80 (HTTP + ACME), 443 TCP/UDP (HTTPS + HTTP/3), 22 optional

The docs specifically warn against starting from a "1-click LAMP/LEMP" marketplace image; use a minimal OS.

Deploys: what happens on `git push`

According to the git-push feature page:

Runtime detection, no Dockerfile required. It looks for package.json (Node.js), pyproject.toml (Python), go.mod (Go), Cargo.toml (Rust), composer.json (PHP) or Gemfile (Ruby). You can also bring your own Dockerfile, or deploy a ready-made image from Docker Hub or GHCR.
The image is built on your own server, not on Rubuz's infrastructure. No separate CI runner.
Zero-downtime swap: the new container starts next to the old one, passes a health check, then traffic moves over and the old one stops. If the build fails, nothing is swapped.
One-click rollback to an earlier commit from the deploy history.
Persistent /data: each app gets a /data directory that survives redeploys, for uploads and generated files.

That last point is the classic VPS footgun (uploads written inside the app folder vanish on rebuild), so it's good to see it called out.

Databases and one-click apps

Per the pricing page, you can run Postgres, MySQL, MongoDB and Valkey beside your apps, each in its own container, with credentials injected as environment variables. Apps, domains and databases are not capped; the limit is your server's CPU, RAM and disk.

There's also a catalog of open-source apps; the About page lists n8n, Ghost, Metabase, Umami, Chatwoot, Vaultwarden and Open WebUI. Each gets its own container, domain, SSL and backups.

⚠️ Docs inconsistency: the Deploying Apps & Databases doc page lists MariaDB, PostgreSQL and Redis instead, and the quickstart mentions Next.js, Astro and Go as example app types rather than the six detected runtimes. The product pages look newer, but if you depend on a specific database, confirm it in the trial.

SSL and domains

From Custom Domains & Automated SSL:

Point an A record at the server's IPv4 (and a www CNAME if you want it).
In the app's Domains tab, Add Custom Domain → Verify DNS & Issue SSL.

Certificates come from Let's Encrypt via Traefik. Rubuz runs a DNS pre-flight check before requesting a certificate (to avoid burning Let's Encrypt rate limits on a misconfigured record) and renews 30 days before expiry. No certbot cron to babysit.

Security model

From Rootless Containers & Firewall Management and the pricing page:

Each app runs in a rootless Podman container under an unprivileged user namespace (UID 10000+), so root inside the container isn't root on the host.
The firewall (nftables) leaves only SSH, HTTP and HTTPS open publicly. Databases listen on loopback or the internal bridge network (10.88.0.0/16), not on the public interface.
Brute-force rate limiting on SSH and logins; team roles, 2FA and passkeys on the Rubuz account.

Rootless Podman is a meaningful difference from tools that run a privileged Docker daemon, which is the main thing Rubuz's docs lean on.

Backups

The backups page describes:

Restic snapshots, encrypted on your server before upload, of databases, app volumes (including /data) and config.
Sent to your own S3-compatible bucket: Amazon S3, Cloudflare R2, Wasabi, Backblaze B2, DigitalOcean Spaces or MinIO.
Scheduled, with one-click restore onto the same server or a fresh one (useful if the server itself dies).
Monthly restore tests: Rubuz replays database dumps into throwaway databases and alerts you if a test fails.

Automatic restore testing is the part most hand-rolled setups never get around to, so this is one of the stronger points on paper.

Lock-in: what happens if you leave

This is the question I'd ask of any control plane, and the pricing FAQ answers it directly: Rubuz says it installs no agent; it sets up standard Podman containers and system services over SSH. If you stop using it, your apps, databases and scheduled jobs keep running. If the free trial ends without a plan, the Rubuz account is locked but the apps on your servers keep running.

If that holds, the worst case of leaving is "back to managing a plain VPS by hand," not a migration.

Kubernetes, when one server isn't enough

The Managed Kubernetes doc says Rubuz can also run clusters in your own DigitalOcean, Vultr or Linode account (not AWS/GCP for clusters), with high availability, autoscaling and rolling deploys from the same console as your VPS servers. Nodes are billed to you by the provider; Rubuz charges for the platform. This is only in the higher "Kubernetes + VPS" plan.

What I'd check before committing

Price. There is no public price. Both plans say "Talk to us," and the FAQ says pricing depends on how many servers and clusters you run. For a solo dev comparing against a $5–20/month PaaS bill, get a number before you migrate anything.
Database list. Docs and product pages disagree (see above). Verify the one you need.
OS versions. It requires Ubuntu 26.04 LTS or Debian 13+. Older images you already run won't qualify; plan on a fresh server.
Region and provider fit. Any provider reachable over SSH works for VPS, but managed Kubernetes is limited to DigitalOcean, Vultr and Linode.
Compare with the open-source options. Coolify, Dokku, CapRover and Kamal cover overlapping ground for free if you're willing to run and maintain the tool yourself. Rubuz's case rests on being hosted (nothing to operate), rootless Podman, and the backup/restore-testing story.

Trying it

There's a 14-day free trial with no credit card. The cheapest honest test is: a fresh 2 GB VPS, one small repo with a health endpoint, one Postgres database, a test domain, and a backup to an R2 or B2 bucket, then deliberately break a deploy and restore onto a second fresh server.

👉 Start a Rubuz trial

If you try it before I do, I'd like to hear how the build times and the restore-onto-a-new-server flow hold up in practice.

Build a Q&A Bot over Your Company Docs with Pinecone + Claude (Python, SDK v10)

Simon — Wed, 04 Nov 2026 01:04:51 +0000

AI disclosure: This article was written with the help of AI and reviewed by me. Every API call in it was checked against the SDK source of pinecone==10.3.0 and anthropic==2.4.0, and prices and limits were taken from the official pages linked inline (as of November 2026). Treat the code as a starting point: run it against your own keys before relying on it.

What this solves

Once a company has enough internal docs, the same problems show up: new hires can't find the leave policy, nobody remembers which version of the expense rules is current, and wiki search only works if you guess the exact words the author used ("PTO" won't find the page that says "annual leave").

Retrieval-augmented generation (RAG) is the usual fix: split documents into chunks, embed them into a vector database, retrieve the most relevant chunks for each question, and have an LLM answer only from those chunks, with citations.

This walkthrough builds a command-line Q&A bot with Pinecone (managed vector database with hosted embedding and rerank models) and Claude (Anthropic's LLM, for answer generation):

Chunking
Embedding with Pinecone's integrated embedding (text is vectorized on write)
Creating a serverless index and upserting
Retrieval + reranking
Prompting Claude for a cited answer
The full script, dependencies and env vars
Cost estimates and common pitfalls

Versions used (latest on PyPI in November 2026):

Component	Version
Python	3.10+ (Pinecone SDK v10 supports 3.10–3.14)
`pinecone` (Pinecone Python SDK)	10.3.0, API version `2026-07`
`anthropic` (Anthropic Python SDK)	2.4.0
Claude model	`claude-sonnet-5-5`

The Pinecone SDK changed a lot in v10. Many tutorials still use the v5–v7 style (create_index_for_model, upsert_records, ServerlessSpec), which may fail or emit deprecation warnings. The pitfalls section maps old calls to new ones.

1. Architecture

         ┌──────────── Offline ingest ─────────────────┐
 .md/.txt ──> chunk ──> documents.upsert ──> Pinecone index
                              │              (llama-text-embed-v2
                              │               embeds on write)
         └─────────────────────────────────────────────┘

         ┌──────────── Online ask ─────────────────────┐
 question ──> documents.search (embed scoring, recall 12)
          ──> inference.rerank (bge-reranker-v2-m3, keep 4)
          ──> build prompt ──> Claude Messages API ──> answer with [n] citations
         └─────────────────────────────────────────────┘

Embedding, storage and reranking all happen on Pinecone's side, so you don't install PyTorch or host an embedding model. The trade-off: your text is processed by Pinecone's hosted models (more on that under data residency).

2. Setup

2.1 Pinecone API key

Create an account at pinecone.io, then create a key on the API Keys page of the console (https://app.pinecone.io). The free Starter plan is enough for everything here, but note its limits (as of November 2026, see Pricing and Rate limits):

Indexes can only be created in AWS us-east-1.
Up to 5 serverless indexes per project, 100 namespaces per index, 2 GB storage per organization.
1M read units, 2M write units and 1 GB egress per month.
5M tokens per month per hosted embedding model.
bge-reranker-v2-m3: 500 requests per month, 60 per minute. This runs out fast, so the script has a switch to turn reranking off.

2.2 Anthropic API key

Create a key in the Claude Console (https://platform.claude.com). Check Anthropic's supported countries list for your region first.

2.3 Dependencies

requirements.txt:

pinecone==10.3.0
anthropic==2.4.0

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The package is pinecone, not the old pinecone-client.

2.4 Environment variables

Variable	Required	Notes
`PINECONE_API_KEY`	yes	From the Pinecone console
`ANTHROPIC_API_KEY`	yes	From the Claude Console; `anthropic.Anthropic()` reads it by default
`PINECONE_INDEX`	no	Index name, default `company-docs`
`PINECONE_NAMESPACE`	no	Namespace, default `default-tenant`
`USE_RERANK`	no	`1` to rerank (default), `0` to skip and save Starter rerank quota

export PINECONE_API_KEY="your-pinecone-key"
export ANTHROPIC_API_KEY="your-anthropic-key"

Don't hard-code keys or commit them to Git.

3. Step 1: Chunking

Chunking is the first thing that makes or breaks RAG quality. Chunks that are too large mix several topics, so the embedding gets "averaged out" and you waste prompt tokens. Chunks that are too small lose context and the model gets half a sentence.

A simple strategy that works for Markdown or plain-text policy docs:

Split on blank lines into paragraphs, avoiding mid-paragraph breaks.
Pack adjacent paragraphs into chunks of at most 500 characters.
Hard-split any single paragraph longer than 500 characters with a sliding window.
Keep an 80-character overlap between chunks so an answer is less likely to fall across a boundary.

def split_text(text: str, size: int = 500, overlap: int = 80) -> list[str]:
    paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]
    chunks, buf = [], ""
    for p in paragraphs:
        if len(p) > size:
            if buf:
                chunks.append(buf)
                buf = ""
            step = size - overlap
            chunks.extend(p[i:i + size] for i in range(0, len(p), step))
            continue
        if len(buf) + len(p) + 1 <= size:
            buf = f"{buf}\n{p}" if buf else p
        else:
            chunks.append(buf)
            buf = (buf[-overlap:] + "\n" + p) if overlap else p
    if buf:
        chunks.append(buf)
    return chunks

This counts characters, not tokens, to avoid depending on a tokenizer. 500 characters of English is roughly 100–150 tokens, well under both limits that matter here:

llama-text-embed-v2: max 2048 tokens per input, truncated by default beyond that (Embedding models).
bge-reranker-v2-m3: query + document combined max 1024 tokens (Rerank results).

For finer strategies (by sentence, by token, by heading), see Pinecone's Chunking Strategies.

Each chunk carries metadata: source (relative path), category (top-level folder such as hr/ or finance/) and chunk_index. category is used for filtering, source for citations and for cleaning up on re-ingest.

4. Step 2: Choosing an embedding approach

Anthropic doesn't offer an embedding model (Embeddings), so you need one from somewhere else. Three common options:

Option	How	Pros	Cons
A. Pinecone integrated embedding (used here)	Declare `embed` on a text field in the index schema; write and query plain text	Least code, no dimension mismatches	Limited to Pinecone-hosted models; text is processed by those models
B. Pinecone Inference, called separately	`pc.inference.embed(...)` returns vectors you write to a `dense_vector` field	You own the vectors and can reuse them	You must keep the model and dimension consistent between write and query
C. Third-party or self-hosted model	Voyage AI (one of the providers Anthropic's docs point to), another cloud API, or an open-source model	Any model, can be fully private	Another service to run; you align dimensions and normalization yourself

Pinecone-hosted dense embedding models (docs as of November 2026):

Model	Dimensions	Max input	Notes
`llama-text-embed-v2`	1024 (default) / 2048 / 768 / 512 / 384	2048 tokens	Model page lists 26 supported languages
`multilingual-e5-large`	1024	507 tokens	Multilingual; good for short queries + medium passages
`text-embedding-3-small` / `-large`	up to 1536 / 3072, configurable	8192 tokens	OpenAI models processed via Microsoft Azure; document indexes only
`gemini-embedding-2`	128–3072	8192 tokens	Google model; not available on Starter

On the Standard plan, llama-text-embed-v2 is listed at $0.16 per million tokens and multilingual-e5-large at $0.08 (Pricing).

This walkthrough uses llama-text-embed-v2: multilingual, a roomier input limit than e5, and available on Starter. If your docs are mostly short Q&A pairs, try multilingual-e5-large: change the model name in the schema and rebuild the index.

5. Step 3: Creating a serverless index

In SDK v10 the recommended index type is the document index. You declare the scoring fields with SchemaBuilder; every other field you write is stored as metadata and indexed for filtering automatically. Adding an embed config to a string field makes Pinecone embed that field's text on write, and embed the query text at search time (Integrated embedding).

import os
from pinecone import Pinecone, SchemaBuilder

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])

schema = (
    SchemaBuilder()
    .add_string_field(
        name="chunk_text",
        embed={"model": "llama-text-embed-v2", "dimension": 1024},
    )
    .build()
)

if not pc.indexes.exists(name="company-docs"):
    pc.indexes.create(
        name="company-docs",
        schema=schema,
        deployment={"deployment_type": "managed", "cloud": "aws", "region": "us-east-1"},
    )

Notes:

deployment_type: "managed" means serverless. In v10, spec=ServerlessSpec(...) is deprecated (the SDK source says it's translated into deployment). If you omit deployment, the SDK defaults to a managed index in AWS us-east-1; it's spelled out here so you know where to change the region.
pc.indexes.create blocks until the index is ready by default. Pass timeout=-1 to return immediately.
Only declare scoring fields in the schema. Putting metadata like source or category in it returns a 400 at creation.
The schema can't be changed after creation, including the embedding model and dimension. Switching models means deleting and rebuilding the index.
An embed field also stores the original text and enables full-text search by default (store_text defaults to true), so search results can return chunk_text. Set store_text=False if you don't want the raw text stored in Pinecone, but then you'll have to look it up by ID in your own storage.

6. Step 4: Upserting documents

index = pc.index(name="company-docs")

documents = [
    {
        "_id": "3be9998a6b0d7b83#0",          # ASCII only
        "chunk_text": "Employees get 5 days of paid annual leave after one year of service...",
        "source": "hr/annual-leave.md",       # metadata, filterable automatically
        "category": "hr",
        "chunk_index": 0,
    },
    # ...
]

for i in range(0, len(documents), 96):
    index.documents.upsert(namespace="default-tenant", documents=documents[i:i + 96])

Key points:

Indexes with an embed field accept at most 96 documents per upsert (1000 for regular indexes), because each request is embedded in a single call.
Document _id must be 1–512 ASCII characters. Non-ASCII file names (accents, CJK) will fail SDK validation, so the script uses the first 16 hex chars of the path's SHA-1 plus the chunk number.
Each document (including metadata) is capped at 2 MB; the embed field text at 100 KB; metadata at 40 KB.
Indexing is asynchronous: a query right after a write may miss it. The official quickstart also sleeps a few seconds before querying.
usage.embed_total_tokens in the response tells you how many embedding tokens the write used, which helps reconcile costs.

Re-ingesting: if a file shrinks and produces fewer chunks, the old extra chunks linger. The script calls documents.delete(filter=...) by source before writing, to clear a file's old chunks.

7. Step 5: Retrieval + rerank

7.1 Vector recall

resp = index.documents.search(
    namespace="default-tenant",
    top_k=12,
    score_by=[{"type": "embed", "fields": ["chunk_text"], "query": "How much annual leave do I get?"}],
    include_fields=["chunk_text", "source", "chunk_index"],
    filter={"category": {"$eq": "hr"}},     # optional metadata filter
)
for m in resp.matches:
    print(m.id, m.score, m.get("source"))

With embed scoring, Pinecone embeds the query with the model fixed at index creation, so write and query always use the same model and dimension.
Without include_fields you only get _id and _score. This is a classic beginner trap: the search "works" but returns no text. ["*"] returns every field.
An embed clause must appear alone in score_by; it can't be combined with a BM25 text clause. For hybrid search, the docs suggest running both searches and merging with RRF (Integrated embedding).

7.2 Why rerank

Vector recall uses a bi-encoder: question and documents are embedded separately and compared by distance. It's fast but coarse. A rerank model (cross-encoder) scores the question and each document together, which is more accurate but slower. The usual pattern is to over-recall with vectors (12 here) and rerank down to the best few (4 here) for the prompt.

ranked = pc.inference.rerank(
    model="bge-reranker-v2-m3",
    query="How much annual leave do I get?",
    documents=[{"id": m.id, "chunk_text": m.get("chunk_text", "")} for m in resp.matches],
    rank_fields=["chunk_text"],
    top_n=4,
    return_documents=False,
    parameters={"truncate": "END"},
)
for r in ranked.data:
    print(r.index, r.score)   # r.index is the position in the documents you passed in

Model choice: Pinecone currently hosts three rerank models: bge-reranker-v2-m3 (multilingual, available on Starter), cohere-rerank-4-fast (Standard and above, processed via Azure, possibly outside the US), and pinecone-rerank-v0 (deprecated; don't use it in new projects).

bge-reranker-v2-m3 defaults to truncate: NONE, which errors on over-long input, so END is set explicitly.

Older tutorials often do recall + rerank in one call via index.search(..., rerank={...}). That's for legacy integrated indexes (created with create_index_for_model, using the Records API). With document indexes, rerank is a separate pc.inference.rerank call.

8. Step 6: Prompting Claude

Wrap each retrieved chunk in numbered XML tags so the model has anchors to cite:

SYSTEM_PROMPT = """You are an internal knowledge-base assistant. Answer employees' questions using ONLY the material inside <documents>.
Rules:
1. If the answer isn't in the material, say "I couldn't find this in the knowledge base." Don't make anything up.
2. After each key statement, cite the source as [n], where n matches <document index="n">.
3. If sources conflict, list the conflict instead of picking one.
4. Answer concisely."""

def build_user_message(question, hits):
    blocks = "\n".join(
        f'<document index="{i}" source="{h["source"]}">\n{h["text"]}\n</document>'
        for i, h in enumerate(hits, start=1)
    )
    return f"<documents>\n{blocks}\n</documents>\n\nQuestion: {question}"

Calling the Messages API:

import anthropic

client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY

response = client.beta.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=16000,
    system=SYSTEM_PROMPT,
    messages=[{"role": "user", "content": build_user_message(question, hits)}],
    output_config={"effort": "medium"},
    fallbacks="default",
    betas=["server-side-fallback-2026-07-01"],
)

if response.stop_reason == "refusal":
    print("The model declined to answer.")
else:
    print("".join(b.text for b in response.content if b.type == "text"))

What's different from older code:

response.content is a list of blocks, not necessarily a single text block. Claude Sonnet 5.5 uses adaptive thinking by default, so the response can include thinking blocks. Filter on type == "text" instead of reading content[0].text.
output_config.effort controls reasoning depth and token spend: low / medium / high / xhigh / max. Sonnet 5.5 defaults to high. Grounded Q&A rarely needs deep reasoning, so medium is used here; compare low and medium on your own question set.
Don't pass temperature. API version 2026-10-21, the default in Anthropic SDK 2.x, rejects any request containing temperature, top_p or top_k with a 400. Delete the temperature=0 that old code often has.
stop_reason == "refusal": Sonnet 5.5 has safety classifiers, and a refusal comes back as a normal response, not an exception. fallbacks="default" plus the beta header server-side-fallback-2026-07-01 lets the server retry a refused request on a recommended fallback model (Refusals and fallback). Because it's a beta, it goes through client.beta.messages.create. To skip the beta, drop the last two arguments and use client.messages.create; keep the refusal check either way.
max_tokens is a cap, not a prepayment; you're billed for actual output. Set it too low and answers get cut off.

9. The full script

Save as rag_bot.py:

"""
rag_bot.py - company-docs Q&A bot on Pinecone (integrated embedding + rerank) + Claude

Usage:
    python rag_bot.py ingest ./docs                        # chunk and upsert
    python rag_bot.py ask "How much annual leave do I get?"
    python rag_bot.py ask "expense process" --category finance   # search one category only

Versions at time of writing: pinecone==10.3.0, anthropic==2.4.0, python>=3.10
"""

import argparse
import hashlib
import os
import sys
import time
from pathlib import Path

import anthropic
from pinecone import Pinecone, SchemaBuilder
from pinecone.errors import NotFoundError

# ---------- config ----------
INDEX_NAME = os.environ.get("PINECONE_INDEX", "company-docs")
NAMESPACE = os.environ.get("PINECONE_NAMESPACE", "default-tenant")
EMBED_MODEL = "llama-text-embed-v2"      # Pinecone-hosted embedding model
EMBED_DIM = 1024                          # this model supports 384/512/768/1024/2048
RERANK_MODEL = "bge-reranker-v2-m3"      # Pinecone-hosted rerank model (multilingual)
USE_RERANK = os.environ.get("USE_RERANK", "1") == "1"
CLAUDE_MODEL = "claude-sonnet-5-5"

CHUNK_SIZE = 500        # target characters per chunk
CHUNK_OVERLAP = 80      # overlap between adjacent chunks
UPSERT_BATCH = 96       # max docs per upsert for indexes with an embed field
RECALL_TOP_K = 12       # vector recall size
FINAL_TOP_N = 4         # chunks sent to the prompt after rerank

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
claude = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY


# ---------- 1. chunking ----------
def split_text(text: str, size: int = CHUNK_SIZE, overlap: int = CHUNK_OVERLAP) -> list[str]:
    """Split on blank lines, pack paragraphs up to `size`, hard-split long paragraphs with overlap."""
    paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]
    chunks, buf = [], ""
    for p in paragraphs:
        if len(p) > size:                       # long paragraph: flush buffer, then sliding window
            if buf:
                chunks.append(buf)
                buf = ""
            step = size - overlap
            chunks.extend(p[i:i + size] for i in range(0, len(p), step))
            continue
        if len(buf) + len(p) + 1 <= size:
            buf = f"{buf}\n{p}" if buf else p
        else:
            chunks.append(buf)
            # carry the tail of the previous chunk to reduce answers cut in half
            buf = (buf[-overlap:] + "\n" + p) if overlap else p
    if buf:
        chunks.append(buf)
    return chunks


def load_documents(root: Path) -> list[dict]:
    """Read .md / .txt files under root and build the documents to upsert."""
    docs = []
    for path in sorted(root.rglob("*")):
        if path.suffix.lower() not in {".md", ".txt"} or not path.is_file():
            continue
        rel = path.relative_to(root).as_posix()
        text = path.read_text(encoding="utf-8")
        # _id must be ASCII; hash the path so non-ASCII file names work
        file_key = hashlib.sha1(rel.encode("utf-8")).hexdigest()[:16]
        category = rel.split("/")[0] if "/" in rel else "general"
        for i, chunk in enumerate(split_text(text)):
            docs.append({
                "_id": f"{file_key}#{i}",
                "chunk_text": chunk,        # Pinecone embeds this field
                "source": rel,              # the rest is metadata, filterable automatically
                "category": category,
                "chunk_index": i,
            })
    return docs


# ---------- 2. index ----------
def ensure_index() -> None:
    if pc.indexes.exists(name=INDEX_NAME):
        return
    schema = (
        SchemaBuilder()
        .add_string_field(
            name="chunk_text",
            embed={"model": EMBED_MODEL, "dimension": EMBED_DIM},
        )
        .build()
    )
    print(f"Creating index {INDEX_NAME} ...")
    pc.indexes.create(
        name=INDEX_NAME,
        schema=schema,
        # managed = serverless. The free Starter plan only allows aws / us-east-1
        deployment={"deployment_type": "managed", "cloud": "aws", "region": "us-east-1"},
    )  # blocks until the index is ready by default


# ---------- 3. ingest ----------
def ingest(root: Path) -> None:
    ensure_index()
    index = pc.index(name=INDEX_NAME)
    docs = load_documents(root)
    if not docs:
        print("No .md / .txt files found")
        return

    # on re-ingest, delete each file's old chunks first so none are left behind
    for source in sorted({d["source"] for d in docs}):
        try:
            index.documents.delete(namespace=NAMESPACE, filter={"source": {"$eq": source}})
        except NotFoundError:
            pass  # namespace doesn't exist yet (first ingest)

    for i in range(0, len(docs), UPSERT_BATCH):
        batch = docs[i:i + UPSERT_BATCH]
        resp = index.documents.upsert(namespace=NAMESPACE, documents=batch)
        tokens = resp.usage.embed_total_tokens if resp.usage else "?"
        print(f"Upserted {i + len(batch)}/{len(docs)} chunks, embedding tokens this batch: {tokens}")

    time.sleep(5)  # indexing is asynchronous; wait a moment before querying
    print("Ingest complete")


# ---------- 4. retrieve + rerank ----------
def retrieve(question: str, category: str | None = None) -> list[dict]:
    index = pc.index(name=INDEX_NAME)
    resp = index.documents.search(
        namespace=NAMESPACE,
        top_k=RECALL_TOP_K,
        score_by=[{"type": "embed", "fields": ["chunk_text"], "query": question}],
        include_fields=["chunk_text", "source", "chunk_index"],
        filter={"category": {"$eq": category}} if category else None,
    )
    hits = [
        {
            "id": m.id,
            "score": m.score,
            "text": m.get("chunk_text", ""),
            "source": m.get("source", ""),
        }
        for m in resp.matches
    ]
    if not hits or not USE_RERANK:
        return hits[:FINAL_TOP_N]

    ranked = pc.inference.rerank(
        model=RERANK_MODEL,
        query=question,
        documents=[{"id": h["id"], "chunk_text": h["text"]} for h in hits],
        rank_fields=["chunk_text"],
        top_n=FINAL_TOP_N,
        return_documents=False,
        parameters={"truncate": "END"},
    )
    # r.index is the document's position in the list we passed in
    return [dict(hits[r.index], score=r.score) for r in ranked.data]


# ---------- 5. prompt Claude ----------
SYSTEM_PROMPT = """You are an internal knowledge-base assistant. Answer employees' questions using ONLY the material inside <documents>.
Rules:
1. If the answer isn't in the material, say "I couldn't find this in the knowledge base." Don't make anything up.
2. After each key statement, cite the source as [n], where n matches <document index="n">.
3. If sources conflict, list the conflict instead of picking one.
4. Answer concisely."""


def build_user_message(question: str, hits: list[dict]) -> str:
    blocks = "\n".join(
        f'<document index="{i}" source="{h["source"]}">\n{h["text"]}\n</document>'
        for i, h in enumerate(hits, start=1)
    )
    return f"<documents>\n{blocks}\n</documents>\n\nQuestion: {question}"


def answer(question: str, category: str | None = None) -> None:
    hits = retrieve(question, category)
    if not hits:
        print("Nothing was retrieved from the knowledge base.")
        return

    response = claude.beta.messages.create(
        model=CLAUDE_MODEL,
        max_tokens=16000,
        system=SYSTEM_PROMPT,
        messages=[{"role": "user", "content": build_user_message(question, hits)}],
        output_config={"effort": "medium"},
        # if a safety classifier refuses, the server retries on a recommended model (beta)
        fallbacks="default",
        betas=["server-side-fallback-2026-07-01"],
    )

    if response.stop_reason == "refusal":
        print("The model declined to answer.")
        return

    text = "".join(b.text for b in response.content if b.type == "text")
    print(text)
    print("\n---- Sources ----")
    for i, h in enumerate(hits, start=1):
        score = f"{h['score']:.4f}" if h["score"] is not None else "n/a"
        print(f"[{i}] {h['source']}  (score={score})")
    u = response.usage
    print(f"\n(model={response.model}, input_tokens={u.input_tokens}, output_tokens={u.output_tokens})")


# ---------- entry point ----------
def main() -> None:
    parser = argparse.ArgumentParser(description="Pinecone + Claude docs Q&A")
    sub = parser.add_subparsers(dest="cmd", required=True)
    p_ingest = sub.add_parser("ingest")
    p_ingest.add_argument("path", type=Path)
    p_ask = sub.add_parser("ask")
    p_ask.add_argument("question")
    p_ask.add_argument("--category", default=None)
    args = parser.parse_args()

    if args.cmd == "ingest":
        ingest(args.path)
    else:
        answer(args.question, args.category)


if __name__ == "__main__":
    try:
        main()
    except anthropic.APIStatusError as e:
        sys.exit(f"Claude API error {e.status_code}: {e.message}")

Running it

Prepare a folder whose top-level subfolders act as categories:

docs/
├── hr/
│   ├── annual-leave.md
│   └── onboarding.md
└── finance/
    └── expenses.md

python rag_bot.py ingest ./docs
python rag_bot.py ask "Do I get annual leave in my first year?"
python rag_bot.py ask "What's the hotel limit on business trips?" --category finance
USE_RERANK=0 python rag_bot.py ask "How far ahead do I need to request leave?"   # no rerank quota used

The output shows the answer, the source file behind each [n] with its score, and the token usage for the call so you can check it against your bill.

10. Estimating cost

These are list-price estimates, not a measured bill; your console usage is the source of truth.

Pinecone (as of November 2026, Pricing):

Plan	Price	Notes
Starter	Free	Limits in 2.1; over the limit requests get 429, never billed
Builder	$20/month flat	Higher limits; overages are also blocked, not billed
Standard	$50/month minimum, then usage	Storage $0.33/GB/month; reads $16–18 per million RU; writes $4–4.5 per million WU (varies by cloud/region); `llama-text-embed-v2` $0.16 per million tokens; `bge-reranker-v2-m3` $2 per 1,000 requests; egress $0.10/GB (100 GB/month included)
Enterprise	$500/month minimum	99.95% uptime SLA

An internal knowledge base of a few hundred policy docs usually fits within Starter's 2 GB and 5M embedding tokens a month. The first limit you'll hit is usually 500 reranks a month. Either disable rerank on Starter, call it only when recall scores are too close to separate, or upgrade. Read/write unit accounting is detailed in Understanding cost.

Claude (as of November 2026, Pricing): claude-sonnet-5-5 is $2 per million input tokens, $10 per million output tokens, and $0.10 per million cache-hit tokens.

Rough math for one question with these settings: four chunks plus system prompt and question is a few thousand input tokens; the answer is a few hundred. Input cost is well under a cent; output cost depends on answer length, and thinking tokens are billed as output, which is why effort is worth tuning. For real numbers, multiply the input_tokens / output_tokens the script prints by the unit prices.

Ways to save:

Bigger top_k and FINAL_TOP_N aren't automatically better: 10 chunks instead of 4 is more than double the input tokens.
The system prompt never changes, so at volume consider Claude prompt caching. There's a minimum cacheable length, so a very short system prompt won't benefit; see the docs.
Use Pinecone namespaces per department or tenant; queries only scan their namespace, which also cuts read units.

11. Common pitfalls

1. Dimension mismatch

If you generate vectors yourself (options B/C), the index dimension must match the model output. Writing with model A and querying with model B gives essentially random results even if the dimensions happen to match. Using an embed field with embed scoring avoids this structurally, because Pinecone uses the model fixed at creation for both. Remember: the schema and embedding config can't change after creation; a new model means a new index and a full re-ingest.

With option B, the quickstart pattern is pc.inference.embed(model=..., inputs=[...], parameters={"input_type": "passage"}), written into a field declared with add_dense_vector_field, and input_type="query" at search time. Omitting or swapping input_type won't error, but retrieval quality drops.

2. Wrong namespace: "successful" query, empty results

If the write and query namespaces differ, you get no error, just empty results. The script keeps the namespace in one env var. Namespaces are good for multi-tenant isolation (one per department or customer), but Starter caps an index at 100 namespaces, so many tenants need a higher plan or metadata filtering instead.

3. Metadata filtering rules

Any field not declared in the schema is metadata and is indexed for filtering on write, supporting $eq, $in, $gte and friends.
Scoring fields declared in the schema (like chunk_text) can't use comparison operators like $eq (400 error); only text-match operators like $match_phrase / $match_all / $match_any.
Metadata is capped at 40 KB per document; don't stuff the full source text into it.
Keep each field's type consistent, e.g. chunk_index always an integer.

Metadata filters are not access control. If finance policies should only be visible to finance staff, the server must add the filter based on the logged-in user; never trust a category sent from the client. For stricter isolation, use separate namespaces or even separate indexes.

4. Chunk granularity

Too large: input beyond the embedding model's limit is silently truncated (default truncate=END), so the tail never makes it into the vector.
Too small: the search hits, but the model gets half a sentence.
For policy docs, prefix each chunk with the doc or section title, e.g. "Travel Expense Policy, Section 3, Lodging: ...". It helps retrieval noticeably and costs almost nothing.
Tables and scanned PDFs need text extraction first; that's outside Pinecone's job.

5. Document IDs and batch size

IDs must be ASCII; convert non-ASCII file names.
Indexes with an embed field: max 96 documents per upsert.
Indexes with an embed field don't support bulk import from object storage; use upsert / update only.

6. Not found right after writing

Indexing is asynchronous, so there's a short eventual-consistency window. A few seconds of sleep is enough in an ingest script; if your app needs read-after-write, handle it with a retry or a UI hint.

7. Mixing old and new SDK styles

What old tutorials show	In v10
`pip install pinecone-client`	Package renamed to `pinecone`
`pc.create_index(..., spec=ServerlessSpec(...))`	`spec` deprecated; use `pc.indexes.create(..., deployment={...})`
`pc.create_index_for_model(...)` + `index.upsert_records(...)` + `index.search(..., rerank=...)`	Creates a "legacy integrated index" on the old Records API; still works, but no text updates or import
`from pinecone import ...preview...`	`pinecone.preview` removed in v10; document indexes and `SchemaBuilder` import from `pinecone` directly
Testing locally with Pinecone Local	Pinecone Local reached end of life in 2026 (Release notes)

8. Claude-side pitfalls

Passing temperature returns 400 (SDK 2.x / API version 2026-10-21).
Don't prefill an assistant message at the end of messages to steer formatting; current models don't support prefill and return 400. Use output_config.format for structured output.
Filter text on type == "text".
Handle stop_reason == "refusal" and truncation (stop_reason == "max_tokens").

9. Data residency and compliance

Check these before rolling this out inside a company:

Starter indexes live only in AWS US East. Other regions (including Singapore ap-southeast-1 and Europe) need a paid plan (Cloud regions).
Hosted embedding and rerank models have their own processing locations; some run in global, possibly outside the US (Processing locations).
Pinecone's site says GDPR and ISO 27001 coverage doesn't apply to Starter, and HIPAA needs a Standard add-on or Enterprise. For stricter needs, look at Enterprise BYOC (deployed in your own cloud account).
Calls to the Claude API also send your text to Anthropic; check the supported-countries list and your company's data policies.

If your docs contain personal data or sensitive business information, get legal and security review first. If data can't leave your environment, this managed stack isn't the right fit; look at self-hostable options like pgvector or Milvus with a model you can run or contract for in-region.

12. Summary

Chunk: pack paragraphs, hard-split long ones, keep overlap; attach source and category metadata.
Embed + write: declare embed on a string field of a document index, write plain text, let Pinecone embed it with llama-text-embed-v2; at most 96 per batch.
Retrieve: documents.search with embed scoring recalls 12, optionally filtered by metadata.
Rerank: pc.inference.rerank with bge-reranker-v2-m3 keeps the best 4.
Generate: numbered chunks go into <documents> tags; Claude Sonnet 5.5 answers only from them, with citations.

The upside is very little code and no models or databases to operate, which makes it a good way to test whether "Q&A over company docs" is worth building. The downsides are just as clear: Pinecone is a closed managed service and your data lives in its cloud; Starter's rerank and region limits are tight; and costs grow linearly with data and query volume, so reconcile your bill regularly.

Where to go next:

Prefix chunks with titles, or chunk by Markdown heading.
Hybrid search: enable full-text search on the same field and merge keyword and vector results with RRF; it helps with exact terms like product codes or policy numbers.
Build a small eval set from real questions, track "answer correct" and "citation correct", then tune CHUNK_SIZE, top_k and effort.
Hook it into Slack or Teams as a bot, with server-side filtering by user identity.

Pinecone and Anthropic iterate quickly; check versions before running, and where this article and the official docs disagree, trust the docs.

References

Pinecone, Ingest your own files: https://docs.pinecone.io/guides/get-started/quickstart/ingest-files.md
Pinecone, Create an index: https://docs.pinecone.io/guides/index-data/create-an-index.md
Pinecone, Integrated embedding: https://docs.pinecone.io/guides/search/integrated-embedding.md
Pinecone, Index configuration (regions, embedding models): https://docs.pinecone.io/guides/index-data/index-configuration.md
Pinecone, Rerank results: https://docs.pinecone.io/guides/search/rerank-results.md
Pinecone, Rate limits: https://docs.pinecone.io/reference/api/database-limits/rate-limits.md
Pinecone, Object limits: https://docs.pinecone.io/reference/api/database-limits/object-limits.md
Pinecone, Understanding cost: https://docs.pinecone.io/guides/manage-cost/understanding-cost.md
Pinecone, Processing locations: https://docs.pinecone.io/guides/index-data/processing-locations.md
Pinecone, 2026 release notes: https://docs.pinecone.io/release-notes/2026.md
Pinecone, Pricing: https://www.pinecone.io/pricing/
Pinecone Python SDK (PyPI): https://pypi.org/project/pinecone/
Claude, Pricing: https://platform.claude.com/docs/en/about-claude/pricing
Claude, Refusals and fallback: https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback
Claude, Embeddings: https://platform.claude.com/docs/en/build-with-claude/embeddings
Anthropic Python SDK: https://github.com/anthropics/anthropic-sdk-python
Anthropic supported countries: https://www.anthropic.com/supported-countries